Archives

Gadgets

Startups look beyond lidar for autonomous vehicle perception

Last CES was a time of reckoning for lidar companies, many of which were cratering due to a lack of demand from a (still) non-existent autonomous vehicle industry. The few that excelled did so by specializing, and this year the trend has pushed beyond lidar, with new sensing and imaging methods pushing to both compete with and complement the laser-based tech.

Lidar pushed ahead of traditional cameras because it could do things they couldn’t — and now some companies are pushing to do the same with tech that’s a little less exotic.

A good example of addressing the problem or perception by different means is Eye Net’s vehicle-to-x tracking platform. This is one of those techs that’s been talked about in the context of 5G (admittedly still somewhat exotic), which for all the hype really does enable short-distance, low-latency applications that could be life-savers.

Eye Net provides collision warnings between vehicles equipped with its tech, whether they have cameras or other sensing tech equipped or not. The example they provide is a car driving through a parking lot, unaware that a person on one of those horribly unsafe electric scooters is moving perpendicular to it ahead, about to zoom into its path but totally obscured by parked cars. Eye Net’s sensors detect the position of the devices on both vehicles and send warnings in time for either or both to brake.

CG illustration of a bicyclist and car being warned of an imminent collision.

Image Credits: Eye Net

They’re not the only ones attempting something like this, but they hope that by providing a sort of white-label solution, a good size network can be built relatively easily, instead of having none, and then all VWs equipped, and then some Fords and some e-bikes, and so on.

But vision is still going to be a major part of how vehicles navigate, and advances are being made on multiple fronts.

Brightway Vision, for instance, addresses the issue of normal RGB cameras having limited visibility in many real-world conditions by going multispectral. In addition to ordinary visible-light imagery, the company’s camera is mated to a near-infrared beamer that scans the road ahead at set distance intervals many times a second.

CG illustration of a camera using infrared to see further ahead at night.

Image Credits: Brightway Vision

The idea is that if the main camera can’t see 100 feet out because of fog, the NIR imagery will still catch any obstacles or road features when it scans that “slice” in its regular sweep of the incoming area. It combines the benefits of traditional cameras with those of IR ones, but manages to avoid the shortcomings of both. The pitch is that there’s no reason to use a normal camera when you can use one of these, which does the same job better and may even allow another sensor to be cut out.

Foresight Automotive also uses multispectral imagery in its cameras (chances are hardly any vehicle camera will be limited to visible spectrum in a few years), dipping into thermal via a partnership with FLIR, but what it’s really selling is something else.

To provide 360-degree (or close) coverage, generally multiple cameras are required. But where those cameras go differs on a compact sedan versus an SUV from the same manufacturer — let alone on an autonomous freight vehicle. Because those cameras have to work together, they need to be perfectly calibrated, aware of the exact position of the others, so they know, for example, that they’re both looking at the same tree or bicyclist and not two identical ones.

Image showing Foresight cameras being attached magnetically to a car's body.

Image Credits: Foresight Automotive

Foresight’s advance is to simplify the calibration stage, so a manufacturer or designer or test platform doesn’t need to be laboriously re-tested and certified every time the cameras need to be moved half an inch in one direction or the other. The Foresight demo shows them sticking the cameras on the roof of the car seconds before driving it.

It has parallels to another startup called Nodar that also relies on stereoscopic cameras, but takes a different approach. The technique of deriving depth from binocular triangulation, as the company points out, goes back decades, or millions of years if you count our own vision system, which works in a similar ways. The limitation that has held this approach back isn’t that optical cameras fundamentally can’t provide the depth information needed by an autonomous vehicle, but that they can’t be trusted to remain calibrated.

Nodar shows that its paired stereo cameras don’t even need to be mounted to the main mass of the car, which would reduce jitter and fractional mismatches between the cameras’ views. Attached to the rear view mirrors, their “Hammerhead” camera setup has a wide stance (like the shark’s), which provides improved accuracy because of the larger disparity between the cameras. Since distance is determined by the differences between the two images, there’s no need for object recognition or complex machine learning to say “this is a shape, probably a car, probably about this big, which means it’s probably about this far away” as you might with a single camera solution.

Image Credits: Nodar

The industry has already shown that camera arrays do well in harsh weather conditions, just as human eyes do,” said Nodar COO and co-founder Brad Rosen. “For example, engineers at Daimler have published results showing that current stereoscopic approaches provide significantly more stable depth estimates than monocular methods and LiDAR completion in adverse weather. The beauty of our approach is that the hardware we use is available today, in automotive-grade, and with many choices for manufacturers and distributors.”

Indeed, a major strike against lidar has been the cost of the unit — even “inexpensive” ones tend to be orders of magnitude more expensive than ordinary cameras, something that adds up very quickly. But team lidar hasn’t been standing still either.

Sense Photonics came onto the scene with a new approach that seemed to combine the best of both worlds: a relatively cheap and simple flash lidar (as opposed to spinning or scanning, which tend to add complexity) mated to a traditional camera so that the two see versions of the same image, allowing them to work together in identifying objects and establishing distances.

Since its debut in 2019 Sense has refined its tech for production and beyond. The latest advance is custom hardware that has enabled it to image objects out to 200 meters — generally considered on the far end both for lidar and traditional cameras.

“In the past, we have sourced an off-the-shelf detector to pair with our laser source (Sense Illuminator). However, our 2 years of in-house detector development has now completed and is a huge success, which allows us to build short-range and long-range automotive products,” said CEO Shauna McIntyre.

“Sense has created ‘building blocks’ for a camera-like LiDAR design that can be paired with different sets of optics to achieve different FOV, range, resolution, etc,” she continued. “And we’ve done so in a very simple design that can actually be manufactured in large volumes. You can think of our architecture like a DSLR camera where you have the ‘base camera’ and can pair it with a macro lens, zoom lens, fisheye lens, etc. to achieve different functions.”

One thing all the companies seemed to agree on is that no single sensing modality will dominate the industry from top to bottom. Leaving aside that the needs of a fully autonomous (i.e. level 4-5) vehicle has very different needs from a driver assist system, the field moves too quickly for any one approach to remain on top for long.

“AV companies cannot succeed if the public is not convinced that their platform is safe and the safety margins only increase with redundant sensor modalities operating at different wavelengths,” said McIntyre.

Whether that means visible light, near-infrared, thermal imaging, radar, lidar, or as we’ve seen here, some combination of two or three of these, it’s clear the market will continue to favor differentiation — though as with the boom-bust cycle seen in the lidar industry a few years back, it’s also a warning that consolidation won’t be far behind.

Apple said to be planning new 14- and 16-inch MacBook Pros with MagSafe and Apple processors

Apple has planned new upgraded MacBook Pros for launch “later this year” according to a new report from Bloomberg. These new models would come in both 14-inch and 16-inch sizes, with new and improved Apple Silicon processors like those that Apple debuted on the new MacBook Air and 13-inch MacBook Pro model late last year. They would also see the return of Apple’s MagSafe charger, a magnetic dedicated charging port that would replace USB-C for power, and they could potentially do away with the Touch Bar, the small strip of OLED display built in to the keyboard on modern MacBook Pros.

Bloomberg’s report suggests that these MacBook Pro models will have processors with more cores and better graphics capabilities than the existing M1 chips that power Apple’s current notebooks with in-house silicon, and that they’ll also have displays with brighter panels that offer higher contrast. Physically, they’ll resemble existing notebooks, according to the report’s sources, but they’ll see the return of MagSafe, the dedicated magnetic charging interface that Apple used prior to switching power delivery over to USB-C on its laptops.

MagSafe had the advantage of easily disconnecting in case of anyone accidentally tripping across the power cord while plugged in, without yanking the computer with it. It also meant that it kept all data ports free for accessories. Bloomberg says that the revitalized MagSafe for new notebooks will also offer faster charging vs. USB-C, in addition to those other benefits.

As for the Touch Bar, it has been a topic of debate since its introduction. Pro users in particular seem to dislike the interface option, especially because it replaces a row of dedicated physical keys that could be useful in professional workflows. The report claims that Apple has “tested versions that remove the Touch Bar,” so it seems less clear that Apple will finally unring that particular bell, but I personally know a lot of people who would be excited if that does come to pass.

Finally, Bloomberg says Apple is also planning a new redesigned MacBook Air. That was updated most recently just a couple of months ago, and the report says it’ll only follow “long after” these new MacBook Pros, so it seems unlikely to arrive in 2021.

Thimble teaches kids STEM skills with robotics kits combined with live Zoom classes

Parents with kids stuck learning at home during the pandemic have had to look for alternative activities to promote the hands-on learning experiences kids are missing out on due to attending class virtually. The New York-based educational technology startup Thimble aims to help address this problem by offering a subscription service for STEM-based projects that allow kids to make robotics, electronics and other tech using a combination of kits shipped to the home and live online instruction.

Thimble began back in 2016 as Kickstarter project when it raised $300,000 in 45 days to develop its STEM-based robotics and programming kits. The next year, it then began selling its kits to schools, largely in New York, for use in the classroom or in after-school programs. Over the years that followed, Thimble scaled its customer base to include around 250 schools across New York, Pennsylvania, and California, who would buy the kits and gain access to teacher training.

But the COVID-19 pandemic changed the course of Thimble’s business.

“A lot of schools were in panic mode. They were not sure what was happening, and so their spending was frozen for some time,” explains Thimble co-founder and CEO Oscar Pedroso, whose background is in education. “Even our top customers that I would call, they would just give [say], ‘hey, this is not a good time. We think we’re going to be closing schools down.”

Pedroso realized that the company would have to quickly pivot to begin selling directly to parents instead.

Image Credits: Thimble

Around April, it made the shift — effectively entering the B2C market for the first time.

The company today offers parents a subscription that allows them to receive up to 15 different STEM-focused project kits and a curriculum that includes live instruction from an educator. One kit is shipped out over the course of three months, though an accelerated program is available that ships with more frequency.

The first kit is basic electronics where kids learn how to build simple circuits, like a doorbell, kitchen timer and a music composer, for example. The kit is designed so kids can experience “quick wins” to keep their attention and whet their appetite for more projects. This leads into future kits like those offering a Wi-Fi robot, a little drone, an LED compass that lights up, and a synthesizer that lets kids become their own D.J.

Image Credits: Thimble

While any family can use the kits to help kids experience hands-on electronics and robotics, Pedroso says that about 70% of subscribers are those where the child already has a knack for doing these sorts of projects. The remaining 30% are those where the parents are looking to introduce the concepts of robotics and programming, to see if the kids show an interest. Around 40% of the students are girls.

The subscription is more expensive than some DIY projects at $59.99/per month (or $47.99/mo if paid annually), but this is because it includes live instruction in the form of weekly 1-hour Zoom classes. Thimble has part-time employees who are not just able to understand teach the material, but can do so in a way that appeals to children — by being passionate, energetic and capable of jumping in to help if they sense a child is having an issue or getting frustrated. Two of the five teachers are women. One instructor is bilingual and teaches some classes in Spanish.

During class, one teacher instructs while a second helps moderate the chat room and answer the questions that kids ask in there.

The live classes will have around 15-20 students each, but Thimble additionally offers a package for small groups that reduces class size. These could be used by homeschool “pods” or other groups.

Image Credits: Thimble

“We started hearing from pods and then micro-schools,” notes Pedroso. “Those were parents who were connected to other parents, and wanted their kids to be part of the same class. They generally required a little bit more attention and wanted some things a little more customized,” he added.

These subscriptions are more expensive at $250/month, but the cost is shared among the group of parents, which brings the price down on per-household basis. Around 10% of the total customer base is on this plan, as most customers are individual families.

Thimble also works with several community programs and nonprofits in select markets that help to subsidize the cost of the kits to make the subscriptions more affordable. These are announced, as available, through schools, newsletters, and other marketing efforts.

Since pivoting to subscriptions, Thimble has re-established a customer base and now has 1,110 paid customers. Some, however, are grandfathered in to an earlier price point, so Thimble needs to scale the business further.

In addition to the Kickstarter, Thimble has raised funds and worked on the business over the year with the help of multiple accelerators, including LearnLaunch in Boston, Halcyon in D.C., and Telluride Venture Accelerator in Colorado.

The startup, co-founded by Joel Cilli in Pittsburgh, is now around 60% closed on its seed round of $1 million, but isn’t announcing details of that at this time.