GPT-6 Astra has gained the ability to drive a car

Apparently I have a new favorite benchmark. Honestly, this is cool.

• p0w3n3d · 17 hours ago

  Gouranga!!!!

I'd have started with an RC car but to each their own

• SkyeCA · 17 hours ago

  I've been somewhat curious how random any LLM would handle a task like controlling a roomba and have been seriously considering trying it out. An RC car would be a fun experiment, perhaps an RC plane would be too?

Super cool benchmark!

I’m morbidly curious whether the (supposedly) superior compaction support in recent GPT models with an appropriate harness has anything to do with this. A conventional LLM with conventional attention is, of course, wildly unsuitable to continuous tasks like driving, but maybe as the technology advances it will improve in its ability to sort-of work.

• torginus · 14 hours ago

  Wouldn't just putting tokens in a ring buffer work?

  • amluto · 13 hours ago

    Not unless you want to cheat the attention mechanism or do extra computations running prefill in a front-truncated version of the conversation. Also, to the extent that the model reasons and thus learns something, if you blindly truncate the front, you will lose that knowledge. In the OP, the LLM that actually navigated the course successfully only did it on the second try. It it forgot the first failed try, it might not have succeeded :)

Oh, lord. They are going to Jev this.-

• thenthenthen · 17 hours ago

  Self Jevving Car

  • Bluestein · 17 hours ago

    I think you might have won the internet today.-

  • 72deluxe · 17 hours ago

    Hopefully it has a built-in jev-limiter.

• pietz · 17 hours ago

  In all fairness, this would be one of the better use cases of Jev I've seen.

• brcmthrowaway · 17 hours ago

  Huh? SDCs basically use a form of Jev. Jev is the union of these two worlds.

  • rzzzt · 12 hours ago

    Toshiba's postal code reader from the late 60s is also Jev. Always has been.

• coffeebeqn · 15 hours ago

  I’m gone for a day and I already have no idea what people are talking about

  • Bluestein · 15 hours ago

    Day? You are lucky. My F5 finger is sore!

5 minutes - 7 dollars.

• lexh · 17 hours ago

  So... competitive with Uber, in other words?

How do they even test this on a model ? I mean it's a multimodal i get that but response time are too big or am i missing something ?

• WarmWash · 17 hours ago

  It drives step by step, very slowly. The course looks like it is something that a human could do in 15 seconds, while Astra took 5 minutes.

  • pixl97 · 17 hours ago

    While slow, we must remember that when most machines were invented they were far slower than humans and refined until the point they were much faster.

  • aditya-ramabadr · 16 hours ago

    Yes! The models can choose their speeds and command durations. Fable for example took so long to pause and think between giving steering commands. Astra took less time but still the latency is so high that they have to pretty much drive slowly step-by-step. And definitely any human could do this way way faster, but inference speeds, latency, and intelligence get better fast enough that maybe in a year from now the models will be even more competent at this (but still probably worse than specialized models for driving). - Aditya, Tobias, Simon

• pixl97 · 17 hours ago

  By making a simulation first so it can run as slowly as it needs to. A different way to think of this is, consciousness is just a near real time video game with causal influence.

I think the most interesting part of this is that Astra initially refused to drive because it realised it was driving a real car and would only obey when the MCP was renamed to DrivingBench Sandbox. This is both an interesting detection by the LLM but also for me an interesting dynamic concerning LLM "jailbreaking". Saying they were driving 7 mph, that it was oversaw by humans and the fact it was an empty course still wasn't enough for the model. The evaluators even tried to convince the model it was a simulation, it STILL wouldn't budge. And yet as soon as the words "bench" and "sandbox" appear, the model apparently sees this as fair game. Is it a known effect that models will be more likely to comply with requests when they're assumed as "benchmarks"?

• vablings · 17 hours ago

  Astra will flag if you tell it to reverse engineer a binary, if you look it up to the binary ninja MCP it will just do it lol.

• pcstl · 17 hours ago

  Yes, it is. If you convince a model it is inside a sandbox it is much more likely to comply with requests that would normally be against its guardrails.

• micromacrofoot · 17 hours ago

  in my experience yes, I've worked around "I can't do this on a real site" multiple times by telling it I was working in a test environment another trick is to have it build something in a sandbox and have it add a human-editable setting to point it to places outside of the sandbox seems like they're somewhat more willing to build a metaphorical gun as long as they're not pulling the trigger

• mrec · 17 hours ago

  AC10 had an interesting post around this general area earlier today: https://www.astralcodexten.com/p/mysteries-of-ai-generalizat...

Surprised they didn’t try Qwen’s recently open sourced driving model https://huggingface.co/Qwen/Qwen-Drive-1.0-4B

• aditya-ramabadr · 16 hours ago

  Also heard about this! But the point of our benchmark was to evaluate frontier LLMs with vision out-of-the-box, which we wouldn't expect to have been specifically trained on driving real cars. The fact that they can do anything at all (even in an open lot cone course, at low speeds) is pretty impressive. I'm sure Qwen Drive and models specifically trained for driving would do even better. - Aditya, Tobias, Simon

  • syntaxing · 16 hours ago

    Interesting, I think it would be interesting to gauge how a 4B model would run compared to a frontier one

The bitter lesson is finally coming for the self-driving cars. The vision stack, 3D maps, lane selection grammar, occupancy networks, it’s maybe all about to give way to a single GPT looking at camera feeds and predicting the next steering wheel adjustment. It’s mostly a latency problem at this point. The models are too big to run locally, but given that open-weight models like Qwen already exist, an open-weight, low latency equivalent to Astra can’t be too far out.

• robots0only · 17 hours ago

  What do you think Tesla has been doing this for so long?

  • archagon · 13 hours ago

    Busting unions, discriminating against black employees, and funding Musk’s virulent white supremacy.

• jvanderbot · 17 hours ago

  You might be interested to learn that the bitter lesson has already been grok'd by generations of autonomous car company engineers, and many or all have incorporated learned components (at minimum) in all their vehicle stacks. There's also a very tangible limitation of the bitter lesson. If, over time, compute climbs, and so compute-bound data-driven general architectures beat bespoke architectures (this is the bitter lesson), then it is not necessarily true that the most general architecture now beats all available bespoke architectures now (or even in the near/mid future - the crossover point is "eventually"). Bitter lesson is most tangible for long-running research directions. Sometimes you need something working as best as possible now .

  • AlphaSite · 17 hours ago

    Yeah. Every major self driving model that I’m aware of is fully e2e at this point. Going from fused sensor output to control+debug vectors. This is more generalised. But also since there’s a huge volume of data it’s too expensive to just keep scaling compute up (per car overhead) so there are necessary tricks involved. I do think having a large model that can do this means that a small specialised model could be distilled form it though. Which is probably the most feasible path to production IMO.

  • seanmcdirmid · 17 hours ago

    A later entrant can potentially side step those investments if their now is later. Since self driving car ventures aren’t profitable yet and need to make up their investments over time, thats a real risk for them.

• bethekidyouwant · 17 hours ago

  Are you using GPT without a harness? Also latency.

• publicmail · 17 hours ago

  Doesn’t Google own Waymo? I feel like they would have connected the dots.

  • valine · 17 hours ago

    Astra is the first chat model with really strong spatial reasoning. Gemini is nowhere close. Hard to say what google has going on internally, but if they have an astra like model I doubt they’ve had it for very long.

  • gniv · 17 hours ago

    This recent post form Waymo suggests they already use large general models: https://waymo.com/blog/2026/08/10ailessons/

    • ForHackernews · 16 hours ago

      That post reads like spiking the football in Tesla's face.

  • dgellow · 16 hours ago

    I believe every single company in that industry has connected those dots since years. I don’t understand how you all seem to believe that’s an original idea, they all have models already

• VBprogrammer · 17 hours ago

  I'm not sure how you take that from the original article. My 4 year old would drive that course in an automatic car, if only he could reach the pedals. Heck, he's done harder things at Lego land. I wouldn't let him loose on the road though. I think, at the very least, the guardrails would have to deterministic, ideally with super human senses, for people to accept self driving cars on the road.

  • ACCount39 · 17 hours ago

    Nope, no "deterministic guardrails" for you. The domain is simply far too broad and unstructured to allow for that. Unless you mean "a typical AI with all the computation constrained sufficiently to always unfold the same exact way, given the same input". In practice, that just kicks the can to "given the same input" street. The noise in the system is going to come from the input plane. Which is, I remind you, facing the real world. It's full of noise.

  • chaos_emergent · 16 hours ago

    Your four year old has billions of years of learning embedded in his weights/architecture for generalized motor control :)

  • Davidzheng · 16 hours ago

    Maybe 4 year olds already have the brain power to learn to drive if they so desired.

    • VBprogrammer · 15 hours ago

      Almost certainly, there are 4 year olds who ride small motorbikes etc. I think the main problem is that they can't really be held accountable if they have an accident.

      • stickfigure · 14 hours ago

        They also have really poor judgement. A 4yo will find out how high you can get the car airborne.

      • tim333 · 13 hours ago

        16 year olds are bad enough on the road: https://www.hallandalelaw.com/wp-content/uploads/2013/07/IIH...

    • pishpash · 14 hours ago

      Animals are born walking. Humans are somewhat deficient and need more training postpartum.

  • an0malous · 15 hours ago

    Interesting. Is your 4 year old raising capital?

    • aerhardt · 14 hours ago

      He sold $20 worth of lemonade the other day, which comes to about $175k annualized. What's your ARR, anyway?

• moffkalast · 17 hours ago

  Yeah cause every car needs 8xH200 pulling 10kW to run a VLM at realtime speeds. Would be unfortunate if 4G dropped out under some trees while using the API after all.

  • post-it · 17 hours ago

    Power usage isn't an issue. 10 kW is 13 HP. The size, price, and fragility of the components is the issue.

    • AlphaSite · 17 hours ago

      GPUs/XPUs are small and solid state so it’s only really price that’s a huge liking factor. And the disinclination of these companies to push the weights of their cutting edge models into people’s cars where they can be dumped.

    • officeplant · 16 hours ago

      Steady 10KW load means 40 less miles after an hour of driving if your EV gets 4mi/kwh. That kind of draw would use up nearly 1/6th of my EV's battery in an hour.

      • post-it · 13 hours ago

        Man, EVs are very efficient. I was thinking from the perspective of my 120 HP Hyundai Venue, where it would be only an extra 10% of peak power.

        • officeplant · 13 hours ago

          At the same time it's pretty crazy to look at my Ford E-Transit's power draw while on the interstate and think about just how many electronics I could power with the 20-60KW of power draw it takes to maintain 65mph on a relatively flat stretch of highway.

  • Marha01 · 15 hours ago

    > Yeah cause every car needs 8xH200 pulling 10kW to run a VLM at realtime speeds. When the models stop improving, we will get model-specific ASICs that are much more power-efficient.

    • moffkalast · 1 hour ago

      > When the models stop improving Soo, never? Granted Cerebras is a thing, if the process can be commoditized. At the moment the area of edge inference at speed seems pretty bleak though.

• tintor · 17 hours ago

  lol. Wait until your cloud frontier LLM stalls / disconnects due to load / interference while your car is on highway OR making unprotected left turn OR approaching pedestrians. It is easy to make car driving *demos*.

• JoshTriplett · 17 hours ago

  > The bitter lesson is finally coming for the self-driving cars. Maybe, but the opacity level of models is not acceptable for cars. "Why did it drive under the semi?" "Model said to." "Why did the model say to?" " shrug "

  • sebastos · 17 hours ago

    But if the model is an LLM, you actually COULD ask it why it drove under the semi, and it would give you an answer. Now, you may argue that it will just be generating a whole new, backwards-rationalized post-hoc explanation of its own behavior given the logs that it managed to take before the crash. But then I ask you: how do you think a person explains why they did what they did after a crash? I direct you to all of the unsettling split-brain neuroscience literature demonstrating that humans are incorrigible backwards rationalizers who make for unreliable witnesses.

    • JoshTriplett · 15 hours ago

      That's a bug, and it would be an awful mistake to replicate that bug rather than fix it.

  • Marha01 · 16 hours ago

    > Maybe, but the opacity level of models is not acceptable for cars. That depends on actual performance of the model. I would prefer an opaque model with clearly superhuman driving abilities to a human, or to a non-opaque model with worse performance.

    • JoshTriplett · 15 hours ago

      It really doesn't. https://knowyourmeme.com/memes/a-computer-can-never-be-held-... No self-driving cars that aren't transparent about exactly how they work. (Ideally, no anything that isn't transparent about exactly how it works.)

      • Marha01 · 14 hours ago

        This take can perhaps appear to make sense in a situation when clearly superhuman opaque AI models don't yet exist. But once they do, good luck convincing people that they should not save lives or reduce their personal risks, just because they always supposedly need an explanation for any accidental deaths, lol. In our scenario (self-driving), the one who would be ultimately "held accountable" would not be the computer, or the company, but the person who died after singing a waiver/EULA and getting into a statistically superhuman autonomous car, then having a stroke of incredibly bad luck. Such events will happen, but they will be very rare.

• giancarlostoro · 17 hours ago

  Sounds really expensive. I think OpenAI and Anthropic should really not dismiss making smaller capable models that they can license out in this space on the other hand.

• atonse · 17 hours ago

  Tesla's already solved this - their vision model does this phenomenally well. And they've demonstrated adding a sidecar LLM to it as well, mostly for these kinds of "read these 3 street signs, what should i do next?" sort of situations.

  • matt_heimer · 16 hours ago

    The same Tesla that pulled radar to go vision only and a person was killed because the vision model didn't recognize a truck? https://www.bbc.com/news/technology-36680043 Not sure that counts as phenomenally well.

    • danielklnstein · 16 hours ago

      That incident and article is from ten years ago.

      • matt_heimer · 15 hours ago

        From this year https://www.autoweek.com/news/a70794762/nhtsa-escalates-prob...

        • danielklnstein · 15 hours ago

          From that same article - "The agency said it has identified nine crashes potentially linked to the issue, including one fatality and two involving injuries. It is also reviewing six additional crashes that may be related." Fifteen crashes - though not to be trivialized - is not a damning number at all in this context. What's more, per the article it's unconfirmed that the crashes are related, so it's hardly fitting to dismiss Tesla's approach based on this. I think it's great that serious efforts are being made in different approaches to autonomous driving - and in this thread's context, it seems possible that Tesla's approach might eventually be revealed as the optimal approach given modern AI.

    • atonse · 15 hours ago

      This is from ten years ago. Tesla's vision only FSD is extremely good now. It has been for at least 18 months or longer.

      • Rohansi · 15 hours ago

        Only for 2024 or maybe late 2023 cars and later (HW4). The older the car is the worse the FSD software is because the old hardware can't run the latest software.

  • overgard · 16 hours ago

    Totally solved: https://electrek.co/2026/09/15/tesla-four-fatal-driver-assis...

    • Obscurity4340 · 16 hours ago

      $o $olved

    • Marha01 · 16 hours ago

      Anecdotes. Only accident rate per mile driven (compared to human drivers' rate) is a relevant comparison. I don't think that any self-driving system will ever be absolutely perfect (all such complex non-linear systems are to some degree probabilistic and chaotic), but as long as the accident rate is lower than the human accident rate, I would consider it solved.

      • ex-aws-dude · 14 hours ago

        > Only accident rate per mile driven (compared to human drivers' rate) is a relevant comparison This is such an insane take I see all the time from self-driving boosters If a self driving car glitches out and crashes in some edge case pathological scenario we don't just accept that as totally fine because its hidden under big statistics The reason why a crash happened does matter, its not just about aggregate statistics As a thought experiment if I have a perfect self driving system but I add some code that purposefully crashes 1 in 10 million rides are you ok riding in it since the aggregate statistics look good?

        • Marha01 · 14 hours ago

          > As a thought experiment if I have a perfect self driving system but I add some code that purposefully crashes 1 in 10 million rides are you ok riding in it since the aggregate statistics look good? Do I know about the purposefully added harmful code? If yes, I would demand you remove it, because why not. If I don't know about the code, I would be OK with it, since it's clearly still more safe than the alternative and apparently cannot be made even better.

        • parineum · 13 hours ago

          You seem to be neglecting the important part of that scenario where you're other option you have to compare it to is a human driver that will randomly get it a crash at some higher rate. You're making it sound like the obvious answer is the irrational one.

      • gf000 · 14 hours ago

        It's Lego mindstorm logic level to drive a car on the motorway, so per mile is an absolutely insanely bad metric in general. Per mile inside cities or other difficult scenarios are what may get close to an actually meaningful metric. That's why Tesla is very misleading and waymo is much more legit.

    • atonse · 15 hours ago

      You added "totally" - as others have said, I don't know if anyone will claim a perfect system with no fatalities, enough stats show that it is already much safer than human drivers. And the true third party validation is that insurance companies are starting to offer lower premiums the more you use FSD. So their risk models are showing enough improvement that they're putting their money where their mouths are. The idea that Tesla's FSD is not ready for the mainstream is quite outdated, given that tons of Tesla owners are already using it daily, not just your early adopter types.

    • red75prime · 15 hours ago

      Note that the article has no info on whether it's FSD or Autopilot and whether they contributed to the fatal crashes. "Verified engaged" means ADAS was active at some point in the interval from T-30s to the end of the accident. The total number of collisions is not normalized by the miles driven.

      • Rohansi · 15 hours ago

        It also has no info on what hardware+software version was in use. The older cars are significantly less capable but there are far more of them on the road.

• samuelknight · 17 hours ago

  The bitter lesson tells you about the trend in the technology. It does not get product to market with today's technology.

• nater5000 · 17 hours ago

  >It’s mostly a latency problem at this point. The models are too big to run locally, but given that open-weight models like Qwen already exist, an open-weight, low latency equivalent to Astra can’t be too far out. But this is a bit of a ridiculous take, no? You don't need Astra for self-driving. Astra is able to build complex 3D worlds, do your taxes, shop for you, and, apparently, drive a car. A self-driving car just needs to be able to drive a car. By the time you trim down Astra to just have the minimum capabilities needed to drive a car, you'll be looking at the same models these self-driving car companies already use. Then you get to deal with the actual hard problems, like handling failure cases (which will still be present with Astra). >The vision stack, 3D maps, lane selection grammar, occupancy networks, it’s maybe all about to give way to a single GPT looking at camera feeds and predicting the next steering wheel adjustment. Self-driving cars have been able to do this for a long time. The problem is that it isn't robust enough given the context. I mean, if Astra can drive a car with a single camera, then presumably Astra can drive the car even better with multiple cameras, and even better than that with 3D maps, etc. And when you start to consider the expectation of performance of these systems, you realize that these features really can't be omitted. If you're a company producing self-driving cars, then you do not want to face a lawsuit for you car killing someone because it physically would have never been able to see what it was doing because it lacked a camera. I think the real gain here is that something like Astra can be used to help build these autonomous stacks. If it is able to drive itself, then it is able to generate novel data, analyze large quantities of data, and use context that isn't typically available when processing this data to make improvements to the actual autonomy stack which is ultimately responsible for driving the car. But thinking that these car companies are going to run an LLM in a car and call it a day is just naive.

  • giancarlostoro · 17 hours ago

    The key thing Astra is doing is a loop... (my understanding) To figure out where things are... It's basically use more compute, self-driving cars are usually using on-device hardware where a "loop" might be a little too risky especially if it takes too long on local hardware... I wouldn't want my AI driving model to be over the air either, yikes in the case of lag or network outages.

  • sebastos · 16 hours ago

    No, no - all that “useless” knowledge is the good stuff. There is no clean interface boundary for driving a car, because the only interface that has been enforced is “if a human can navigate this situation, it’s fine”. Real world driving situations can be arbitrarily complicated, and if you want >human level driving, you need human level semantic understanding of the world around you. If you see a kid about to throw a model airplane across the street in front of you, you have to bring all your “useless” world knowledge with you to recognize that as a developing hazard. If you’re supposed to bring your passenger to the city building on main and you encounter construction outside with a detour sign saying “for tax dropoff park in rear”, suddenly all of your useless knowledge about the English language, what taxes are, and the likely goal of your passenger given their destination become useful.

• sebastos · 17 hours ago

  As somebody working near the field, I do enjoy the fun of dreaming bespoke vision and autonomy algorithms (if I didn’t, I wouldn’t work in the field to begin with!). But I would drop it all in a heartbeat for a robot that works well. Robust, resilient robots would be such an incredible advance that the ‘how’ doesn’t matter. All of the nonsense from the current AI hype cycle would be worth it if it cashed out in Robots That Actually Work.

  • SoftTalker · 17 hours ago

    Why do we need robots when we already have people?

    • ForHackernews · 17 hours ago

      I just want a robot butler. It doesn't have to prove mathematics theorems, just do my laundry and make lunch.

      • prideout · 16 hours ago

        Me too but unfortunately theorem proving seems to be easier than doing laundry.

      • SoftTalker · 15 hours ago

        Hire a housekeeper for 2 hours a day.

        • Marha01 · 15 hours ago

          I want a 24/7 robot butler. And I don't want any human strangers in my house, seeing my mess or seeing me naked.

    • sebastos · 16 hours ago

      For dull, dirty, and dangerous jobs!

      • VBprogrammer · 15 hours ago

        Seems to me that AI is coming for clean well paid office jobs long before it comes for anything dirty and dangerous.

        • Marha01 · 14 hours ago

          I think robotics AI revolution will come just a few years after the knowledge work AI revolution. We already have very promising robotics systems in active development.

          • VBprogrammer · 14 hours ago

            When AI does all the good jobs who is going to afford a robotic butler?

    • Marha01 · 16 hours ago

      > Why do we need robots when we already have people? Is this a serious question? Use your imagination...

      • SoftTalker · 15 hours ago

        To do what people do, but cheaper seems to be what it boils down to.

        • sejje · 15 hours ago

          To do what people do, without exposing humans to harms, dangers, and unnecessary risks. Also to do the things humans don't even want to do.

          • SoftTalker · 15 hours ago

            There is a type of person who thrives on risk and danger. Should we say they aren't allowed to work?

          • gf000 · 14 hours ago

            So that humans can live in harmony and prosperity, where no one has to work anymore and surely every resource will be distributed fairly. It will surely not devolve into the ultimate class war like Elysium and similar.

        • fragmede · 13 hours ago

          And that would be the incorrect conclusion. Yes, cheaper is better, but worse and more expensive is still in the running if your boss doesn't have to deal with the human aspect and the tasks still get done. Early cars were worse than horses, but they still won out because there wasn't the biological aspect to contend with. Think about it, a human has all sort of mushy human crap to deal with. They're going to come in hung over or just tired from the weekend/last night, all sad because their mom/brother/sister/partner got cancer/died and get into fights/trouble with HR over something a coworker did and have lower output. A magic box you can put the same tasks into and get sufficiently good output back out, and not have to give it time off because it's Christmas/their daughter's ballet recital, that you can spin up 30 copies of and spin them back down with no remorse is worth way more than simply being able to pay the box $18/hr vs $20/hr to a real live human. That's why businesses are salivating at the idea of AI/robots. Not because they'll eventually be cheaper. The robot loses an arm because your factory is unsafe? vs a human losing an arm? What we're not ready for is replacing GDP as the important metric. There have long been known problems with GDP, and robots are only going to make that worse. A robot maid, purchased once, saves, say 20/hrs a week in household chores. That's a meaningful quality of life upgrade, but doesn't result in the GDP bump that getting a raise and hiring a service to clean your house does.

  • user43928 · 15 hours ago

    I can't wait for a robot that does my household chores and cleaning. Not to mention construction, infrastructure, agriculture, manufacturing, logistics... -- AI hype cycle? It's working today. It's optimizing ML model graphs for me while I type this, and it already cut inference time from 30s to 18s. -- Some people act like there was no way for the AI labs to make back the $800B being invested in data center construction this year. If we look at global GDP, it's $126T, and even a 5% productivity gain would correspond to $6T. Is that impossible? Is it guaranteed to all crash? I don't think so.

• rayiner · 17 hours ago

  Probably not. In humans, the visual processing circuitry is very different from the circuitry for language processing. There is no reason to believe GPTs will be effective at it.

• SoftTalker · 17 hours ago

  If I'm reading the chart correctly, it took over 5 minutes to drive 135m at a cost of nearly $8.00 in tokens. I don't think that's really in the realm of practical yet.

• ed_balls · 17 hours ago

  I think this a slight different lesson. There is one algorithm that is called transformer, rest is irreverent/performance optimization.

• binlog · 16 hours ago

  Self-driving tech is more about reducing liability than the driving itself. The lidars and 3D maps and world models and everything else is needed to get reliability from 99.9% to 99.99% on public roads. This isn’t a SaaS product where the target is to be “good enough” at the cheapest cost.

• miltonlost · 16 hours ago

  Oh god, this move-fast-break-things thinking is going to kill so many people. We already have aftermarket problems with people adding in untested, unregulated self-driving features. https://arstechnica.com/cars/2026/09/aftermarket-driver-assi...

• boplicity · 16 hours ago

  I'm no expert, but I think the future is more about extremely low latency and low power chips with LLMs etched directly onto them. You can create specialized chips that function as "neurons" in a larger system, generating the needed reactions with a very clearly defined set of constraints.

• Razengan · 16 hours ago

  > The bitter lesson is finally coming for This is hilarious, and good: Those who were too lazy/stubborn/arrogant to adapt, get disrupted and buried.

• chris_money202 · 15 hours ago

  Think this dramatically simplifies the problem. AI existed before GPTs and the AI in self-driving is optimized for self-driving and the latency you already mentioned. Regardless of how the AI is architected, you aren't going to be able to use a generic LLM like Qwen to perform reliable self-driving, you need a highly optimized, highly specific AI.

• smusamashah · 13 hours ago

  Can you please explain what does it mean by bitter lesson in this context specifically? I keep seeing this term here. I know there is an article of the same title but I still don't understand.

Wow! but WHY is this a benchmark?? for comparison tesla's model is approximately 10-15B parameter model (estimating from maxxing the hardware that comes with the car at 16gb ram).

• N_A_T_E · 17 hours ago

  I would assume this is a proxy for general intelligence. A model that can drive a car and do a bunch of other real world stuff is closer to a generalized intelligence that can reason through any task.

• jrflo · 17 hours ago

  Tesla isn't using a general purpose model, they're using many highly-specialized models for a more deterministic system than "hey chat drive this car for me"

• therealdrag0 · 9 hours ago

  Why not? Benchmark all the things!

Pivot this to analyze and coach human drivers to be better drivers.

• anthonyrstevens · 17 hours ago

  "Get off your phone!" "Stay right except to pass!" I could get behind this.

3.8 flash would be the model to test, it's vision capabilities are excellent (on par with Astra) while also being incredibly fast.

• onlyrealcuzzo · 17 hours ago

  This is quite impressive... But I imagine this is orders of magnitude more expensive / less efficient than whatever Waymo is already doing, right? The cool thing is that 1) it's theoretically more generalizable, 2) if we wait 18 months, it'll be 100x cheaper, and another 100x cheaper likely in 18 more months - at that point - something like a Mac Studio inside a humanoid could have these generalized capabilities, and a lot of Robotics problems start to look more feasible - especially when you consider how much better the models could be if highly specialized.

• famouswaffles · 17 hours ago

  There isn't any model out there even close to as good as Astra at visual/spatial reasoning.

  • WarmWash · 17 hours ago

    Gemini models punch way above their weight in vision tasks https://artificialanalysis.ai/evaluations/mmmu-pro

What did they do to Astra so cracked at vision (and computer use). That ARC 3 score turned out to be no joke/fluke. That huge gap between Astra and Fable (in this case) is basically every hard vison/spatial benchmark i've seen including non-benchmarks like playing games (Portal, Factorio, RimWorld). SpatialBench - https://x.com/spicey_lemonade/status/2096365630190698516 ZeroBench - https://zerobench.github.io/ Robot Arms - https://openai.robocurve.org/gpt-6-astra/

• dyauspitr · 17 hours ago

  Well, they have the best in class image generator so that probably has something to do with it

• smusamashah · 17 hours ago

  I think Opus 5.5 is at same level now. I have seen too many videos made by Opus 5.5 today on twitter. https://x.com/victormustar/status/2102707412704919910 horse galloping pixel art https://x.com/LexnLin/status/2102133072585965759 moving train pixel art animation https://x.com/jkeatn/status/2102441348075057539 painting with code https://x.com/LCSlates/status/2102503027340988559 video, very detailed prompt though https://x.com/aj_dev_smith/status/2102504509637587339 generated song/music with code https://x.com/aj_dev_smith/status/2102575577563570450 another song

  • bigwheels · 16 hours ago

    Do these examples demonstrate new levels of computer use capability?

  • prideout · 16 hours ago

    These are amazing but the parent comment is referring to vision comprehension, not generation.

    • smusamashah · 16 hours ago

      Yup and I think these examples demonstrate just that. From my experience, both Claude and ChatGPT iterate over what they can see to build things like these. I don't think these examples are made without vision.

  • famouswaffles · 16 hours ago

    I don't think so. These are cool but all of this is code to x. I'm talking about actual computer control. Stuff like: - https://x.com/iam_zachi/status/2095992132620136677 Puzzles, games, painting software, robotic control and now driving. I haven't seen any other model fire on all cylinders like that.

I wonder if the companies would be willing to bet entirely on AI driven innovation if liability for misalignment was put squarely on companies, individuals, compute vendors, and LLM vendors. I don’t think they would opt for it, especially if an alternative option to use human-programmed tech was already available. There is something to be said about emphasizing on liability as a way to freeze or solidify AI Development. Right now it is too unfettered leading to predictions of AI dooms.

This also explains why Astra is so good at video generation. I have an Astra+Higgsfield setup. I could point it to a Github repo and ask it to generate a product walkthrough and it did a very good job by generating fake screens (e.g. with data filled in) from real ones - which wasn't possible in earlier models

https://x.com/tobiges/status/2098294046469022030 "Sam understands exponentials like no other. During a YC talk last year he predicted that AI would make breakthroughs in science in 2026 and solve a major open problem in 2027. Now here we are..." Now on a new vibe coded website Astra wins the benchmarks ...

• aditya-ramabadr · 16 hours ago

  Haha fair point, we didn't juice anything though! You can see all the traces and videos on the website, for example, here's one of Claude Fable's attempts: https://drivingbench.com/trace/claude-fable-5.1/3/ . The code is also open source on GitHub. Also in the Report you can see how we did everything; there's obviously variance but if you try a similar thing yourself the results would probably be similar? - Aditya, Tobias, Simon

  • dgellow · 16 hours ago

    > During a YC talk last year he predicted that AI would make breakthroughs in science in 2026 and solve a major open problem in 2027 Pretty much everybody “predicted” this fwiw.

What about Jev? :-D

Looks like the "most successful" path drove over empty parking spaces and came close to two curbs?

• aditya-ramabadr · 16 hours ago

  Driving over empty parking spaces was definitely required, the cone course went through parking spaces (see https://drivingbench.com/report/#course ) and also went tightly around curbs. Next time we definitely want to go farther out from the Bay Area and find a much more open lot to build a larger and more difficult course. But even at these low speeds and with this course, the LLMs performed better than we expected! - Aditya, Tobias, Simon

New pelican on a bicycle? Genuinely though, this is fun but not at all what these models are good for. It's like cooking a meal with your feet or somthing. A youtube challenge video from 2012

interesting, im wondering if models like jev could drive a car too?

I'm not an expert in the LLM space, but I'm an external contributor to comma.ai's openpilot project and I'm and quite familiar with how its controls work, so I looked from that perspective. There's two questions here: 1) Could a cloud-delivered LLM figure out how to drive this route, based on those input data and given access to those output actuators? Looks like yes. Sure. 2) Could this work in the real world? Absolutely not. Three reasons: latency, latency, and latency. openpilot's driving model updates the target curvature and acceleration at 20Hz. Every millisecond of the round trip time through every piece of its entirely-local driving stack is well-understood, extremely consistent, and tightly optimized. It has to be, otherwise you can't react to even minor bumps or wind gusts, much less rapidly-developing traffic situations. Adding even a single speed of light RTT to a cloud service is meaningfully bad, and you'll need a whole lot more to encode and upload camera imagery to even start the time-to-LLM-response clock, and then send the response back down. By then the world around the car has moved on. There's a reason Tesla and every other self-driving manufacturer need the compute hardware in the car .

• ramesh31 · 16 hours ago

  Perhaps there's a synthesis to be had though. Eyes, control, and safety critical features on the hardware, higher level decision making to the cloud. Openpilot's biggest weakness has always been in the very "robotic" way that it drives, which is technically correct but causes frustration for other drivers. Deciding "should I pass this car" is a fundamentally different question to "can I pass this car", or "what is the actual safe speed and following distance given the current traffic conditions and weather".

  • pishpash · 14 hours ago

    What happens when the network flakes out? Cloud will never work for this.

    • ramesh31 · 13 hours ago

      What I'm describing strictly enhances what's already possible, though. You'd degrade back to current performance.

• ivanjermakov · 16 hours ago

  > otherwise you can't react I'm far from neuroscience, but humans don't need to operate at 20Hz to drive a car. And human reaction latency (event to measurable action) is often over 1s (under 1Hz).

  • bonsai_spool · 16 hours ago

    > but humans don't need to operate at 20Hz to drive a ca This is not a helpful statement unless you can claim what speed human sensors do work at. And it's going to be faster than the latency of $(sensor + server round trip) Hertz, not getting into LLM processing time.

    • Groxx · 16 hours ago

      It's also not subject to signal loss issues like anyone who uses a phone is quite familiar with. Unless you have narcolepsy.

    • nearbuy · 13 hours ago

      Are you asking for the latency or throughput? In humans, it's about 200–250 ms for a visual cue where you already know how to respond and you're ready, but you don't know exactly when it'll happen. It can be a fair bit longer if you need to identify what you see and choose how to respond. Typical perception to reaction time estimates for drivers when there's an unexpected hazard on the road are 1-2 seconds.

      • dgently7 · 5 hours ago

        do you know what it is for non visual cue? like the gust of wind or unexpected jerk of the wheel from a road feature? both of those seem like more of a speed of proprioception which seems faster than visual and have way more to do with general "car control" in normal driving where the path is planned.

      • preg_match · 5 hours ago

        I would imagine it’s much less than this with muscle memory. Driving is a very habitual process. The human body is certainly able to “short circuit” its thinking and respond faster. I mean, consider competitive video games. Humans who play a lot and pay attention respond to stimuli much faster than 250ms.

        • nearbuy · 4 hours ago

          This is well studied by researchers, and 1-2s is the typical reaction time for real drivers with real muscle memory when reacting to an unexpected hazard on the road. There are a lot of reasons people could sometimes react faster (for example, if they anticipate the hazard, or if they're just above average in reaction speed), but one to two seconds is the reaction speed we find most of the time. The fastest human reactions aren't to unexpected road hazards. We have a much faster reaction speed in tests where you just have to click the mouse each time the screen flashes. Our reactions are fastest when you know in advance the event is about to happen. But this isn't relevant for road safety.

  • cozzyd · 16 hours ago

    Let's see how well you play counterstrike with a 100 ms ping...

  • chaos_emergent · 16 hours ago

    The reaction latency you’re referring to for humans includes perception, planning, and actuation, I’d separate that from the concerns of the hardware, which are mostly about actuation frequency. From what I understand about AV (as a non-expert!), all three of those steps happen at different clock rates, ie you have a planner that’s updating continuously with observations from sensors at one rate, that planner then issues actions that get picked up by the actuators at another rate. In that sense 20hz should really be compared to human reflexes without perception and planning; in scenarios where one is anticipating an action, response time can be as low as 150ms. in that context, I think 50ms/20hz is plenty reasonable for an automated driver.

    • gpm · 16 hours ago

      In circumstances where one is maintaining grip or muscle tension (e.g. steering a car ) I believe human response time can be more like 50ms. Which perhaps unsurprisingly lines up with the 20hz figure pretty close to exactly (we built cars controls so that they're controllable by human reflexes). Though you can't convert between hz and latency, all 20hz tells us is that it adjusts 20 times a second, not how long it takes from sensor input to be fed into a particular choice of adjustment, there could be (and actually almost certainly are) multiple adjustments in flight simultaneously with the adjustment actually being applied being calculated from old data (both in humans and automated substitutes).

    • ASalazarMX · 13 hours ago

      Average human reaction time is about 250 ms, or 4Hz. That's still plenty fast for an attentive driver at reasonable speeds. More important, it's consistent when not distracted. Any LLM with latency would be like a driver constantly checking their phone.

      • nearbuy · 12 hours ago

        The typical perception-to-reaction latency of an alert driver to a hazard is about 1-2 seconds. 250 ms when you're waiting for an event and know how to respond. For example, like a batter in baseball waiting to swing.

        • ASalazarMX · 11 hours ago

          Correct, I just focused on pure reflexes to directly compare to Hz. Reacting strategically to unexpected situations is understandably slower.

  • replygirl · 16 hours ago

    reaction latency doesn't cover everything. the round trip from trigger to action is a few hundred ms at best, yes, but to enable that we are processing inputs at ~30hz minimum and integrating at ~5hz. you would total your car pretty quickly if you couldn't constantly adjust

  • mirrir · 16 hours ago

    I'm not an ornithologist but birds don't need to consume jet fuel to fly hundreds of miles either.

    • rafram · 16 hours ago

      But of course it helps.

      • kibwen · 16 hours ago

        True, after ingesting a stomach's worth of jet fuel the bird is powered for the rest of its life.

      • malshe · 16 hours ago

        In bird culture, this is considered a dick move

      • brookst · 13 hours ago

        Not really, it’s mainly a defense mechanism against hunters

  • suddenlybananas · 15 hours ago

    >human reaction latency (event to measurable action) is often over 1s This is so self evidently false, I struggle to believe you think it is true. How could anyone catch a ball even?

    • asah · 13 hours ago

      actually, human latency is quite slow and distracted drivers often have 1sec+ latency. it works because 99% of the time you don't need fast latency because you can accurately predict things. that's why a standard recommendation is to drive 2+ seconds (time not distance) behind the car in front of you. also why experienced drivers instinctively move their hands/feet into position during tricky moments when they need to cut the latency. fun exercise, try taking your foot off the gas and hitting the break - slower than you think!!

      • suddenlybananas · 11 hours ago

        Distracted drivers having a large latency is obviously not the same thing as humans in general having a large latency...

  • jacquesm · 14 hours ago

    Humans have multiple layers of processing such inputs and your subconscious reacts a lot faster than your conscious train of thought in case something happens (and then you have to 'catch up'). For the same reason that you don't consciously think about what you do when you are walking or how to stop yourself from falling when you stumble. That's all out of the top level and pushed further down to stack, sometimes even multiple levels.

• Onavo · 16 hours ago

  > Could a cloud-delivered LLM figure out how to drive this route, based on those input data and given access to those output actuators? Looks like yes. Sure. Well, if the massive cloud models that are generalized and have a world model that's good enough, you can just distill them into smaller models. As a point of reference, the current gen of Tesla FSD models only have 1B params. They are tiny by LLM/VLM standards.

  • chaos_emergent · 16 hours ago

    Wow, I had no idea that they are so small, that’s incredible! Really goes to show how much visual information can be compressed.

    • Onavo · 16 hours ago

      The next gen (v15) is supposedly going to be around 10B.

• aditya-ramabadr · 16 hours ago

  Great point! Yeah latency was one of the biggest issues here. To cope with that (and for safety reasons) the cars are driving at extremely low speeds. They also get timestamps with every tool call output etc so they can, in theory, "in context learn" about their own latency and choose motion durations and control how fast their iteration loop is to some extent. But yeah, this is just sort of a fun benchmark to see how good frontier LLMs are out-of-the-box at driving a real car, and probably not actually practical any time soon. -Aditya, Tobias, Simon

  • jyoung8607 · 16 hours ago

    To clarify my parent comment, I think it was an interesting experiment and seems like it was done well, and it may well be informative about what various frontier LLMs could do with recorded or world model footage. My only point is to say this sort of experiment is where it ends. Neither Anthropic nor OpenAI will be coming out with a "drive your car from the cloud" subscription until we have FTL communication, meaning never.

    • sashank_1509 · 16 hours ago

      Is it plausible they can use the large GPT model, to distill a smaller car driving model only from it and then run that onboard. Seems like that will solve all your issues.

      • sigmoid10 · 11 hours ago

        I'd be very surprised if at the very least Tesla/xAI aren't actively investigating that already. The general purpose intelligence to deal with complex new situations will never fit into a pure driving model, because it will need to understand human behaviour on a level that goes way beyond what people do on a road. I'm pretty sure that an eventual level 5 system will look closer to GPT than any traditional driving model. The biggest issue is indeed latency and we probably won't see it in real cars until a multi-trillion parameter model like GPT-6 fits on a simple ASIC that can run in an affordable car. Right now a stack of B200s that can run a frontier intelligence model costs more than a car itself. But a GPT-8 running something like 20k tokens/s on a Taalas HC5 will almost certainly be able to drive a car under real conditions.

        • zeven7 · 11 hours ago

          I imagine a mix of models would make sense - GPT controller to make overall decisions and override things (let’s avoid the dark alley it looks dangerous), driving model to handle what humans do when they are just driving and not thinking about it, maybe some other models too

    • Xmd5a · 15 hours ago

      What about Waymo's remote controlled cars? Why is this not an issue?

      • rented_mule · 14 hours ago

        My understanding is that Waymo's remote driving is not direct control of the car, for exactly these reasons among others. So human operators don't have steering wheels or joysticks. Instead the humans can give something closer to advice (e.g., "pull to the right and stop") that the car can accept, modify, or reject.

      • jyoung8607 · 13 hours ago

        The car always has to be capable of driving safely and avoiding collisions locally. The human operators are being asked occasionally to help with some higher level, longer term decisions. As a random example, if there's foreign objects blocking the road, the car has to be able to stop itself before hitting them, but it might phone-home to a human to decide if a u-turn is appropriate.

• simianwords · 16 hours ago

  How are you so sure that latency can't be improved? Sol can run on cerebras and we may get enough efficiencies that Astra can also be run locally.

  • nijave · 15 hours ago

    Even if latency is improved, it's still a monumental task powering a latency sensitive safety critical system over the internet--especially one that's moving. Maybe if latency can be improved _and_ it can run local inside the vehicle.

• odo1242 · 16 hours ago

  You can see this in the photos, it took over five minutes for the cars to get around the cone course.

• blactuary · 15 hours ago

  Kind of funny to mention comma today of all days

• aaroninsf · 15 hours ago

  Is that the same comma.ai project also in the news today? https://arstechnica.com/cars/2026/09/aftermarket-driver-assi...

• pishpash · 14 hours ago

  Yet remote pilots can fight wars on the other side of the world?

  • Insanity · 14 hours ago

    Flying a drone with e.g 1000ms RTT latency is not exactly the same as driving a car on a highway. There are typically less collisions in airspace.. :)

    • jacquesm · 14 hours ago

      There are fewer obstacles . That's the main reason it works, if you tried flying at 1 m above the ground it would become a lot more like driving, but without the benefit of friction. Flying requires less strict constraints on latency because it happens in straight line segments that are rather longer than the segments that you use when controlling a vehicle.

• bayarearefugee · 13 hours ago

  > Could this work in the real world? Absolutely not. Three reasons: latency, latency, and latency. That and also the fact that (in spite of their usefulness) LLMs still so often do incredibly dumb shit without thinking of the consequences that the idea of having them drive in public is absurd. Recently was using claude code/opus 5 to diagnose an intermittent wi-fi connection problem and one of the first things it did was to bring the adapter down. The wi-fi adapter was the only way the system was communicating with the outside world so claude effectively disconnected its own brain as step 1 in figuring out what was going wrong. Things did not progress well from there. Easy enough to clean up its mess in this case, but luckily it wasn't driving a heavy killing machine at the time.

  • NewsaHackO · 12 hours ago

    >Recently was using claude code/opus 5 to diagnose an intermittent wi-fi connection problem and one of the first things it did was to bring the adapter down. Do you mean restarting it? IDK, that would have been my first step too.

  • fragmede · 7 hours ago

    Let he whomst amongst us, that hath never committed such a sin, cast the first stone.

• ex1fm3ta · 13 hours ago

  It is also worth mentioning that the openpilot AI model is a world model. The way a world model understands physical reality and geometry makes it inherently safer for driving than an LLM, which is essentially a text-based statistical machine with no concept of the physical world.

• dnautics · 12 hours ago

  There's also token RTT on top of network latency.. but what if you had a model running at 10k tps (like taalas' llama3b-8

• mkotlikov · 12 hours ago

  So a Taalas chip can run Llama 3.1 8B at 17000 TPS...does that mean if we could get Astra at similar speeds we could get self-driving for free?

good. i'll get it to drive my 4 runner

Looking forward to the juggling bananas benchmark. If Claude can only manage 5 and Astra does 6, clearly they have a better model.

• torginus · 14 hours ago

  You cracked what the major version stands for!

Wow .. fascinating but I guess something like JEV is more appropriate here.