TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Import AI’s 476th issue covers three developments: a new benchmark where leading AI systems scored below human participants on intuitive visual reasoning, a benchmark intended to assess AI research taste, and analysis of how AI could accelerate activity in space. The supplied report gives detailed results for the visual benchmark but only partial information about TasteVal, and presents the space discussion as forward-looking analysis rather than a report of a new deployment.
The newsletter Import AI 476 reports that a new visual-reasoning benchmark found a wide gap between human participants and tested AI systems, while also examining a benchmark for AI research judgment and a possible future role for AI in space. The issue brings together reported test results and forward-looking analysis; it does not report a new space mission or establish that AI systems can independently improve themselves.
The benchmark, called Humanity’s Sixth Sense (HSS), was developed and released by AI startups Elorian and Scale AI. Its stated aim is to measure whether a model can infer information a visual scene implies but does not directly show. The set contains 522 questions based on 288 images and 234 short video clips, which together run for 17.6 hours. Tasks cover temporal and causal dynamics, physical and spatial reasoning, social understanding, and abstract or contextual inference.
According to the results summarized by Import AI, human participants scored 93.1 aggregate pass@1, compared with 53.6 for the top-scoring system listed, GPT 6 Astra. The newsletter reports that Chinese open-weight models GLM-5.3-flash and Kimi K3 scored 26.8 and 25.5, respectively. The reported figures are benchmark results, not a measure of general intelligence. The HSS authors say common errors included missing a decisive detail, accounting for 21% of failures, and misidentifying an object, person or role, accounting for 20%.
The issue also introduces TasteVal, a benchmark from P-Zero Research intended to test AI systems’ research judgment across eight tasks related to frontier model development. The supplied source excerpt describes areas such as training-data curation, language-model training and fine-tuning, preference modeling, and resistance to adversarial prompts, but cuts off before giving scores or a complete account of its findings. A separate section draws on an AI Frontiers article about how autonomous spacecraft, space-based infrastructure and manufacturing could change the strategic role of AI; these are scenarios, not confirmed developments already underway at scale.
Where Visual Reasoning Still Falls Short
The HSS results offer a concrete comparison between human performance and the systems tested on questions about what a scene implies. The gap may help researchers identify where multimodal models struggle, including attention to relevant details and correct identification of people and objects. It also puts a measurable test around a broader debate: whether capable visual reasoning requires dedicated spatial world models or can emerge, to some extent, from large multimodal language models.
Import AI argues that the reported performance of leading models complicates claims that such tasks necessarily require a wholly different kind of system. That is an interpretation, not a conclusion established by one benchmark. The human score is also not perfect, and results on a defined set of 522 questions cannot by themselves establish how well systems reason across everyday settings. The listed performance of Chinese open-weight models is relevant to comparisons on this test, but does not establish a general ranking across AI capabilities.
TasteVal addresses a different question: whether AI can make useful choices about research directions and methods, rather than simply execute a specified task. If a system can reliably select promising experiments, that could affect how quickly research progresses. The source material, however, provides no TasteVal scores or evidence that the benchmark demonstrates recursive self-improvement. The space discussion matters as a strategic possibility because autonomous systems could operate where communication is delayed and human presence is difficult, but the newsletter presents that as analysis of possible future consequences.
visual reasoning AI training datasets
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three Threads in One Issue
Import AI is a research-focused newsletter, and this issue combines benchmark coverage with commentary on longer-term implications. HSS tests intuitive inferences from images and clips rather than asking only for explicit object recognition. Examples described in the newsletter include judging whether a bookcase can hold more books or whether chains can bear the drag of a plough. Such questions require relating visible details to unstated physical or situational information.
The results should be read within the benchmark’s scope. The human baseline is high but below 100, and the AI figures are pass@1 scores for the systems named in the report. The supplied material does not provide evaluation dates, full testing procedures, model settings, or independent replication details. Those omissions limit how far readers can compare the numbers with results from other tests.
For the space discussion, the AI Frontiers article cited by Import AI considers how greater launch capacity could support space-based power, computing and manufacturing in the 2030s. It outlines possible advantages from autonomous operation and in-space assembly, as well as risks if infrastructure creates a self-reinforcing economic or strategic advantage. The newsletter’s discussion is a projection about what may become more relevant; it is not evidence that these capabilities or power dynamics have already emerged.
““measures how a model interprets and reasons through information a visual scene implies but doesn’t explicitly show.””
— Humanity’s Sixth Sense benchmark description, as quoted in Import AI
As an affiliate, we earn on qualifying purchases.
Limits of the Reported Evidence
The supplied material does not include the full HSS paper, evaluation conditions, or independent replications, so it is not possible from this account alone to assess how representative the benchmark is or how scores might change under different testing methods. It also does not state when the model evaluations were run. The results should be understood as performance on this particular test, not as a complete comparison of the systems’ capabilities.
For TasteVal, the source excerpt ends before reporting results, so it does not support a conclusion about which systems performed well or whether AI research taste has improved. The issue’s title refers to RSI, or recursive self-improvement, but the available text does not establish that systems can recursively improve themselves. The space analysis likewise describes possible future developments; timelines, technical feasibility, governance and the likelihood of strategic concentration remain unresolved.
As an affiliate, we earn on qualifying purchases.
What the Benchmarks Need to Show
The next useful evidence would include the full HSS paper and transparent evaluation details, followed by tests of whether models improve on the benchmark and whether gains transfer to unfamiliar scenes. Comparisons across model versions and independent replications could help establish how stable the reported gap is. The supplied issue does not announce a specific next release or evaluation date.
For TasteVal, readers would need the complete benchmark description and its results before judging what it says about AI research judgment or possible contributions to faster AI development. On space, the cited discussion points to expanding launch capacity and autonomous infrastructure as developments to watch, but the newsletter gives no confirmed project milestone. Until more evidence is available, claims about AI-driven space industrialization and compounding strategic advantage remain scenarios rather than established outcomes.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is Humanity’s Sixth Sense?
Humanity’s Sixth Sense (HSS) is a benchmark from Elorian and Scale AI that tests whether models can infer information implied by images and video clips. Its 522 questions cover temporal, physical, social and contextual reasoning.
How did the tested AI systems compare with people?
Import AI reports an aggregate pass@1 score of 93.1 for human participants and 53.6 for the highest-scoring listed model, GPT 6 Astra. The scores describe performance on HSS, not general intelligence.
What does the issue say about Chinese open-weight models?
The newsletter reports scores of 26.8 for GLM-5.3-flash and 25.5 for Kimi K3 on HSS. Those figures apply to this benchmark; they do not establish how the models compare across other tasks.
Does TasteVal show that AI can improve itself?
No such finding is included in the supplied source material. TasteVal is described as a benchmark for research judgment across eight AI-development-related tasks, but the excerpt gives no results establishing recursive self-improvement.
Has AI already created a major space-based industrial advantage?
The issue does not report that this has happened. It summarizes analysis of how autonomous spacecraft and future space infrastructure could affect industry and strategic power; the outcomes and timelines remain uncertain.
Source: rss
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
