DS-Lighting: Making Agent Harnesses Explicit for Data-Science Automation
Source: ArXiv AI ↗
A new toolkit called DS‑Lighting has been released to make the “harness” that underpins large‑language‑model (LLM) agents for data‑science automation explicit rather than implicit. The harness, which defines how tasks are represented, how execution state is managed, how output artifacts are constrained and how evaluation feedback is provided, has historically been hidden inside agents, hampering reproducibility and comparison across heterogeneous workflows. DS‑Lighting decomposes this harness into four reusable layers—data, workflow, execution and evaluation—and encodes diverse agents as executable operator programs that can run both fixed pipelines and adaptive search strategies. The toolkit also bundles several open‑source data‑science benchmarks into a unified MLE‑Bench‑style task format, offering a shared interface, sandboxed runtime and common metric protocol for systematic evaluation.
The primary benefit of making the harness explicit is a measurable improvement in reproducibility, comparability and reliability of end‑to‑end data‑science pipelines. Experiments that varied agents, harness configurations, underlying LLMs and ablation settings demonstrated fewer system‑level failures and more consistent results when the harness was defined according to DS‑Lighting’s layered schema. By providing a standardized way to describe and enforce task constraints, the toolkit enables researchers to isolate the contribution of the LLM agent itself from the surrounding infrastructure, facilitating fairer benchmarking and more transparent attribution of performance gains.
The release includes open‑source code hosted on GitHub, inviting the community to adopt and extend the framework. With DS‑Lighting, developers can plug in new benchmarks, customize harness layers, and share operator programs that adhere to the common task format, potentially accelerating progress in automated data‑science research. The authors anticipate that broader adoption will reduce the prevalence of avoidable failures in complex workflows, lower the barrier to reproducing published results, and create a more collaborative ecosystem for evaluating and improving LLM‑driven data‑science agents.
⚡ Effects Interpreter
🌍World Economylow impact
What this means for you
- ▶Trade and investment between countries could shift a little if things escalate.
- ▶Overseas suppliers might quietly rethink their pricing after a story like this.
🏙️Local Economylow impact
What this means for you
- ▶Prices at your local shops may feel a soft, indirect squeeze from this.
- ▶Your local economy has a way of catching these currents eventually.
🏦Rates & Banksmedium impact
What this means for you
- ▶Rate-setters usually take their time digesting a story like this.
- ▶The next rate-setting meeting is a more likely trigger than this news alone.
❤️Healthhigh impact
What this means for you
- ▶The strain, if any, tends to show up gradually in everyday life.
- ▶Local surgeries and clinics might see a short-lived rise in enquiries.
💷Wealthmedium impact
What this means for you
- ▶Retirement plans are seldom derailed by news of this size alone.
- ▶A calm head usually serves savers better than a brief reaction.
🏠Housingmedium impact
What this means for you
- ▶A cooling or warming market usually takes months to fully show up in prices.
- ▶Home costs usually respond later, once the bigger picture settles.
🌍 Effects for your country
The U.S.'s central bank rate currently stands at 3.75%, so a global rates move like this could feed through to local borrowing costs. The U.S.'s economy is growing at 2.3% with inflation at 3.2%, the backdrop this story's effects play out against locally. Housing-market effects like this are usually felt most where mortgage rates track the U.S.'s own central bank rate closely. With the U.S.'s unemployment at 4.4%, personal-finance effects like this land on a labour market that is currently fairly stable. Health-system effects are harder to localise from economic indicators alone — the real impact in the U.S. depends more on local healthcare capacity than the figures above.
The UK's central bank rate currently stands at 3.75%, so a global rates move like this could feed through to local borrowing costs. The UK's economy is growing at 0.8% with inflation at 3.2%, the backdrop this story's effects play out against locally. Housing-market effects like this are usually felt most where mortgage rates track the UK's own central bank rate closely. With the UK's unemployment at 5.6%, personal-finance effects like this land on a labour market that is currently fairly stable. Health-system effects are harder to localise from economic indicators alone — the real impact in the UK depends more on local healthcare capacity than the figures above.
France's central bank rate currently stands at 2%, so a global rates move like this could feed through to local borrowing costs. France's economy is growing at 0.9% with inflation at 1.8%, the backdrop this story's effects play out against locally. Housing-market effects like this are usually felt most where mortgage rates track France's own central bank rate closely. With France's unemployment at 7.9%, personal-finance effects like this land on a labour market that is already under strain. Health-system effects are harder to localise from economic indicators alone — the real impact in France depends more on local healthcare capacity than the figures above.
Romania's central bank rate currently stands at 6.5%, so a global rates move like this could feed through to local borrowing costs. Romania's economy is growing at 0.7% with inflation at 7.8%, the backdrop this story's effects play out against locally. Housing-market effects like this are usually felt most where mortgage rates track Romania's own central bank rate closely. With Romania's unemployment at 6%, personal-finance effects like this land on a labour market that is currently fairly stable. Health-system effects are harder to localise from economic indicators alone — the real impact in Romania depends more on local healthcare capacity than the figures above.
The UAE's central bank rate currently stands at 3.65%, so a global rates move like this could feed through to local borrowing costs. The UAE's economy is growing at 3.1% with inflation at 2.5%, the backdrop this story's effects play out against locally. Housing-market effects like this are usually felt most where mortgage rates track the UAE's own central bank rate closely. Health-system effects are harder to localise from economic indicators alone — the real impact in the UAE depends more on local healthcare capacity than the figures above.