Real Tasks Lab
Hello! We are a group of enthusiasts from different countries, curious about how the world works. We put language and coding models to work on prototypes of real tasks, and look for the subtle issues that only surface in practice.
Findings about each model go directly to its developer. Publicly, we share which models we are working with, how we work, and our general conclusions.
We chose three areas that let us study the models’ practical value and quality as fully as we can, from tasks with quick, measurable results to one with no simple measure at all.
We chose algorithmic trading because it is a technically demanding field that shows the quality of a result quickly. Our strategies trade live with small deposits. They have shown a positive P&L on trades, but the results do not cover the cost of servers and models, let alone our time. We use them to measure how well the models code and build strategies. We also want to understand how AI may change investing and trading, and what pitfalls lie along the way.
How well can models help build systems that detect risks with real social significance? We are prototyping monitoring in four areas.
These are research prototypes. RT Lab is not a law-enforcement body and makes no findings against any person or company.
Can AI work out a strategy for healthy ageing and longevity? Starting from depersonalised information our participants provide (we do not know their names, addresses or phone numbers), the models analyse publicly available baseline strategies and recommendations in depth, work through many combinations of factors, and look for the best strategy and principles for a good life.
This research does not use medical test results, diagnoses or doctors’ prescriptions, and it does not test or recommend any medication. If you have a health problem or question, we strongly urge you to consult a qualified medical professional.
This is our hardest track. There is no clear, fast measure by which to assess the models’ work here. But healthy ageing is among the most important challenges society faces, so we took it on, and how to evaluate models in this field may prove the most interesting question of all.
We have taken out subscriptions and started work with three models.
We do not intend to rank these models against one another. They change constantly, and every market leader is advancing at an extraordinary pace.
We may ask AI developers and infrastructure providers for in-kind support for this research, such as API credits or computing resources.
None to date. Our members pay for the project themselves. Anything we receive will be listed here, with the provider and the type of support.
Support gives no one control over our methods or conclusions, and does not mean that a provider endorses our work. We use it only for this research.
To join the team as a full member, please fill in a short form and we will be in touch.
Does it work for you? Let us know, then wait for the research results to judge the project itself.
Real Tasks Lab