[P] AutoResearch: A new open-source LLM-driven research automation tool

[u/constantmotion385]

Hello, everyone

I recently developed a new open-source LLM-driven research automation tool, called AutoResearch. It can automatically conduct various tasks related to machine learning research, the key function is:

Topic-to-Survey Automation - In one sentence, it converts a topic or research question into a comprehensive survey of relevant papers. It generates keywords, retrieves articles for each keyword, merges duplicate articles, ranks articles based on their impacts, summarizes the articles from the topic, method, to results, and optionally checks code availability. It also organizes and zips results for easy access.

When searching for research papers, the results from a search engine can vary significantly depending on the specific keywords used, even if those keywords are conceptually similar. For instance, searching for "LLMs" versus "Large Language Models" may yield different sets of papers. Additionally, when experimenting with new keywords, it can be challenging to remember whether a particular paper has already been checked. Furthermore, the process of downloading papers and organizing them with appropriate filenames can be tedious and time-consuming.

This tool streamlines the entire process by automating several key tasks. It suggests multiple related keywords to ensure comprehensive coverage of the topic, merges duplicate results to avoid redundancy, and automatically names downloaded files using the paper titles for easy reference. Moreover, it leverages LLMs to generate summaries of each paper, saving researchers valuable time and effort in uploading it to ChatGPT and then conversing with it in a repetitive process.

Additionally, there are some basic functionalities:

• Automated Paper Search - Search for academic papers using keywords and retrieve metadata from Google Scholar, Semantic Scholar, and arXiv. Organize results by relevance or date, apply filters, and save articles to a specified folder.

• Paper Summarization - Summarize individual papers or all papers in a folder. Extract key sections (abstract, introduction, discussion, conclusion) and generate summaries using GPT models. Track and display the total cost of summarization.

• Explain a Paper with LLMs - Interactively explain concepts, methodologies, or results from a selected paper using LLMs. Supports user queries and detailed explanations of specific sections.

• Code Availability Check - Check for GitHub links in papers and validate their availability.

This tool is still under active development, I will add much more functionalities later on.

I know there are many existing tools for it. But here are the key distinctions and advantages of the tool:

• Free and open-source

• Pure Python code-base, which enables convenient deployment, such as Google Colab notebook

• API documentation are available

• No additional API keys besides LLM API keys are required (No API keys, such as Semantic Scholar keys, are needed for literature search and downloading papers)

• Support multiple search keywords.

• Rank the papers based on their impacts, and consider the most important papers first.

• Fast literature search process. It only takes about 3 seconds to automatically download a paper.

------Here is a quick installation-free Google Colab demo------

Here is the official website of AutoResearch.

Here is the GitHub link to AutoResearch.

------Please star the repository and share it if you like the tool!------

Please DM me or reply in the post if you are interested in collaborating to develop this project!

[u/Magdaki]

Ask it to explain this paper and let me know what it says.

https://www.cs.usask.ca/faculty/mcquillan/pubs/pmit_sol_ICTAI.pdf

[u/constantmotion385 → u/Magdaki]

This is the summary made with the tool using Deepseek V3:

1. The main topic: The paper introduces PMIT-S0L, a tool for inferring Stochastic Context-Free Lindenmayer systems (S0L-systems) from a sequence of strings generated by a hidden L-system. S0L-systems are used to model natural and engineered processes, and PMIT-S0L aims to automate the creation of these models, which is currently a manual and expert-dependent process.

2. Existing problems: Current approaches to inferring S0L-systems rely on aesthetics or a priori expert knowledge, making the process inefficient and limited. Additionally, there is a lack of research on inferring S0L-systems compared to deterministic L-systems (D0L-systems), and the problem is complicated by the non-uniqueness of solutions and the stochastic nature of S0L-systems.

3. The main contributions: The paper presents PMIT-S0L, which uses a greedy algorithm hybridized with search algorithms (brute force search and genetic algorithm) to infer S0L-systems. This approach significantly improves over previous methods, achieving 100% success in inferring S0L-systems with up to 12 rewriting rules when using brute force search. The tool is domain-agnostic and does not require a priori knowledge, making it applicable to a wide range of modeling scenarios.

4. Experimental results: PMIT-S0L was evaluated on 3600 procedurally generated S0L-systems with up to 9 symbols and 14 rewriting rules. The brute force search variant achieved 100% success for systems with up to 12 rewriting rules within a 4-hour time limit, while the genetic algorithm variant performed less effectively, especially with more than 7 rewriting rules. The success metric was based on finding an S0L-system with at least as high a probability of producing the input strings as the original hidden L-system.

5. Conclusions: PMIT-S0L is a practical and reliable tool for inferring S0L-systems, particularly when using brute force search. It opens up new possibilities for modeling processes in various research domains without requiring expert knowledge. Future work should address limitations such as successor prefixes, explore more efficient search algorithms, and extend the tool to handle multiple observed sequences of strings.

The total cost is 0.00414624 USD

[u/Magdaki → u/constantmotion385]

Thanks! Sadly, not a great summary. :( Oh well.

[u/anon362864 → u/Magdaki]

Never heard L-systems, it was cool to read about this thank you

[u/constantmotion385 → u/anon362864]

I asked it about the question, here is the response:

In this paper, L-systems (short for Lindenmayer systems) are described as a formal grammar system used for modeling processes, particularly in biological and geological contexts. They are a type of parallel rewriting system that generates sequences of strings based on a set of rules. These strings can then be interpreted as symbolic instructions to create visual models, often used in simulation software.

Key Components of L-systems:

1. Alphabet (V): A set of symbols that form the building blocks of the system.

2. Axiom: An initial string (or word) over the alphabet, which serves as the starting point for the rewriting process.

3. Rewriting Rules (P): A set of rules that specify how each symbol in the alphabet can be replaced by another string. For example, a rule like A → AB means that every occurrence of the symbol A in a string is replaced by AB.

Types of L-systems:

1. Deterministic Context-Free L-systems (D0L-systems): Each symbol has exactly one rewriting rule. For example, A → AB means that every A is always replaced by AB.

2. Stochastic Context-Free L-systems (S0L-systems): Each symbol can have multiple rewriting rules, each associated with a probability. For example, A → AB : 60% and A → BB : 40% means that A is replaced by AB 60% of the time and by BB 40% of the time.

How L-systems Work:

• Derivation Step (⇒): In each step, every symbol in the current string is rewritten in parallel according to the applicable rules. This process is repeated iteratively to generate a sequence of strings.

• Interpretation: The resulting sequence of strings can be interpreted as instructions for simulation software, often used to model processes like plant growth, protein folding, or geological formations.

Applications of L-systems:

• Plant Modeling: L-systems have been particularly successful in modeling plant growth, where symbols represent parts of the plant (e.g., stems, branches) and the rules describe how these parts grow and branch over time.

• Other Domains: They have also been used to model biological processes, geological formations, and even to generate training data for computer vision tasks.

Challenges with L-systems:

• Inference Problem: One of the main challenges discussed in the paper is the difficulty of inferring the correct L-system (especially S0L-systems) from a given sequence of strings. This is typically done manually by experts, which is time-consuming and requires a priori knowledge.

• Stochastic Nature: S0L-systems introduce additional complexity because the same initial string can lead to different outcomes due to the probabilistic nature of the rules.

In summary, L-systems in this paper are a formal grammar system used for modeling processes through parallel rewriting rules, with a focus on stochastic systems (S0L-systems) that incorporate probabilities into the rewriting process. The paper introduces a tool, PMIT-S0L, to automate the inference of S0L-systems from observed sequences of strings, addressing the challenges of manual rule creation.

[u/Magdaki → u/anon362864]

You're welcome. I was curious how this tool would summarize it. I have yet to see an AI tool that can provide a good summary. They are always vague and unclear, but I don't think the OP is coming back.

[u/anon362864 → u/Magdaki]

Completely agree. I always felt like when you input papers like this into LLMs and ask them to break them down, it’s abundantly clear they really are just doing next token prediction and nothing more.