With the advent of large language models (LLMs), many studies have begun to explore simulations of more specialized professional scenarios. In addition to improvements in automatic text comprehension, the ability to automatically generate analytical reports has also advanced [1]. In the Earnings2Insights shared task, we aim to generate investment guidance based on earnings call transcripts. This task can be approached in two settings: one uses only the earnings call transcript, while the other enriches the content by retrieving additional relevant information based on timestamps. This shared task encourages exploration of both approaches.

In terms of evaluation, previous studies [1,2] have noted that using traditional metrics to compare with ground truth answers may not be meaningful, and that current LLMs are not yet suitable to serve as judges. Therefore, we adopt an evaluation method in [3], where annotators are asked to make investment decisions based on the generated reports. The correctness of these decisions will serve as the evaluation metric. As a result, the generated reports not only need to point in the right direction, but also must be persuasive enough to convince investors to follow their guidance. We believe this presents an exciting opportunity to explore human-AI interaction.

[1] Goldsack, Tomas, et al. From Facts to Insights: A Study on the Generation and Evaluation of Analytical Reports for Deciphering Earnings Calls. Proceedings of the 31st International Conference on Computational Linguistics. 2025.

[2] Chen, Chung-Chi, et al. Semeval-2024 Task 7: Numeral-Aware Language Understanding and Generation. Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). 2024.

[3] Takayanagi, Takehiro, et al. Can GPT-4 Sway Experts' Decisions?. Findings of the Association for Computational Linguistics: NAACL 2025. 2025.

We provide two sets of Earnings Call Transcripts. The first set contains 40 transcripts that correspond to ECTSum [4], and the second set includes 24 transcripts that are matched with professional analyst reports, referred to as the "Professional" subset. In the ECTSum subset, each folder includes a “ref” file, which represents the corresponding summary. Participants may choose whether or not to utilize this information. In the Professional subset, only the transcripts are provided; comparisons with the analyst reports will be conducted later by the organizers. All participating teams are required to submit reports generated for all 64 earnings calls.

Please fill out this form to access the data: https://forms.gle/k8Nzk2ganbURbzcv9

In [1], several automatic evaluation methods are mentioned. Participants are welcome to use the prompts and LLM-based evaluations provided in the reference to assess the quality of their generated reports. We encourage participants to propose a variety of evaluation methods in their papers. However, the official ranking will ultimately be based on the results of human evaluation, with a primary focus on whether the report can effectively guide and persuade investors to make correct decisions. Annotators will be asked to make investment choices (Long or Short) for the next day, week, and month based on the provided report. Each report will be evaluated according to the accuracy of these investment decisions. The final score will be the average accuracy across the three different time frames.

Each team is allowed to submit only one JSON file. Each entry in the file should include two keys: "ECC" and "Report". The value of "ECC" should correspond to the name of the relevant folder, such as "ABM_q3_2021", while the value of "Report" should be the model-generated report. Please send the results to takayanagi-takehiro590@g.ecc.u-tokyo.ac.jp with the subject line: "[TeamName] - Earnings2Insights Submission"

Example:

                                        [
                                          {
                                            "ECC": "ABM_q3_2021",
                                            "Report": "This is an analytical report generated for the ABM_q3_2021 dataset."
                                          },
                                          {
                                            "ECC": "XYZ_q1_2022",
                                            "Report": "This is the model-generated analysis report for the XYZ_q1_2022 dataset."
                                          }
                                        ]
                                          

The ACL Template MUST be used for your submission(s). Each may consist of up to 5 pages of content, plus unlimited references and appendix. Accepted papers proceedings will be published at ACL Anthology.

TBA

• Takehiro Takayanagi - The University of Tokyo, Japan
• Tomas Goldsack - University of Sheffield, UK
• Kiyoshi Izumi - The University of Tokyo, Japan
• Chenghua Lin - University of Manchester, UK
• Hiroya Takamura - AIST, Japan
• Chung-Chi Chen - AIST, Japan