Issue with LSEG Python Library News API Request Limit

Question

An issue has been encountered while using the LSEG Python library to retrieve news data. Although the expected daily request limit is 10,000, the number of successful requests appears to be limited to approximately 2,000 to 3,000 per day.

def refinitiv_get_news_body(row):

if row['status'] == "COMPLETED":

#print("body found locally")

return row["body"], "COMPLETED"

print(f"Requesting body for headling: {row['storyId']}")

return ld.news.get_story(row["storyId"]), “COMPLETED”

ld.open_session()

df_body = df_input.copy()

try:

for idx, row in df_body.iterrows():

body, status = refinitiv_get_news_body(row)

df_body.loc[idx, "body"] = body

    df_body.loc[idx, "status"] = status

except Exception as e:

logger. Error(f"Error: {e}”)

Jirapongse · Answer

The client may implement retry logic when an exception occurs. For example:

def get_story(storyId):
    try:
        story = ld.news.get_story(storyId, format=ld.news.Format.TEXT)
        return story, "success"
    except Exception as e:
        print("Exception:")
        print(e)
        return None, "Failure"

count = 1
retry_max = 3
for storyId in df['storyId'].to_list():
    retry_count = 1
    err = "Failure"
    while err == "Failure" and retry_count <= retry_max:
        story, err  = get_story(storyId)
        if err == "Failure":
            retry_count = retry_count + 1
        else:
            if story == None:
                print("### "+str(count)+": None")
            else:
                print("### "+str(count)+": "+story[:100])
            count = count + 1

if err == "Failure":
        break;

If the problem relates to the limit, the error could be like this:

Error code 429 | Too many requests, please try again later.

wasin.w V3 · Answer

Hello @GabrielS

Could you please let me know what is the problem with the given code? I have tried it on my environment using the Desktop session (consume data from Workspace) and it works fine.

Please see the daily rate limit, the Data Library news.get_headlines limit and how to calculate data points on the Data Library Usage and Limits Guideline document and

https://developers.lseg.com/en/article-catalog/article/the-data-library-for-python-maximum-usage-reference-guide

article.

GabrielS · Answer

I have created a syntax below for the NEWS pull request of the client

import pandas as pd

import lseg.data as ld

from datetime import datetime, timedelta

# Open session

ld.open_session()

def get_all_headlines(start_date, end_date):

all_data = []

current = start_date

while current < end_date:

next_day = current + timedelta(days=1)

print(f"Fetching: {current.date()}")

try:

df = ld.news.get_headlines(

query="R:AAPL.O AND Language:EN",  # VERY BROAD

start=current.strftime("%Y-%m-%d"),

end=next_day.strftime("%Y-%m-%d"),

count=500

).reset_index()

if not df.empty:

all_data.append(df)

# ✅ Pagination loop

while "next" in df.columns and df["next"].notna().any():

next_token = df["next"].iloc[-1]

df_next = ld.news.get_headlines(

next=next_token

).reset_index()

if df_next.empty:

break

all_data.append(df_next)

df = df_next

except Exception as e:

print(f"Error on {current}: {e}")

current = next_day

return pd.concat(all_data, ignore_index=True) if all_data else pd.DataFrame()

# ✅ RUN

df_all = get_all_headlines(

start_date=datetime(2025, 1, 1),

end_date=datetime(2025, 4, 7)

)

print("Total headlines pulled:", len(df_all))

# ✅ SAVE TO CSV

if not df_all.empty:

df_all.to_csv("lseg_headlines.csv", index=False, encoding="utf-8")

print("✅ Saved to lseg_headlines.csv")

else:

print("⚠️ No data to save")

# Close session

ld.close_session()

This pulls the news available for the RIC AAPL.O and Language English. I have tried to pull it on historical basis and it pulls the news.

Can you please advise on this?

EXPLORE OUR SITES