How to resolve News API Backend error?

Question

Error: "Backend error. Failed to deserialize backend response. Expected valid JSON. Error: invalid character 'R' looking for beginning of value"

My account is hitting rate‑limit when calling News API for batch news retrieval. The server returns plain‑text Rate limit exceeded instead of standard JSON payload, resulting in deserialization error: invalid character 'R' looking for beginning of value.

This issue does not happen when I log in with my colleague’s account and run the same batch news request. Can I know the reason and how can I resolve this?

wasin.w V3 · Answer

Hello @Tim09

The

https://developers.lseg.com/en/api-catalog/lseg-data-platform/lseg-data-library-for-python

have advantage over the pre-historic, legacy, ancient ek.get_news_headlines as follows:

* The new library retrieves headlines from the new platform (Data Platform) while the old library retrieves headlines from the old platform.
* The new library Content Layer supports get_data_async() method for News to retrieve data asynchronously. This helps request a lot of news data more effective.

Regarding the News Content Layer and Requesting data with Asynchronously, please check the following resources:

* Content/News example
* LSEG Data Library Content Layer: News definitions and methods.
* https://github.com/LSEG-API-Samples/Example.RDP.DataLibrary.Python.Concurrency examples (demonstrating with Historical Pricing, but the logic and be applied to News

Example Code: The first step is importing the required libraries, open the session.

import lseg.data as ld

from lseg.data.content import news

import pandas as pd

import asyncio

from datetime import datetime, timezone, timedelta

pd.options.display.max_colwidth = 100

Next, loading the RICs list from the excel file.

# Load customer records from the Excel file.

customers = pd.read_excel("customer list_202607_clean.xlsx")

# Build a clean, unique RIC list from the Ticker column.

rics = (

    customers["Ticker"]

    .dropna()  # Remove missing values.

    .astype(str)  # Normalize to strings for text operations.

    .str.strip()  # Remove leading/trailing whitespace.

    .loc[lambda tickers: tickers.ne("")]  # Drop blank strings.

    .drop_duplicates()  # Keep first occurrence only.

    .tolist()

 )

# Quick check: number of usable RICs.

len(rics)

Next, requesting News Headlines with Content Layer News module get_data_async() method using asyncio.gather(…, return_exceptions = True) and asyncio.Semaphore

* asyncio.gather(..., return_exceptions=True): Returns exceptions as results instead of raising them immediately, allowing all tasks to complete.
* asyncio.Semaphore(n): Limits concurrent execution so that at most n coroutines can enter the protected section at the same time.

# Limit the number of in-flight requests to avoid flooding the API.

semaphore = asyncio.Semaphore(10)  # Run requests concurrently

async def fetch_news(ric):

    """Fetch headlines for one RIC."""

    # Each task must acquire a slot before making the API request.

    async with semaphore:

        definition = news.headlines.Definition(

            query=f"R:{ric}",

            count=50,

            date_from="06.08.2026",

            date_to=timedelta(days=-4)

        )

# Return the asynchronous response for this single RIC.

        return await definition.get_data_async()  # pylint: disable=await-outside-async

# Keep successful responses and failures in separate lists.

news_data = []

failed_rics = []

try:

    results = await asyncio.gather(

        *(fetch_news(ric) for ric in rics),

        return_exceptions=True,

    )

except Exception as error:

    print(f"Unable to gather news requests: {error}")

else:

    for ric, result in zip(rics, results):

        if isinstance(result, BaseException):

            failed_rics.append((ric, result))

        else:

            news_data.append(result)

print(f"Total successful requests: {len(news_data)}")

print(f"Total failed requests: {len(failed_rics)}")

Jirapongse · Answer

@Tim09

The code uses three legacy APIs (refinitiv.dataplaform, Eikon Data API, and refinitiv.data).

import refinitiv.dataplatform.eikon as ek

ek.set_app_key('Your_App_Key') #Enter App Key Here

from refinitiv.data.content import news

import refinitiv.data as rd

Please use the LSEG Data Library for Python which is the latest one.

import lseg.data as ld
from lseg.data.content import news
ld.open_session()
…

The examples are on GitHub.

* Access/News
* Content/News

You can also refer to this Essential Guide to the Data Libraries - Generations of Python library (EDAPI, RDP, RD, LD) article.

Tim09 · Answer

Hi @ujjawal_khandelwal Attached is the full syntax.

I have some questions regarding the following suggestions:

time.sleep(0.25)

I have already experimented with adding delays between story retrieval calls. In fact, I also tested a longer delay of 2 seconds between requests, but I still encountered the same backend error.

Could you please advise whether the value in time.sleep() should generally be increased or decreased? Is there a recommended range that aligns with the News API rate limits? Since the issue persists even with a 2-second delay, I am wondering whether the root cause may be related to something other than the per-second request limit.

I will also try enabling debug logging and review the additional articles you referenced. If I am still unable to resolve the issue, I will share the relevant logs for further investigation.

News API.txt

EXPLORE OUR SITES