LSEG Data Library for Python is Slow

Hello,

We want to pull settlement price data for commodity futures using lseg's data library for python. Currently we're using the workspace plug-in for excel, but we wish to migrate this to using the API. When I try to pull the data for ~900 contracts using the API there is a substantial amount of latency (we let it run for ~10 min and it did not complete) but when we pull this data in excel it is significantly faster (< 1min).

We run this query once a day at EOD. We are using the LSEG Data Library for Python, see the link below. We are connecting to the API through workspace by providing an APP KEY, and the code uses API calls that can be found in the samples in the codebook app in workspace.

https://developers.lseg.com/en/api-catalog/lseg-data-platform/lseg-data-library-for-python

Please see the sample code below: Note, ct list is a list of ~900 commodity future contracts

ld.open_session(app_key = APP_KEY)

df = ld.get_data(universe = ct_list,
fields = ['CF_LAST','CF_NETCHNG', 'OPEN_PRC', 'CF_HIGH', 'CF_LOW','SETTLE','CF_VOLUME','TR.OPENINTEREST','TR.InstrumentDescription', 'EXPIR_DATE', 'TR.FOFIRSTNOTICEDAY', 'LSTTRDDATE'],
parameters = {'SDate': '0CY', 'Curn': 'USD'})

ld.close_session()

Thanks!

Find more posts tagged with

python

#product

lseg-data-library

Accepted answers

All comments

raksina.samasiri

Hi @meagan ,

It could be because of the instrument list is too huge for a single request. Could you please try splitting the input instrument lists as batches and retrieve the data of each batch as a time with some wait time between each batch. Please be aware that the data requested is not exceed the Usage and Limits Guideline listed here.

I hope this helps and please let me know in case you have any further questions.

meagan

20260102-1329-17500-lseg-data-lib.log

Please see the attached log below. Again to re-iterate, in two minutes we only pulled settlement prices for ~36 contracts. Whereas with the excel workspace plug in, we can extract data for ~900 contracts in a matter of seconds. Is there a different type of API I can be using to pull raw data using REST without the pandas data formatting? Would this help with the latency issue?

Jirapongse

@meagan

I checked the log file and found that the problem should be in the backend service.

The library sent the following request to the backend service.

[2026-01-02T13:29:10.684793-05:00] - [sessions.desktop.workspace.0] - [DEBUG] - [12044 - MainThread] - [http_service] - [build_request] - HTTP Request id 3url = http://localhost:9000/api/udfmethod = POSTheaders = {'Content-Type': 'application/json', 'Authorization': 'Bearer …', 'x-tr-applicationid': '4d9da5c1b43c464abe725b4014bef6ced4defa57'}json = {'Entity': {'E': 'DataGrid_StandardAsync', 'W': {'requests': [{'instruments': ['LCOF6', 'LCOG6', 'LCOH6', 'LCOJ6', 'LCOK6', 'LCOM6', 'LCON6', 'LCOQ6', 'LCOU6', 'LCOV6', 'LCOX6', 'LCOZ6', 'LCOF7', 'LCOG7', 'LCOH7', 'LCOJ7', 'LCOK7', 'LCOM7', 'LCON7', 'LCOQ7', 'LCOU7', 'LCOV7', 'LCOX7', 'LCOZ7', 'LCOF8', 'LCOG8', 'LCOH8', 'LCOJ8', 'LCOK8', 'LCOM8', 'LCON8', 'LCOQ8', 'LCOU8', 'LCOV8', 'LCOX8', 'LCOZ8'], 'fields': [{'name': 'TR.OPENINTEREST'}, {'name': 'TR.INSTRUMENTDESCRIPTION'}, {'name': 'TR.FOFIRSTNOTICEDAY'}, {'name': 'CF_LAST'}, {'name': 'CF_NETCHNG'}, {'name': 'OPEN_PRC'}, {'name': 'CF_HIGH'}, {'name': 'CF_LOW'}, {'name': 'SETTLE'}, {'name': 'CF_VOLUME'}, {'name': 'EXPIR_DATE'}, {'name': 'LSTTRDDATE'}], 'parameters': {'SDate': '0CY', 'Curn': 'USD'}}]}}}

However, the service returned an estimated duration along with a ticket for polling. The library then continued polling until the data was retrieved.

You need to contact the product support team directly through LSEG Support to verify the backend service.

EXPLORE OUR SITES