Use DictionaryType::FileDictionary with dictionary files from jar

Hello,

I am using EMA Java latest version (3.9.2.0)

I am implementing an RTO WebSocket implementation, and from what I encountered and read, you cannot download the dictionary from the WebSocket version of the RTO.

Given this fact, I am using the FileDictionary to make the EMA load it from local files.

The problem I have is that the dictionary files are inside a jar. The only solution that works seems to be to read the file and copy it into a temporary file and use that temporary file's path and give it to the EMA library to load it from there.

Is there any other way to load a dictionary file that is inside a jar besides that?

Thank you

Find more posts tagged with

java

ema-api

elektron-data-dictionary

#technology

Accepted answers

All comments

Jirapongse

@Zeno

Thank you for reaching out to us.

EMA can load dictionaries from local files or over the network, configurable through its configuration files. It cannot load dictionaries packaged inside a JAR file.

I tested and confirmed that RTO supports dictionary requests. However, this is not recommended for WebSocket connections because the dictionary size in JSON format is very large, which can negatively impact application performance. Additionally, in WebSocket/JSON connections, a dictionary is not required to decode data.

If your account is permitted to connect to RTO using the RSSL protocol, it is recommended to use RSSL instead. The RSSL protocol provides better performance than WebSocket/JSON.

Zeno

Yes, RTO supports dictionary requests on the WebSocket connections (port 443), but when using RTO WebSockets together with the EMA Java, I get the following error:

WARN [Thread-4] OmmConsumerImpl - loggerMsg
ClientName: ChannelCallbackClient
Severity: Warning
Text: Received ChannelDownReconnecting event on channel Channel_rto_eu-central-1-aws-1-sm.optimized-pricing-api.r
efinitiv.net_1112417419_1
RsslReactor @3e69cadb
RsslChannel @249352b3
Error Id -1
Internal sysError 0
Error Location Reactor.performChannelRead
Error text Failed to parse JSON message: JSON Converter parse error: Invalid UTF-8 middle byte 0x22
at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 1, column: 3156]
loggerMsgEnd

From what I've read and know, this happens because the encoding of the Dictionary is done in ISO_8859 instead of UTF8, which leads to the reading of the message to fail, as it tries to read it using UTF8. I remember having the same problem with HMDS WebSockets, where there it was also encoded in ISO.

Also, looking at example ex450_MP_QueryServiceDiscovery in the RT SDK Java Github repo, there it also mentions that when using EMA with RTO WebSockets, the dictionary should be loaded from local files as there is an issue trying to download it (line 388 in the Consumer.java file). From the looks of it, it is quite an old issue, from at least 2021.

Just to make sure, when you said "I tested and confirmed that RTO supports dictionary requests", you're referring to only the standalone WebSocket and not by using the EMA Java library with RTO WebSockets, right?

Jirapongse

@Zeno

Yes, I used the standalone WebSocket.

You are correct. EMA Java can't download the Dictionary from the server when using WebSocket connections.

To confirm that "Users cannot download the dictionary from the WebSocket version of the RTO", please contact the API support team directly via Contact Premium Support or raise this issue on GitHub.

Jirapongse

@Zeno

I received an update from the support team. EMA Java was unable to download the data dictionary from the RTO via a WebSocket connection.

Zeno

Okay, thank you for checking with them as well.

Out of curiosity, will they try to fix this issue or they will let it be?

For now, we are okay with creating temporary files and trying to keep the dictionary up to date with the one from the EMA Java Github.

Also, do you know the best place to always get the latest dictionary files?

Jirapongse

@Zeno

For the fix, please contact the API support team or the development team directly.

You can get the latest data dictionary files from the Software Downloads.

They are in the Product Family: MDS - General and Products: LSEG Real-Time Template Service Pack.

wasin.w V3

Hello @Zeno

There are two factors to let the EMA API can download the data dictionary from the LSEG Real-Time Server with the WebSocket connection:

The API must support the feature
The LSEG Real-Time Server must enable the enableMultipartDictionary : True on its configuration to enable downloading the dictionary in multiple parts.

Factor One: The API must support the feature

I and my colleague have tested the EMA Java and EMA C++ to connect to local ADS server (enableMultipartDictionary : True) using the WebSocket connection (DictionaryType::ChannelDictionary).

EMA C++: Works fine
EMA Java: Got the same "Failed to parse JSON message" error message.

I have opened a GitHub issue-338 to the RTSDK team. You may contact the API support team to let the team knows other clients are facing the same issue.

Factor Two: Setting enableMultipartDictionary : True on the server.

My colleague has tested the EMA C++ connect to RTO using the WebSocket connection (DictionaryType::ChannelDictionary). The EMA C++ could not download a complete data dictionary. My colleague said the problem looks like when he set enableMultipartDictionary : False on the ADS server.

I suggest you contact the RTO Support team to verify if the RTO endpoint supports the multiple parts dictionary download.

To contact the RTO support team, please visit https://support.lseg.com/s/?formtype=ProductQuery website.

wasin.w V3

Hello @Zeno

Just a quick question. Why are you using the WebSocket connection over the RSSL for RTO?

The RTO supports both RSSL and WebSocket connection. The RSSL is propriety TCP-based connection that is designed for data distribution by encoding-decoding data in binary format. RSSL always provides a better performance for data distribution then the WebSocket (which is JSON string).

The RSSL connection also supports RTO dictionary download without issue.

Zeno

Hello

Thanks for the earlier information and for rising a case on GitHub.

To answer your question, our first choice is to use the TCP RWF RSSL, but depending on the client's firewall settings and such, they might have the ports blocked and only allow the WebSocket port to connect to, so we would like to have fallbacks when it comes to such cases. In general, we will try to make the clients have the TCP ports open (14002, 14012 and 15000, or at least these were the ones I found), but you never know…

wasin.w V3

Hello @Zeno

Thank you for the explanation. I understand that the TCP port 443 (for WSS) is more common to open than our RSSL 14002 port.

EXPLORE OUR SITES