We are extremely proud to announce the public availability of Lightstreamer 7.1, the best Lightstreamer ever!
Together with Lightstreamer Server 7.1, we released updated SDKs for Clients and Adapters.
This release contains hundreds of exciting improvements, optimizations, and fixes. Some of the main new features are highlighted below. To learn about all the other improvements and fixes, please go through the complete changelog.
Web Push NotificationsPush messages to offline web apps like you already do with mobile apps. |
 Adaptive Send BufferSend large data blocks faster than ever. |
 BackpressureThrottle massive reconnections and never block. |
 Session DrainingDrain client connections smoothly during blue-green and rolling updates. |
 Extended Metadata AdapterEnforce unsubscriptions and session terminations from your Adapter code and more. |
 Secure Remote AdaptersEncrypt and authenticate your Remote Adapter connections. |
 Web Client with the TLCP ProtocolThe new Web Client uses a faster, more secure, and more standard protocol. |
 Web Client on npm (with ES Modules, TypeScript, and more)Get the JavaScript lib from npm with support for UMD, CommonJS, ES Modules, TypeScript, React Native, WebPack, Rollup.js, and Browserify. |
 Web Client Open Source on GitHubThe JavaScript library becomes open source and is on GitHub. |
 New .NET Standard Client and AdapterBrand-new .NET Client supports .NET Standard, Unified Client API, and TLCP protocol. The .NET flavor of Lightstreamer Adapters embraces .NET Standard. |
Web Push Notifications
Lightstreamer has been supporting native Push Notifications for mobile clients (Android and iOS) for a few years. The big news is that it now supports native Web Push Notifications too. This means you can easily push messages to Chrome, Firefox, and Safari browsers even when your web app is not running.Adaptive Send Buffer
One of the flagship features of Lightstreamer has always been adaptive throttling, that is, the ability to detect network congestions and resample/conflate the data flow accordingly, rather than simply queuing messages like a “dumb pipe”. This came to the price of keeping a pretty small send buffer on the TCP sockets. While this is not an issue in most scenarios, there are cases where large snapshots or large data updates need to be sent, resulting in low throughput. The solution was to configure a larger send buffer, resulting in minimizing the adaptive throttling effects.
Backpressure is “the resistance or force opposing the desired flow of fluid through pipes”. This concept can be extended to a flow of data moving through software. Basically, if the server is under tremendous pressure, it should let other components in the pipe know they should slow down.
Somehow related to backpressure, there is session draining. Imagine you know you need to terminate a Lightstreamer Server instance within a given amount of time. The reason might be you are doing a rolling update and you need to move clients from the old server to the new server within a predefined time. Or you are using Spot EC2 Instances on the AWS cloud and you receive the notification of imminent shutdown 2 minutes before. 
Based on several requests from the field, we decided to extend the Metadata Adapter interface of in-process Java Adapters to support some new features. For example, you can now enforce unsubscriptions for a given item for a given session from within your Adapter code, without requiring the collaboration of the client. Similarly, you can force a session termination without having to go through JMX.- An operation method, named forceUnsubscription, which allows Adapter code to enforce the unsubscription of the involved subscription on behalf of the client. See the TableInfo docs for details.
- New getters that provide further information on the involved subscription. We added, in particular, getDataAdapter, getSubscribedItems, and getSubscriptionStatistics. The latter, available upon subscription closing, provides stats on the number of updates sent to the client for that subscriptions. See TableInfo and look for the new SubscriptionStatistics class in the docs for details.
Originally, Remote Lightstreamer Adapters were used only to enable back-end technologies other than Java or to be able to deploy Java adapters on a different box than the server. With time, users have started to deploy Remote Adapters further and further away, on different sub-networks, on different networks, in different data centers, and in the cloud. In many cases, the Lightstreamer Server and its Remote Adapter counterpart need to communicate over the public Internet.Starting from Lightstreamer 7.1, all the communications between the Server and the Remote Adapters can be encrypted through TLS certificates. Furthermore, Remote Adapter can be required to authenticate upon connection.
This enables having Remote Adapters in the cloud, as well as in hybrid cloud environments and connected through the public Internet with no VPNs while assuring thorough security.
Using TLCP in the Web Client makes it more efficient, with reduced bandwidth usage and latency, as well as more secure (with the complete elimination of any eval calls in the library code).
TLCP is based on HTTP and WebSockets. The TLCP specification is publicly available and is registered as a WebSocket subprotocol at IANA (the Internet Assigned Numbers Authority).
The Web Client is available both as a core library (including only the core modules and smaller) and as a full library (including all the modules). Each of them is available in three flavors:
- UMD: can be used directly in the browser via a <script> tag; the UMD library is available both minified and un-minified.
- ES Module: for use with modern bundlers like Webpack 2+ or Rollup.
- CommonJS: for use with older bundlers like Browserify or Webpack 1.
Furthermore, the Web Client library can be used from TypeScript, through the included types.d.ts file, which declares the API types exported by the library.
Check out the npm page for full details.
The Web Client library now officially supports React Native, the cross-platform mobile development framework. Check out the demo project.
Finally, the JavaScript API documentation reference has been completely redesigned, with a brand-new JSDoc, which you’ll love!
The GitHub project contains the code for both the Web Client and the Node.js Client, as well as the tools needed for building the library.
We will progressively release all the Lightstreamer client libraries as open-source projects on GitHub.
The API has radically changed, conforming to the Lightstreamer Unified Client API model. This means you can now expect the same exact abstractions, behaviors, and features across all the supported Lightstreamer client libraries while preserving the peculiarities of every programming language.
The library is based on the Microsoft .NET Standard API specifications.
On the server-side, the Lightstreamer .NET Adapter library has been made compliant to Microsoft .NET Standard API too, which provides greater uniformity throughout the .NET ecosystem and works seamlessly with .NET Core, .NET Framework, Mono, and UWP apps.
How to Get It and How to Upgrade
To upgrade, please follow the instructions contained in the HOW_TO_UPGRADE.TXT file, available in the root folder of the Lightstreamer distribution.
We recommend upgrading the Lightstreamer Server first, which is guaranteed to be compatible with any current and previous client libraries. This means you can still use older clients, after you upgrade the server and you can even mix older and newer clients, without any service interruption.
Enjoy the new Lightstreamer!
Web Push Notifications
Lightstreamer has been supporting native Push Notifications for mobile clients (Android and iOS) for a few years. The big news is that it now supports native Web Push Notifications too. This means you can easily push messages to Chrome, Firefox, and Safari browsers even when your web app is not running.
Learn more about the differences between Data Streaming and Push Notifications in Chapter 5 of Lightstreamer General Concepts.
The Push Notification module of Lightstreamer has been completely re-engineered and it supports HTTP/2 for Apple APNs and FCM for Google.
Data items that are already managed by your Data Adapters can be pushed to mobile and web clients through native push notifications with no additional effort on the server-side. Push notifications are natively supported by Android and by Apple devices (iOS, macOS, tvOS, and watchOS), as well as by the Chrome, Firefox, and Safari browsers for any platforms.
Check out a couple of live demos of Lightstreamer Web Push Notifications in action:
Adaptive Send Buffer
One of the flagship features of Lightstreamer has always been adaptive throttling, that is, the ability to detect network congestions and resample/conflate the data flow accordingly, rather than simply queuing messages like a “dumb pipe”. This came to the price of keeping a pretty small send buffer on the TCP sockets. While this is not an issue in most scenarios, there are cases where large snapshots or large data updates need to be sent, resulting in low throughput. The solution was to configure a larger send buffer, resulting in minimizing the adaptive throttling effects.
Now, Lightsreamer is able to change the send buffer size dynamically and automatically during the life of a socket, adapting to the actual size of the messages that are being delivered. This means obtaining adaptive throttling and high throughput at the same time!
Backpressure
Backpressure is “the resistance or force opposing the desired flow of fluid through pipes”. This concept can be extended to a flow of data moving through software. Basically, if the server is under tremendous pressure, it should let other components in the pipe know they should slow down.
Lightstreamer has always had several backpressure mechanisms. With version 7.1, backpressure has been deeply revised to be able to cope with extreme cases of massive disconnections and reconnections. Imagine you use a blue-green strategy for software updates in your DevOps pipeline. When you switch the environments, all your clients will need to connect back to the new servers. When dealing with very high numbers of concurrent connections (even in the order of millions), a massive number of clients all trying to connect at the same time might end up blocking the server, because it has to follow up with clients that just connected and keep listening to new clients still connecting. With backpressure, the server is able to self assess when the pressure is too high, thus stopping to accept new sessions for very short intervals. This enables existing sessions to never be blocked due to resource contention. Clients that couldn’t connect will automatically retry soon after until they finally succeed. In other words, instead of trying to serve all the connection requests at the same time, Lightstreamer Server will spread them automatically adapting to its available hardware resources. This makes the server much more robust to massive requests coming from outside.
Backpressure is enabled by default in Lightreamer 7.1 and in most cases, it will work out of the box without specific tuning. But there are several levers that can be tweaked to adapt to specific scenarios (clustering styles, load balancer types, etc.). You can find some details in the server changelog, as well as in the Clustering document and in the configuration files. For any doubts or help, just get back to the Lightstreamer Support Team.
Session Draining
Somehow related to backpressure, there is session draining. Imagine you know you need to terminate a Lightstreamer Server instance within a given amount of time. The reason might be you are doing a rolling update and you need to move clients from the old server to the new server within a predefined time. Or you are using Spot EC2 Instances on the AWS cloud and you receive the notification of imminent shutdown 2 minutes before.
With session draining, you can send a new command to Lightstreamer Server via JMX to make it progressively disconnect all the clients at a given rate. This is better than abruptly disconnecting all the clients together. When they connect back to another server, they will not end up flooding it (for which there is the backpressure mechanism, in any case).
Extended Metadata Adapter
Based on several requests from the field, we decided to extend the Metadata Adapter interface of in-process Java Adapters to support some new features. For example, you can now enforce unsubscriptions for a given item for a given session from within your Adapter code, without requiring the collaboration of the client. Similarly, you can force a session termination without having to go through JMX.
More specifically, the MetadataProvider interface now has the setListener method, through which both session termination and notification of a fatal issue in the Adapter can be sent. Look for the new MetadataControlListener class in the docs for details. The new method has a default implementation, which ignores the listener, so both source and binary compatibility with existing Adapters is guaranteed.
The MetadataProvider interface now has the getSessionTimeToLive method too, which allows the Adapter to specify a maximum duration for that session, letting Lightstreamer take care of terminating the session when the time-to-live expires.
An extended TableInfo class is available, used in the notifyNewTables and notifyTablesClose methods, which adds the following features:
- An operation method, named forceUnsubscription, which allows Adapter code to enforce the unsubscription of the involved subscription on behalf of the client. See the TableInfo docs for details.
- New getters that provide further information on the involved subscription. We added, in particular, getDataAdapter, getSubscribedItems, and getSubscriptionStatistics. The latter, available upon subscription closing, provides stats on the number of updates sent to the client for that subscriptions. See TableInfo and look for the new SubscriptionStatistics class in the docs for details.
The clientContext map provided to notifyNewSession has been extended with information on the client API in use.
Secure Remote Adapters
Originally, Remote Lightstreamer Adapters were used only to enable back-end technologies other than Java or to be able to deploy Java adapters on a different box than the server. With time, users have started to deploy Remote Adapters further and further away, on different sub-networks, on different networks, in different data centers, and in the cloud. In many cases, the Lightstreamer Server and its Remote Adapter counterpart need to communicate over the public Internet.
Starting from Lightstreamer 7.1, all the communications between the Server and the Remote Adapters can be encrypted through TLS certificates. Furthermore, Remote Adapter can be required to authenticate upon connection.
This enables having Remote Adapters in the cloud, as well as in hybrid cloud environments and connected through the public Internet with no VPNs while assuring thorough security.
Web Client with the TLCP Protocol
Starting from version 8.0 of Lightstreamer Web Client library and Lightstreamer Node.js Client library, the communication protocol between the clients and the server has become TLCP. TLCP is an open protocol created by Lightstreamer and implemented by all the other Lightstreamer client libraries. The previous version of the Web Client and the Node.js Client still used a closed, JavaScript-based protocol. With version 8.0, the Web Client and the Node.js Client are aligned to the other libs, making TLCP the standard Lightstreamer protocol.
Using TLCP in the Web Client makes it more efficient, with reduced bandwidth usage and latency, as well as more secure (with the complete elimination of any eval calls in the library code).
TLCP is based on HTTP and WebSockets. The TLCP specification is publicly available and is registered as a WebSocket subprotocol at IANA (the Internet Assigned Numbers Authority).
Web Client on npm (with ES Modules, TypeScript, and more)
Starting from version 8.0, the Lightstreamer Web Client library is distributed on npm, the standard package manager for the JavaScript programming language. This opens up the possibility of using a variety of library formats to suit the needs of all the modern development flavors.
The Web Client is available both as a core library (including only the core modules and smaller) and as a full library (including all the modules). Each of them is available in three flavors:
- UMD: can be used directly in the browser via a <script> tag; the UMD library is available both minified and un-minified.
- ES Module: for use with modern bundlers like Webpack 2+ or Rollup.
- CommonJS: for use with older bundlers like Browserify or Webpack 1.
Furthermore, the Web Client library can be used from TypeScript, through the included types.d.ts file, which declares the API types exported by the library.
Check out the npm page for full details.
The Web Client library now officially supports React Native, the cross-platform mobile development framework. Check out the demo project.
Finally, the JavaScript API documentation reference has been completely redesigned, with a brand-new JSDoc, which you’ll love!
Web Client Open-Source on GitHub
The news regarding Web Client 8.0 have not finished yet. The library code has been released as an open-source project on GitHub. After many years of evolution, the JavaScript codebase of Lightstreamer is now fully open-source under the Apache 2.0 license, enabling developers to inspect the source code and to fork it.
The GitHub project contains the code for both the Web Client and the Node.js Client, as well as the tools needed for building the library.
We will progressively release all the Lightstreamer client libraries as open-source projects on GitHub.
New .NET Standard Client and Adapter
Version 5.0 of the Lightstreamer .NET Client library has transitioned to the TLCP protocol too (see the Web Client section for more details on TLCP).
The API has radically changed, conforming to the Lightstreamer Unified Client API model. This means you can now expect the same exact abstractions, behaviors, and features across all the supported Lightstreamer client libraries while preserving the peculiarities of every programming language.
The library is based on the Microsoft .NET Standard API specifications.
On the server-side, the Lightstreamer .NET Adapter library has been made compliant to Microsoft .NET Standard API too, which provides greater uniformity throughout the .NET ecosystem and works seamlessly with .NET Core, .NET Framework, Mono, and UWP apps.
How to Get It and How to Upgrade
As always, you can get the latest version of the Lightstreamer software from https://lightstreamer.com/download/.
In addition to the zip distribution, a Docker image and an Amazon EC2 AMI are available.
If you own an Enterprise license with a valid Maintenance Plan, you are automatically allowed to upgrade, in case you are using the Cloud License Manager. If you are still using license files, just reach out to the Lightstreamer Support team to get the updated keys.
To upgrade, please follow the instructions contained in the HOW_TO_UPGRADE.TXT file, available in the root folder of the Lightstreamer distribution.
We recommend upgrading the Lightstreamer Server first, which is guaranteed to be compatible with any current and previous client libraries. This means you can still use older clients, after you upgrade the server and you can even mix older and newer clients, without any service interruption.
Enjoy the new Lightstreamer!
Alessandro Alinone
