Enhancing Railway Innovation Through Improved Data Sharing

KONUX is embracing the future of data sharing with Delta Sharing, a data sharing protocol that simplifies one-way integrations and enhances real-time collaboration. By streamlining data access, we want to unlock seamless insights and operational efficiencies, driving innovation and growth across the industry. We invite our customers and partners to reach out and access our data insights in their favorite business intelligence tools or bring the data into their data lakes to empower data driven decisions and bridge operational gaps.

 

Data transparency and collaboration 

In the railway industry, data is often compartmentalized or siloed, making it challenging to consolidate and leverage for enhanced insights. Our goal at KONUX is to make our data available to customers and approved partners to set a crucial precedent for transparency and collaboration. For the railway industry to truly be innovative, we believe that an ecosystem needs to be fostered where customer systems and partners can share data driven insights freely to unlock significant savings and efficiencies in our customer processes. Through combining insights from different data sources, suppliers can innovate and create new data products that have the potential to further unlock savings in time and resources for our customers. This is especially important to keep in mind in today’s digital landscape where there is a growing trend towards creating comprehensive platforms, with everyone vying to build the next all-encompassing solution. However, the reality is that no single platform can address every use case or pain point. To truly succeed, we must recognize the importance of collaboration over competition. By drawing clear lines and understanding our own limitations, we open the door to partnerships that enhance our capabilities and deliver greater value to all stakeholders. Instead of focusing on capturing the largest share of the market, we should strive to work together, leveraging each other’s strengths to drive collective success and innovation. This interconnected network of data insights not only enhances our relationships with customers but also drives the industry towards a more cohesive, innovative, and cost-effective future.

Current data sharing methods and limitations

Data sharing is often still seen as providing a snapshot of time through file transfer protocols or what is more colloquially referred to as a “data dump”. The railway industry is still catching up to adopting technologies such as REST APIs or webhooks which have become almost a commodity in the IT industry. There are advantages and disadvantages to these approaches and at the base they typically involve setting up custom services to enable data exchange between different systems and organizations.

One advantage of traditional integration methods is that they can be highly tailored to meet specific needs and comply with customer IT and security requirements. Organizations can design these services to fit their unique workflows and data requirements, ensuring that the data exchange process aligns closely with their operational processes and goals. Additionally, once established, these custom solutions can provide a robust framework for secure and controlled data sharing. REST APIs, for instance, are excellent for facilitating real-time data access and integrating various applications seamlessly.

We also utilize event-driven integrations that use protocols like MQTT, which is common with IoT technologies, to publish insights onto a message broker where customers can listen to our messages and integrate the data directly into their business systems. This ensures our insights are seamlessly integrated into their operational workflows and we utilize this approach for our biggest customer today to deliver insights where they are needed the most. This approach allows for asynchronous communication, which enables each entity to decide when to provide data and when to consume it making applications more independent. 

The main drawbacks with these traditional methods is that the process of creating and maintaining these custom services can be very resource-intensive, requiring significant time and expertise. Furthermore, traditional methods can lead to data silos, where information is isolated within specific systems and is inflexible to changes happening in the data structure, making it difficult to gain comprehensive insights and fostering inefficiencies. These integrations can be very valuable when there are clear use cases with defined workflows but can have challenges when users want to explore and analyze the data without the need for resource intensive integrations. 

 

Our approach to efficient data exchange

Enter Delta Sharing, a new approach to data sharing that offers a complementary solution by enhancing our existing data sharing approaches. It enables customers and partners to access the data behind the insights, helping them explore and understand current and historical data as well as the underlying relationships. By providing a standardized method for secure and efficient data exchange, Delta Sharing enhances our ability to collaborate and innovate. This isn’t about replacing existing integration methods but rather about augmenting them to offer more flexibility and depth of insight. With Delta Sharing, we empower our customers to make more informed decisions and drive greater efficiencies across their operations.

One of the key advantages of Delta Sharing is its simplicity and ease. By moving our database architecture from a Data Lake to a Delta Lake we unlock the ability to facilitate real-time collaboration and exploration of new use cases that can bring significant additional value to the railway industry. Yes, the migration to this open-source storage framework requires investment from organizations but once it is completed, sharing live and historical data securely is as simple as sending an email. We pick the tables we want to share and we provide the share to a user who then gets an email with a link and an authorization token. As an example, setting up a delta share into Power BI takes a few minutes without any investment in building or maintaining additional software infrastructure.

By leveraging this protocol, we can create an interconnected ecosystem where data flows freely between customer systems and partners. This openness not only enhances decision-making processes but also unlocks significant cost savings and operational efficiencies. Delta Sharing empowers our customers to integrate data seamlessly across their business units, driving innovation and bridging gaps that were previously difficult to address. By adopting Delta Sharing, we are not only improving the way we manage and disseminate data but also enable a new standard for the industry. This shift towards a more collaborative and dynamic approach to data sharing reflects our commitment to transparency and customer success. As we continue to embrace Delta Sharing, we look forward to a future where data-driven insights are easily accessible, enabling the industry to tackle challenges more effectively and unlock new opportunities for growth and innovation.

If you are interested in accessing our data insights into your tool of choice, please reach out to us at [email protected].