Tech Gadget Central Latest Tech News and Reviews
From video-recommendation engines and online banking to the developing AI revolution, data has become one of the world’s most valuable assets. Yet, consolidating data from disparate sources (e.g., databases, data warehouses, data lakes, and so on) into a format usable in real-time scenarios can be a monumental task in today’s hyper-connected world.
Applications that don’t need constant access to data can save time and effort by combining and processing data in batches at regular intervals. The processing of large amounts of data at once has several applications, including analyzing monthly sales figures. Nonetheless, there are times when an organization needs access to data as it is being created; for instance, customer service software that depends on up-to-the-moment details about each sale may require this. The data processing required by ride-hailing applications to match riders with drivers is too time-consuming to be done over the course of a few days. Stream data processing, in which data is collected and integrated for real-time access, is significantly more complex to configure, but is necessary in these kinds of settings.
Dozer aims to solve this problem by enabling quick, read-only APIs to be powered directly from any source using a modular, plug-and-play data infrastructure.
Dozer is the brainchild of Singapore-based entrepreneurs Vivek Gudapuri and Matteo Pelati, who launched the company a little over a year ago. To help them move beyond the product’s present source available (i.e. not-quite open source) form and into a fully monetizable solution, the duo has assembled a distributed workforce of 10 spanning Asia and Eastern Europe.
Dozer has been in secret mode for some time now, testing with a select group of design partners that remain nameless. In addition, the startup announced that it has secured $3 million in venture capital from Google’s Gradient Ventures, January Capital, and Sequoia Capital India (under the latter’s Surge program).
Distributed
Streaming databases and ETL (extract, transform, load) tools like Apache Flink, Airbyte, and Fivetran; caching layers for transient data storage like Redis; and instant APIs powered by the likes of Hasura or Supabase to funnel data between systems are just a few examples of the countless tools already available to transform, integrate, and harness distributed data.
Dozer, on the other hand, is applicable to every single one of these fields; it takes the finest features from each and eliminates the hassle of constructing the underlying infrastructure and plumbing for real-time data applications.
In order to extract, cache, and index data in real time and make it available through low-latency APIs, users plug Dozer into their preexisting data stack, which may consist of databases, data warehouses, and data lakes. Dozer, as detailed by Gudapuri to , is concerned with “making this data accessible in the most efficient way,” in contrast to tools like Airbyte and Fivetran, which aid in getting data into a data warehouse.
According to Gudapuri, Dozer “takes an opinionated approach” by focusing solely on a few difficulties. Dozer’s main selling point is its ability to provide APIs and real-time data updates, however incumbent streaming databases offer much more comprehensive solutions.
“We solve just the correct amount of difficulties in each of these areas to deliver a speedy construction experience for developers, and ready-to-go performance,” Gudapuri added. To get the same results, “developers (now) have to integrate many tools.”
Existing streaming databases, for instance, will likely provide the whole database experience, complete with query engine, data exploration, online analytical processing (OLAP), and so on, to the user. In their place, Dozer emphasizes what Pelati terms “pre-computed views” that are built with SQL, Python, and JavaScript and are available over low-latency gRPC and REST APIs.
And this is why, according to Pelati, Dozer can guarantee lower latency when querying data.
“Due of these design choices, Dozer provides significantly lower query latency,” explained Pelati. “This is crucial for apps that interact directly with end users.” It only takes minutes for a single developer to create what would normally take months. It saves time and money because the team doesn’t have to design and manage several integrations.
The (almost) open source catch
Dozer is promoted as a “open source” platform, but a look at its licensing on GitHub reveals that it employs an Elastic license 2.0 (ELv2), the same license that enterprise search engine Elastic embraced two years ago as part of its transition away from real open source. In fact, the Elastic license is not considered open source because it forbids anyone from reselling the program in hosted or managed form.
ELv2 is more correctly described as a “source accessible” license, which means that it provides many of the same advantages as a more liberal open source license like MIT, such as access to the underlying code and the freedom to modify, enhance, and otherwise alter Dozer. As long as it’s not AWS or another cloud giant wanting to monetize directly on top of Dozer, this feature alone will likely be enough to win over enterprises of all kinds.
The corporation has stated its intention to transition to a dual-license “very soon,” where all but “one essential module” of the core Dozer project will be MIT-licensed. In addition, the firm emphasizes that all of its client libraries, including Python, React, and JavaScript, are already available under the MIT license.
It’s worth mentioning that several organizations, like Netflix, have established internal infrastructure to handle a similar problem to the one Dozer is solving. Interestingly, Ioannis Papapanagiotou, a key figure in the development of Bulldozer, is now employed by Dozer in the capacity of advisor.
Dozer is in its infancy, but the $3 million in funding it has received from a number of notable investors means that it can move forward with commercialization, which will involve the release of a hosted SaaS version with a number of additional features. According to Gudapuri, this should be live in the next few months.
Gudapuri explained that the hosted service will handle things like auto-scaling, quick deployments, security, compliance, rate limitation, and other features.
