• Home
  • About Us
  • Contact Us
  • Privacy Policy
  • Special Offers
Business Intelligence Info
  • Business Intelligence
    • BI News and Info
    • Big Data
    • Mobile and Cloud
    • Self-Service BI
  • CRM
    • CRM News and Info
    • InfusionSoft
    • Microsoft Dynamics CRM
    • NetSuite
    • OnContact
    • Salesforce
    • Workbooks
  • Data Mining
    • Pentaho
    • Sisense
    • Tableau
    • TIBCO Spotfire
  • Data Warehousing
    • DWH News and Info
    • IBM DB2
    • Microsoft SQL Server
    • Oracle
    • Teradata
  • Predictive Analytics
    • FICO
    • KNIME
    • Mathematica
    • Matlab
    • Minitab
    • RapidMiner
    • Revolution
    • SAP
    • SAS/SPSS
  • Humor

Movin’ On Up: Apache Kafka Gets a Promotion to the Enterprise

November 11, 2016   Big Data

Stream processing and real-time analytics are the driving trends behind the fast acceptance and general popularity of Kafka.

Another big data product for the Hadoop ecosystem, born at LinkedIn and nurtured to adolescence by the open source community at Apache, Kafka works much like the organization’s “central nervous system,” as it was described in Computerworld Magazine. It collects large quantities of data in real time as it streams in via user interactions, logs, application metrics, IoT devices, stock tickers, etc., and delivers it as a real-time data stream ready for use.

blog kafka feather Movin’ On Up: Apache Kafka Gets a Promotion to the Enterprise

LinkedIn has made something of a habit of birthing uber-good big data processing and streaming data products and then letting the Apache open-source community adopt and raise them. Kafka is one of these adopted kids.

Kafka Joins the Hadoop Ecosystem by Way of LinkedIn

Kafka isn’t the first Hadoop-related, open-source brainchild to spring forth from the innovators at LinkedIn. These are also the ones responsible for conceiving other projects that became Apache Incubator graduates, including Samza, Helix, and Voldemort (no, not him).

According to Stephen O’Grady, cofounder and analyst for RedMonk, Kafka is sometimes likened to ActiveMQ and RabbitMQ when it comes to what it does for on-premises data infrastructures, or perhaps with AWS Kinesis in the cloud.

Kafka is Becoming Mainstream within the Data Streaming/Real Time Community

Kafka is no longer relegated to the back closets of big data and the Hadoop ecosystem, either. It’s gone big time. According to a recent poll conducted by Kafka’s unofficial godparents at Confluent, Inc. in Palo Alto, Kafka is already working hard in the enterprise, primarily serving in use cases including:

  • Application monitor – 60% of users
  • Data warehouse – 51% of users
  • Asynchronous application – 47% of users
  • System monitor – 39% of users
  • Recommendation engine – 35% of users
  • Security monitor and/or fraud detector – 26% of users
  • IoT applications – 20% of users
  • Dynamic pricing – 12% of users

Another survey of Kafka community users indicates that 68 percent are getting ready to take on even more stream processing within the next year, and about 65 percent of their users are preparing to hire workers with Kafka skills within the next year.

According to Brian Hopkins, VP and analyst with Forrester Research, if you aren’t getting down with Kafka, you’re already getting behind the 8 ball. “Up to 2014 it was all about Hadoop, then it was Spark,” Hopkins told Computerworld. “Now, it’s Hadoop, Spark, and Kafka. These are three equal peers in the data-ingestion pipeline in this modern analytic architecture.”

blog hogwarts Movin’ On Up: Apache Kafka Gets a Promotion to the Enterprise

Voldemort is database; a distributed key-value storage system. Not a Harry Potter villain. Gotta hand it to Apache, though, it’s another example of their extraordinarily cool product naming system.

Getting Kafka Plugged into Your Enterprise Data Infrastructure

Of course, like most of the Hadoop ecosystem, Kafka doesn’t just waltz in and start singing and dancing with your existing infrastructure. It takes some plugging in and tuning first.

That’s where Syncsort comes in. The latest release of Syncsort DMX-h combines Kafka streaming with the rest of your enterprise batch data sources to greatly simplify integration of data streaming in Hadoop, Spark and Kafka.

For more information, check out the webcast, “Simplifying Big Data Integration with Syncsort DMX and DMX-h”

 Movin’ On Up: Apache Kafka Gets a Promotion to the Enterprise

Let’s block ads! (Why?)

Syncsort blog

Apache, Enterprise, Gets, Kafka, Movin’, Promotion
  • Recent Posts

    • TIBCO4Good and She Loves Data Offer Free Data Skills Workshops During a Time of Vulnerability
    • Aurora partners with Paccar to develop driverless trucks
    • “Without Data, Nothing” — Building Apps That Last With Data
    • SO MUCH FOR GLOBAL WARMING, EH?
    • Important Changes to Microsoft Dynamics 365 Field Service Mobile App
  • Categories

  • Archives

    • January 2021
    • December 2020
    • November 2020
    • October 2020
    • September 2020
    • August 2020
    • July 2020
    • June 2020
    • May 2020
    • April 2020
    • March 2020
    • February 2020
    • January 2020
    • December 2019
    • November 2019
    • October 2019
    • September 2019
    • August 2019
    • July 2019
    • June 2019
    • May 2019
    • April 2019
    • March 2019
    • February 2019
    • January 2019
    • December 2018
    • November 2018
    • October 2018
    • September 2018
    • August 2018
    • July 2018
    • June 2018
    • May 2018
    • April 2018
    • March 2018
    • February 2018
    • January 2018
    • December 2017
    • November 2017
    • October 2017
    • September 2017
    • August 2017
    • July 2017
    • June 2017
    • May 2017
    • April 2017
    • March 2017
    • February 2017
    • January 2017
    • December 2016
    • November 2016
    • October 2016
    • September 2016
    • August 2016
    • July 2016
    • June 2016
    • May 2016
    • April 2016
    • March 2016
    • February 2016
    • January 2016
    • December 2015
    • November 2015
    • October 2015
    • September 2015
    • August 2015
    • July 2015
    • June 2015
    • May 2015
    • April 2015
    • March 2015
    • February 2015
    • January 2015
    • December 2014
    • November 2014
© 2021 Business Intelligence Info
Power BI Training | G Com Solutions Limited