ApacheCon North America will be held at the Intercontinetal Miami in Miami, Florida. Register now for the event taking place May 16-18, 2017. 
Back To Schedule
Tuesday, May 16 • 3:30pm - 4:20pm
10 Things to Consider When Using Apache Kafka: Utilization Points of Apache Kafka Obtained From IoT Use Case - Naoto Umemori & Yuji Hagiwara, NTT DATA Corporation

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
We've been working on IoT for around 3 years, and are recently targeting a use case of connected car that the scale is as follows:
* Concurrent connections over a million
* Throughput over 100 Gbps

We thought that Apache Kafka would be effective as a means of data collection function of IoT platform which satisfies these requirements. We built and tested the platform however we faced various issues below:
* Performance saturation of Kafka Producer NOT caused by depletion of computer resources
* Performance deterioration of Kafka Broker due to sudden disk IO
* Crash of Kafka Consumer due to sizing mistake between the Broker and Consumer

To solve the above issues and make full use of capabilities of the Kafka, it is necessary to clarify mechanism of the Kafka and its surroundings. This presentation will introduce pitfalls, solutions and best practices based on our experiences.


Yuji Hagiwara

Platform engineer, NTT DATA
He had developed PostgreSQL encryption module as an internship student at NTT Data in 2011. At this time, he knew OpenStack, and started to use it. He has joined OpenStack Swift development project since 2013. In this project, He is focusing on activities on the theme of Swift with... Read More →
avatar for Naoto Umemori

Naoto Umemori

Platform Engineer, NTT DATA
Naoto Umemori is a platform engineer at NTT DATA Corporation in Japan. He has worked on infrastructure, network and platform technology area since 2009. His experience includes computer and carrier network, distributed processing system, data monitoring & visualization and IaaS P... Read More →

Tuesday May 16, 2017 3:30pm - 4:20pm EDT