Loading…
ApacheCon North America will be held at the Intercontinetal Miami in Miami, Florida. Register now for the event taking place May 16-18, 2017. 
Back To Schedule
Tuesday, May 16 • 3:30pm - 4:20pm
10 Things to Consider When Using Apache Kafka: Utilization Points of Apache Kafka Obtained From IoT Use Case - Naoto Umemori & Yuji Hagiwara, NTT DATA Corporation

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
Feedback form is now closed.
We've been working on IoT for around 3 years, and are recently targeting a use case of connected car that the scale is as follows:
* Concurrent connections over a million
* Throughput over 100 Gbps

We thought that Apache Kafka would be effective as a means of data collection function of IoT platform which satisfies these requirements. We built and tested the platform however we faced various issues below:
* Performance saturation of Kafka Producer NOT caused by depletion of computer resources
* Performance deterioration of Kafka Broker due to sudden disk IO
* Crash of Kafka Consumer due to sizing mistake between the Broker and Consumer

To solve the above issues and make full use of capabilities of the Kafka, it is necessary to clarify mechanism of the Kafka and its surroundings. This presentation will introduce pitfalls, solutions and best practices based on our experiences.
Speakers
YH

Yuji Hagiwara

Platform engineer, NTT DATA
He had developed PostgreSQL encryption module as an internship student at NTT Data in 2011. At this time, he knew OpenStack, and started to use it. He has joined OpenStack Swift development project since 2013. In this project, He is focusing on activities on the theme of Swift with... Read More →
avatar for Naoto Umemori

Naoto Umemori

Deputy Manager, Senior IT Specialist, NTT DATA
Naoto is a Senior IT Specialist and Deputy Manager at NTT DATA Corporation, working on technology and innovation area. He has spent around a decade in the Platform and Infrastructure field, focusing mainly on the open source software technology stack. He has experiences with talking... Read More →

cfp nttd2 pub pdf
Tuesday May 16, 2017 3:30pm - 4:20pm EDT
Michaelangelo
  ApacheIoT

Attendees (51)