Kafka client to support 1.0.x kafka-python versions

[sibiryakov]

Targeting:

• new async client,
• message corruption,
• DBW and SW hanging.

[sibiryakov]

The summary is:

• With Kafka 0.8.2.1 None of the consumers in Kafka-python works normally, or at least I wasn't able to make them do so. KafkaConsumer doesn't support offsets auto commits (it works starting from kafka 0.9.0). SimpleConsumer crashes when committing offsets and consuming at the same time. See SimpleConsumer throws error when committing offsets dpkp/kafka-python#707
• Sometimes when consuming spider reports error

Traceback (most recent call last):
  File "/Users/sibiryakov/src/sh/kafka-python/kafka/future.py", line 33, in success
    self._call_backs('callback', self._callbacks, self.value)
  File "/Users/sibiryakov/src/sh/kafka-python/kafka/consumer/fetcher.py", line 656, in _handle_fetch_response
    elif error_type in (Errors.NotLeaderForPartitionError,
TypeError: 'FetchResponse_v0' object has no attribute '__getitem__'

[sibiryakov]

I think, it would be better to wait till kafka-python will be working properly for kafka 0.8. Otherwise by merging this code we make frontera incompatible with this Kafka version.

[sibiryakov]

If auto-commit will be fixed it would be possible to drop KAFKA_USE_SIMPLE_CONSUMER option.

[sibiryakov]

closed in favor of #223