Conclusion:
Using Avro for communication between your services significantly reduces data size and network traffic. Additionally choosing encoding (compression algorithm) can improve the results even further.
Deserialization of large collection of Avro objects one by one
using (var reader = AvroConvert.OpenDeserializer<CustomClass>(new MemoryStream(avroObject)))
{
while (reader.HasNext())
{
var item = reader.ReadNext();
// process item
}
}
- Library containing functionalities, which enable communication between microservices via Http using Avro data format
- Library containing components needed for Confluent Kafka integration
License
The project is CC BY-NC-SA 3.0 licensed.
For commercial purposes purchase AvroConvert on website - Xabe.net
Contribution
We want to improve AvroConvert as much as possible. If you have any idea, found next possible feature, optimization opportunity or better way for integration, leave a comment or pull request.
