Rapid Apache Avro serializer for .NET.
Get Started
$ dotnet add package AvroConvertReadme
AvroConvert
Rapid Avro serializer for C# .NET
Docs
Apache Avro format documentation
First steps with Avro in the .NET
Benefits
Avro format combines readability of JSON and data compression of binary serialization.
| AvroConvert | Apache.Avro | Newtonsoft.Json | |
|---|---|---|---|
| Rapid serialization | ✔️ | ✔️ | ✔️ |
| Low memory allocation | ✔️ | ✔️ | ✔️ |
| Readable schema of data structure | ✔️ | ✔️ | ✔️ |
| Support for C# native objects (Dictionary, List, DateTime...) | ✔️ | ❌ | ✔️ |
| Built-in data encryption | ✔️ | ✔️ | ❌ |
| Support for compression codecs | Deflate Snappy GZip Brotli | Deflate | ❌ |
Introducing Avro to the projects brings three main benefits:
- Reduction of data size and storage cost
- Decrease of the communication time and the network traffic between microservices
- Increased security - the data is not visible in plain text format
Article describing Avro format specification and Avro API idea: https://www.c-sharpcorner.com/blogs/avro-rest-api-as-the-evolution-of-json-based-communication-between-mic
Conclusion:
Using Avro for communication between your services significantly reduces data size and network traffic. Additionally choosing encoding (compression algorithm) can improve the results even further.
Features
- Serialization
byte[] avroObject = AvroConvert.Serialize(object yourObject);
- Deserialization
CustomClass deserializedObject = AvroConvert.Deserialize<CustomClass>(byte[] avroObject);
- Read schema from Avro object
string schemaInJsonFormat = AvroConvert.GetSchema(byte[] avroObject)
- Deserialization of large collection of Avro objects one by one
using (var reader = AvroConvert.OpenDeserializer<CustomClass>(new MemoryStream(avroObject)))
{
while (reader.HasNext())
{
var item = reader.ReadNext();
// process item
}
}
- Generation of C# models from Avro file or schema
string resultModel = AvroConvert.GenerateModel(avroObject);
- Conversion of Avro to JSON directly
var resultJson = AvroConvert.Avro2Json(avroObject);
Extended list:
- Serialization of .NET object to Avro format
- Deserialization of Avro data to .NET object
- Other Avro data related operations
- Schema related
License
AvroConvert is licensed under the CC BY-NC-SA 3.0 license.
For commercial use, please purchase a license.
Contribution
We want to improve AvroConvert as much as possible. If you have any idea, found next possible feature, optimization opportunity or better way for integration, leave a comment or pull request.
Thank you a million to all the contributors to the library, including those that raise issues, started conversations, and those who send pull requests. Thank you!
These amazing people have contributed to AvroConvert:
Related Work
-
- online Avro Schema to C# model converter
-
- Library containing functionalities, which enable communication between microservices via Http using Avro data format
-
- Library containing components needed for Confluent Kafka integration