This is the core extension package. Always use it in conjunction with the corresponding database package (e.g., ETLBox.Postgres for PostgreSQL or ETLBox.SqlServer for SQL Server). ETLBox.EntityFramework extends Entity Framework (Core) with additional bulk operations, including insert, update, upsert, delete, and merge. While SaveChanges() is already optimized for performance, these bulk extensions may offer improvements in certain scenarios. These extensions are particularly useful when integrating ETL processes into your DbContext, especially when working with large datasets that cannot be fully loaded into memory. For tutorials, examples, and documentation, visit: https://ef.etlbox.net
$ dotnet add package ETLBox.EntityFrameworkGet Started | Overview | Code Examples | Licensing | Free Trial
ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. It helps you create and manage data integration pipelines to extract data from different sources, transform it into the format you need, and load it into databases, APIs, or files. Whether you're migrating data, synchronizing systems, or automating workflows, ETLBox provides the components to handle these tasks efficiently.
ETLBox is a flexible alternative to traditional ETL tools like SQL Server Integration Services (SSIS) or Azure Data Factory. Unlike those tools, ETLBox does not rely on a graphical user interface, making it ideal for developers who prefer programmatic control. It supports parallel execution and streaming, allowing efficient processing of large data with minimal memory usage.
ETLBox lets you build data pipelines that process structured and unstructured data from various sources, including databases, APIs, and flat files. It automates tasks like data transformation, validation, and cleaning, and supports bulk inserts, updates, and deletes. With its ability to handle streaming data and parallel workflows, ETLBox delivers performance and flexibility for modern data integration tasks. Developers can rely on built-in connectors or extend the library to fit custom requirements.
Example 1: Bulk Insert into SQL Database
string connString =
@"Source=.;Trusted_Connection=true;Initial Catalog=ETLBox;"
var destCon = new SqlConnectionManager(connString);
//Create the dataflow components
var source = new CsvSource(sourceCon, "SourceData.csv");
var dest = new DbDestination(destCon, "DestinationTable");
//Default mapping: Csv header and column name match (case-sensitive)
//Override mapping like this:
dest.ColumnMapping = new[] {
new DbColumnMap() {
DbColumnName = "Id", PropertyName = "OrderNumber"
}
};
//Link the source and destination, run the dataflow
source.LinkTo(dest);
await Network.ExecuteAsnyc(source);Example 2: Migrate Between Databases
string mySqlConnString =
@"Server=localhost;Database=ETLBox;Uid=user;Pwd=password;";
string postgresConnString =
@"Server=.;Initial Catalog=ETLBox;Trusted_Connection=true;";
var sourceCon = new MySqlConnectionManager(mySqlConnString);
var destCon = new PostgresConnectionManager(postgresConnString);
//Create the dataflow components
var source = new DbSource(sourceCon, "SourceTableName");
var dest = new DbDestination(destCon, "DestTableName");
//Link the source and destination, run the dataflow
source.LinkTo(dest);
await Network.ExecuteAsync(source);Example 3: Merge Data Between Tables
public class MyMergeRow : MergeableRow
{
[IdColumn]
public long Key { get; set; }
[CompareColumn]
[UpdateColumn]
public string Value { get; set; }
[DeleteColumn(true)]
public bool DeleteThisRow { get; set; }
}
//Create the dataflow components
DbSource<MyMergeRow> source = new (connection, "SourceTable");
DbMerge<MyMergeRow> mergeDest = new (connection, "DestinationTable");
merge.MergeMode = MergeMode.Full;
merge.CacheMode = CacheMode.Partial;
//Link the source and destination, run the dataflow
source.LinkTo(merge);
Network.Execute(source);For our full list of code examples, tutorials, licensing information, and documentation visit: www.etlbox.net
For more support and inquiries, please email us at: support@etlbox.net
Ready to build your ETL pipelines? Start now at www.etlbox.net.