121 packages tagged with “dataflow”
Analyzer utilities for various analyses, including Dataflow analysis based on ControlFlowGraph API in Microsoft.CodeAnalysis.
TPL Dataflow promotes actor/agent-oriented designs through primitives for in-process message passing, dataflow, and pipelining. TDF builds upon the APIs and scheduling infrastructure provided by the Task Parallel Library (TPL), and integrates with the language support for asynchrony provided by C#, Visual Basic, and F#. Supported Platforms: - .NET Framework 4.5 - Windows 8 - Windows Phone 8.1 - Windows Phone Silverlight 8 - Portable Class Libraries
Utilities for Dataflow analysis based on ControlFlowGraph API in Microsoft.CodeAnalysis.
An easy async pipeline orchestration library
Gridsum.DataflowEx is a dataflow and etl framework redesigned on top of Microsoft TPL Dataflow library with Object-Oriented Programming in mind. DataflowEx enables you to write reusable components and easily link them together as a high-level dataflow graph while still having the power of low-level blocks from TPL Dataflow. Other cool features include: Dataflow lifecycle management, built-in dataflow health monitor, cyclic graph auto completion support, sql bulk insertion support, etc. More information is available at: https://github.com/gridsum/dataflowex
Data Flow is a fully managed service with a rich user interface to allow developers and data scientists to create, edit, and run Apache Spark applications at any scale without the need for clusters, an operations team, or highly specialized Spark knowledge. As a fully managed service, there is no infrastructure to deploy or manage. It is entirely driven by REST APIs, giving easy integration with applications or workflows.
Use GoJS to create directed and undirected graphs, network diagrams, flowcharts, workflow diagrams, entity relationship diagrams, BPMN and BPEL applications, organization charts, circuit diagrams, tree diagrams (e.g. syntax trees, sitemaps), data flow diagrams, mindmaps, SCADA diagrams, smart grid applications, planogram tools, UML diagrams and much more. Also supports JSON save and restore, popular automatic graph layout algorithms (tree, layered digraph and force directed), a Palette control for drag and drop of visual components and an Overview control for large diagrams. This full featured evaluation does not have a time limit, but there is a watermark in the view. Purchasing a license is required for deployment.
An easy Autofac extension for registering nodes in Banzai
ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
Dataflow addition to Navvy (a long-running tasks execution framework).
Log4Net Library for Banzai
Use GoXam for WPF to add diagrams to your application. It can show directed and undirected graphs, network diagrams, flowcharts, workflow diagrams, entity relationship diagrams, BPMN and BPEL applications, organization charts, circuit diagrams, tree diagrams (e.g. syntax trees, sitemaps), data flow diagrams, mindmaps, SCADA diagrams, smart grid applications, planogram tools, UML diagrams and much more. Also supports XML save and restore, popular automatic graph layout algorithms (tree, layered digraph and force directed), a Palette control for drag and drop of visual components and an overview control for large diagrams. This full featured evaluation does not have a time limit, but there is a watermark in the view. Its use is governed by the Evaluation License Agreement: https://nwoods.com/licenses/EvaluationAgreement.pdf Purchasing a development license is required for distribution. https://nwoods.com/licenses/LicenseAgreement.pdf Northwoods Software is only doing maintenance releases for GoXam -- no new development.
Use GoDiagram to create directed and undirected graphs, network diagrams, flowcharts, workflow diagrams, entity relationship diagrams, BPMN and BPEL applications, organization charts, circuit diagrams, tree diagrams (e.g. syntax trees, sitemaps), data flow diagrams, mindmaps, SCADA diagrams, smart grid applications, planogram tools, and much more. Also supports XML save and restore, SVG and PDF output, popular automatic graph layout algorithms (tree, layered digraph and force directed), a Palette control for drag and drop of visual components and an overview control for large diagrams. This full featured evaluation does not have a time limit, but there is a watermark in the view. Purchasing a license is required for distribution.
A helper library for async/await with TPL Dataflow.
This is the JSON connector for ETLBox. It lets you handle structured and semi-structured JSON data, whether working with files or APIs. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
Useful set of extensions and classes for simplifying Dataflow implementations. Part of the "Open" set of libraries.
This is the PostgreSQL connector for ETLBox. It supports reading from and writing to PostgreSQL databases for scalable data pipelines. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
This is the XML connector for ETLBox. It lets you read and write XML data, making it easier to work with hierarchical and structured information. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
TPL Dataflow Extensions
This is the CSV connector for ETLBox. It helps you read and write data in CSV format, making it easy to handle flat-file data for import, export, and transformations. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
This is the MySQL connector for ETLBox. It lets you connect to MySQL databases, handling relational data for ETL tasks and integrations. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
Wrapper of TcpClient what help focus on WHAT you transfer over TCP not HOW (Async, Thread-safe, TcpSerialization with attribute schema, Big/Little endian, Cancellation support)
This is the Parquet connector for ETLBox. It lets you handle Parquet files, offering an efficient way to process columnar data. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
This is the Analysis Services connector for ETLBox. It enables you to read and write data to Microsoft SQL Server Analysis Services (SSAS), making it easier to process and manage analytical models. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
This is the SQLite connector for ETLBox. It allows you to work with complete SQLite databases, ideal for embedded and local applications. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
This is the SQL Server connector for ETLBox. It enables integration with Microsoft SQL Server for handling structured data in ETL workflows. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
This is the MongoDB connector for ETLBox. It helps you work with MongoDB collections, offering support for NoSQL data processing and transformations. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
This is the Couchbase connector for ETLBox. It allows you to integrate with Couchbase databases, providing fast and flexible NoSQL data processing. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/
This is the Neo4j connector for ETLBox. It supports graph database integration, enabling you to process and analyze connected data. ETLBox is a complete ETL (Extract, Transform, Load) library and data integration toolbox for .NET. # Build scalable, code-first ETL pipelines for SQL, NoSQL, APIs, and flat files. # Automate data movement, transformation, and synchronization with minimal memory usage. # Ideal for data warehousing, migrations, and big data processing. Simplify your data integration workflow: ETLBox enables efficient, asynchronous data processing by reading from databases, APIs, and file formats like CSV, Excel, and JSON. Transform data dynamically with row-based, batch, or lookup transformations, and read or write from/to multiple destinations in parallel. Key Features: * Stream large datasets efficiently without loading everything into memory * Maximize performance with parallel, task-based data flow execution * Connect to various data sources with built-in connectors or extend with custom components For tutorials, examples, and documentation, visit: https://www.etlbox.net/