Questions tagged [choetl]

Cinchoo ETL is a popular high-performance ETL framework for .NET.

Cinchoo ETL is an ETL framework for .NET. Simple, intuitive Extract, transform and load (ETL) library for .NET. It is a code-based ETL framework for extracting data from multiple sources, transforming, and loading into your very own data warehouse in .NET environment. Extremely fast, flexible, and easy to use.

Sample shows how to load below CSV (emp.csv) file

Id,Name
1,Tom
2,Carl
3,Mark

Load using iterator

foreach (dynamic e in new ChoCSVReader("Emp.csv").WithFirstLineHeader())
    Console.WriteLine("Id: " + e.Id + " Name: " + e.Name);

Load using loop

var reader = new ChoCSVReader("Emp.csv").WithFirstLineHeader();
dynamic rec;

while ((rec = reader.Read()) != null)
    Console.WriteLine("Id: " + e.Id + " Name: " + e.Name);

Load using POCO object

public class Employee
{
    public int Id { get; set; }
    public string Name { get; set; }
}
foreach (var e in new ChoCSVReader<Employee>("Emp.csv").WithFirstLineHeader())
    Console.WriteLine("Id: " + e.Id + " Name: " + e.Name);

References

57 questions
0
votes
1 answer

Can ChoETL csv reader read a column into an ICollection field?

I have a csv with details about an order. There is one line per order, but in my code this translates to an Order object with an ICollection Lines property. I need to read each order in to an Order object with the order line info added as…
0
votes
2 answers

Exception trying to read parquet data from azure blob storage (Using ChoETL)

I currently using the ChoETL library to read parquet data, this is the code: BlobServiceClient blobServiceClient = new BlobServiceClient(azureStorage); BlobContainerClient container =…
Juan Castillo
  • 103
  • 2
  • 9
0
votes
0 answers

C# - Writing JSON to Parquet file does not work with nested json

I am trying to convert JSON data to parquet file. Below is my input. {"time": 1637045320491, "device": {"type_id": 1}, "message": "Test message", "metadata": {"product": {"name": "prodName", "vendor_name": "XYZ"}, "version": "1.0.0",…
Anand
  • 1
0
votes
1 answer

How to pass Azure blob file path url in ChoParquetReader method?

As per the link, i'm able to pass parquet file from my local folder and able to convert the data into json format. Below is sample code MemoryStream jsonMs = new MemoryStream(); using (var r = new ChoParquetReader(FILE_NAME)) { …
MSB
  • 29
  • 1
0
votes
1 answer

CSV quoted values with line break inside data

This question is specific to ChoETL CSV reader Take this example "Header1","Header2","Header3" "Value1","Val ue2","Value3" (Notepad++ screenshot) All headers and values are quoted There's a line break in "Value2" I've been playing with ChoETL…
The One
  • 4,560
  • 5
  • 36
  • 52
0
votes
0 answers

Using Event trigger Azure function, how to convert csv file to parquet file using c#

I want to get a file in azure blob storage and using event trigger function, I want to convert this csv file into parquet format and upload it to another blob storage. I tried to use choparquet reader, but it takes a lot of time to convert the file…
Amey Pimpley
  • 31
  • 1
  • 4
0
votes
1 answer

How do i get value for dynamic[] if the element matches?

I am trying out the following function to generate below csv file, however, I am not sure how to index the dynamic[] to get values matching to specific elements in json input file. public static void Json_to_Csv(string jsonInputFile, string…
Cataster
  • 3,081
  • 5
  • 32
  • 79
0
votes
1 answer

How to output JSON array as a single field in CSV using ChoETL

I'm using ChoETL to convert JSON to CSV. Currently, if a property in the JSON object is an array it is output into separate fields in JSON. Example: { "id", 1234, "states": [ "PA", "VA" ] }, { "id", 1235, …
cminus
  • 1,163
  • 12
  • 25
0
votes
2 answers

Building insert string to sql has problems from CSV/Json

I'm trying to build a string from a .CSV file. I convert it to Json and want to build an insert statement to my SQL Server. I'm using ChoETL nuget to convert from csv to json. When I build my string I get this: Insert into dbo.JsonMeta (Json)…
SqlKindaGuy
  • 3,501
  • 2
  • 12
  • 29
0
votes
1 answer

Csv stream to JSON conversion using ChoETL.Json failing

In my application I am getting csv stream and I am trying to convert Csv stream to JSON JToken dynamically, Here I will not be knowing the columns in the CSV so not able to define mapping. To convert this stream to JSON I am using ChoETL.Json…
Nithya
  • 582
  • 8
  • 19
0
votes
1 answer

How do I map CSV header to propertyname at runtime with ChoETL

Using choETL I found that I can map csv position to property name like this public class Emp { public string Name { get; set; } public string Other{ get; set; } public string MyId { get; set; } …
0
votes
1 answer

Convert string with escape characters in json object in .net core 3.1

con.Open(); MySqlParameter _ReturnValue = new MySqlParameter("_ReturnValue", MySqlDbType.Int32); _ReturnValue.Direction = System.Data.ParameterDirection.Output; cmd.Parameters.Add(_ReturnValue); using (var parser = new…
0
votes
2 answers

JSON to CSV conversion using ChoETL Some array value missing

I am converting a JSON file to a CSV file. The JSON has multiple nested objects and large size. While converting, I am able to get all the values out of the JSON and into the CSV. However, array values are missing . I am using CHOETL library. the…
0
votes
2 answers

How do I download a CSV file to a string from an https URL?

The code that I have listed here works when I ReadAllText from a local file. What I need to be able to do is replace the path "C:\LocalFiles\myFile.csv" with "https://mySite.blah/myFile.csv". I have tried several methods, but can't seem to be able…
jpolo
  • 35
  • 7
0
votes
1 answer

Header JSON Converting CSV to JSON

Hi im tring to convert CSV String to JSON, but the Header of the JSON has some issues with Encode i think. Here the code and the Output: [{... "Endere_o_4": "", "Endere_o_5": "", "Endere_o_6": "", "C_digo_Postal":…
userprofile1
  • 9
  • 1
  • 7