c# - Read csv logfiles with different headers/columns -

i need read multiple csv files , merge them. merged data used generating chart (with .net chart control).

so far i've done simple streamreader , added 1 datatable:

while (sr.peek() > -1) {   strline = sr.readline();   strline = strline.trimend(';');   strarray = strline.split(delimiter);   datatablemergeddata.rows.add(strarray); } 

but there problem, logfiles can change. can see here, newer logfiles have got additional columns:

my current procedure doesn't work , i'm asking advice how this. performance important due fact, every logfile contains 1500 lines , 100 columns , logfiles merged one-year-period (equals 365 files).

i way: creating datatable, should contain data @ end , reading each logfile seperate datatable. after each read operation add seperate datatable "big" datatable, check if columns have changed , add new columns if did.

but i'm afraid using datatables affect performance.

note: i'm doing winforms, think doesn't matter anyway.

edit: tried csvreader 4 times slower current solution.

after hours of testing did way described in question:

firstly created datatable should contain data @ end. going through logfiles foreach-loop , every logfile create datatable , fill csv-data logfile. table gets added first datatable , no matter if have different columns, added properly.

this may costs performance compared simple streamreader, more easier extend , still faster lumenworks csvreader.