// Add OLE DB Destination similarly... pipeline.ReinitializeMetaData(); pkg.Save();
SSIS‑965 – “Data Flow task fails with The data type of the column is unknown ” TL;DR – SSIS‑965 is a long‑standing “metadata‑loss” bug that appears when a Flat File Source (or OLE DB Source ) is used together with dynamic column discovery in a Data Flow that is later reused by a Script Component or Derived Column . The root cause is the way the SSIS runtime caches the metadata of the source at design‑time but discards it at run‑time when the Connection Manager is refreshed with a new schema. The fix is to force a metadata refresh (re‑initialise the component) or, better, to decouple schema discovery from the data flow by using a staging table or Data Flow parameters . Below is a step‑by‑step forensic analysis, a reproducible test case, the official Microsoft KB work‑around, a clean‑room implementation that eliminates the issue, performance considerations, and a checklist for preventing the bug in future projects. 1. Background & Why It Matters SQL Server Integration Services (SSIS) is the ETL engine for the Microsoft data‑platform. A huge proportion of SSIS packages are data‑flow‑centric – they read from a source, perform transformations, and write to a destination. SSIS-965
contains an additional column Region at the end: // Add OLE DB Destination similarly
$schema | ConvertTo-Json -Depth 3 | Set-Content -Path "$FilePath.schema.json" Write-Host "Schema written to $FilePath.schema.json" using Microsoft.SqlServer.Dts.Runtime; using Microsoft.SqlServer.Dts.Pipeline.Wrapper; using System.IO; using Newtonsoft.Json.Linq; The fix is to force a metadata refresh
$firstLine = Get-Content -Path $FilePath -TotalCount 1 $headers = $firstLine -split $Delimiter
Funded by the European Union, under Grant Agreement N° 101135323. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or REA. Neither the European Union nor the granting authority can be held responsible for them.