Entity Framework (EF) Slow Insert and SqlBulkCopy to the Rescue

There was a need to insert bulk data into the database and we are using Entity Framework as our ORM strategy. However, bulk insert took forever. Disabling Auto Detect Changes helps a little. So I decided to look into SqlBulkCopy and ended up writing one for Entity Framework. It is generic enough and can be found in https://gist.github.com/nikyodo85/b82ffd56bb2f0d45a9860dadcdfdc01d.

It works well so far. Some of the drawbacks are:

  • It won't be able to auto insert relationship
  • No validation check
But very fast. To use it will be very similar to SqlBulkCopy:


Dim efSqlBulkCopy As New EFSqlBulkCopy(Of MyEntityClass)(myDbContext)
efSqlBulkCopy.WriteToServer(listOfMyEntityClass)

Comments

Post a Comment

Popular posts from this blog

Sentinel One Strikes Again. No internet connection. Uninstall Sentinel One Agent.

This happened to a co-worker of mine a while back when his test application file was marked as suspicious by Sentinel One antivirus and had his internet on his laptop disabled. Today, it happened to me without any suspicious file. Probably suspicious activity, who knows. On Microsoft Edge, it says "Hmmm... your Internet access is blocked.", "Firewall or antivirus software may have blocked the connection", and "ERR_NETWORK_ACCESS_DENIED". So, I worked with my IT to uninstall the agent, but uninstalling is not without a fight. Here are the steps that I took: Since it is a Windows 11 machine with Bitlocker, I have to first get the Bitlocker key. From command prompt run: manage-bde -protectors -get C: After I verified it is the same key that the IT has, I saved the key outside of the machine. Then go to system configuration by searching for "sysconfig" or run msconfig. Under "boot" tab, check the "Safe boot" option, then click ...
Read more

Error When Generating OpenAPI Documents: Missing required option '--project'

After I installed Microsoft.Extensions.ApiDescription.Server package, I encountered the following error message when I attempted to generate OpenAPI documents at build-time on .NET 9. Missing required option '--project' The command "dotnet "..."" exited with code 1 Apparently, it was due to end slash on my attempt to change the output directory. On my csproj file, I have the following entry: <PropertyGroup> <OpenApiDocumentsDirectory>../directory/</OpenApiDocumentsDirectory> </PropertyGroup> It works correctly after I removed the end slash: <PropertyGroup> <OpenApiDocumentsDirectory>../directory</OpenApiDocumentsDirectory> </PropertyGroup>
Read more

Swagger .NET 8 Error

Swashbuckle CLI was able to output schema of my API before, but this time, it throws this error message: System.InvalidOperationException: A type named 'StartupProduction' or 'Startup' could not be found in assembly I used top level statement with minimal API on .NET 8 and nothing is changed on that, so I was not able to find anything to do with Startup type. After I investigate further by commenting line by line, I found out that the issue is on my switch statement. So it looks like the following: return config.Section?.Key switch { Value1 => services.AddSingleton<Handler1>(), Value2 => services.AddSingleton<Handler2>(), _ => throw new InvalidOperationException(); } Problem is the Section is pulled from appsettings.json and when the CLI runs, it doesn't have value, so it never returned the services object. Changing the above to the following fixed the issue: return config.Section == null ? services : config.Section.Key switch { Value1 =...
Read more