Nikki's Lunch

 I had the issue where Lambda function launched in private network times out when trying to get object from S3 bucket. Typically, there are two solutions: Use S3 VPC endpoint (either gateway or interface) since it resolves s3 endpoint to private IP. Attach public IP. This is done using NAT Gateway with Elastic IP (EIP). The problem is, in my case, the S3 bucket is in different region, different account, than the Lambda function while the first solution, even though S3 is a global service, the VPC endpoint can't resolve to S3 in different region. In short, the first solution only works when S3 bucket and Lambda function are in the same region. That left us with solution 2 which is more expensive but works. Also I need to make sure that the S3 bucket policy allows cross account access.

So, I have to launch Opensearch in ECS. And I need to add persistent storage. The container ran fine but it threw AccessDeniedException. And even though the container ran, my application was unable to connect to it.  After few tries, I found out that it is due to the permission of the directory where the data are supposed to reside. The container runs in ECS on EC2. The path, in this case, I use /usr/share/opensearch/data on EC2 is owned by root, but the container runs as ec2-user. So, I had to update the user data field on the launch template (since I used ASG) to include the following commands: mkdir -p /usr/share/opensearch/data sudo chown 1000:1000 /usr/share/opensearch/data That fixed the exception and the reachability issue.

This happened to a co-worker of mine a while back when his test application file was marked as suspicious by Sentinel One antivirus and had his internet on his laptop disabled. Today, it happened to me without any suspicious file. Probably suspicious activity, who knows. On Microsoft Edge, it says "Hmmm... your Internet access is blocked.", "Firewall or antivirus software may have blocked the connection", and "ERR_NETWORK_ACCESS_DENIED". So, I worked with my IT to uninstall the agent, but uninstalling is not without a fight. Here are the steps that I took: Since it is a Windows 11 machine with Bitlocker, I have to first get the Bitlocker key. From command prompt run: manage-bde -protectors -get C: After I verified it is the same key that the IT has, I saved the key outside of the machine. Then go to system configuration by searching for "sysconfig" or run msconfig. Under "boot" tab, check the "Safe boot" option, then click ...

After I installed Microsoft.Extensions.ApiDescription.Server package, I encountered the following error message when I attempted to generate OpenAPI documents at build-time on .NET 9. Missing required option '--project' The command "dotnet "..."" exited with code 1 Apparently, it was due to end slash on my attempt to change the output directory. On my csproj file, I have the following entry: <PropertyGroup> <OpenApiDocumentsDirectory>../directory/</OpenApiDocumentsDirectory> </PropertyGroup> It works correctly after I removed the end slash: <PropertyGroup> <OpenApiDocumentsDirectory>../directory</OpenApiDocumentsDirectory> </PropertyGroup>

I found a Logitech mouse and keyboard combo on clearance. The model is MK470 and it looks returned. For the steep discounted price, I decided to give a try. Expectedly, it didn't work, so that starts my troubleshooting. Battery is fine, no on/off button on keyboard, both mouse and keyboard are not working, no sign of damage, dongle is properly inserted into the USB port. Short while later, I found that Logitech has a neat Connection Utility software . I downloaded it and ran it twice, once to reconnect the mouse and once for the keyboard. My guess is the frequency and channel somehow was not lining up between the mouse and keyboard and the dongle. The previous buyer probably returned it because they were not working. But the connection is finally restored.

I was exploring AWS Cognito for authentication. It works great, but I got the following error message after I tested the sign up process: An error was encountered with the requested page. I found out later that I misunderstood the AutoVerifiedAttributes field in my CloudFormation. I thought it would mark an email or phone number as verified without actually verifying them. Apparently, it means it will try to verify either email or phone number. So, when I set it to email, it sent a verification email and the sign up process went without error.

I encountered a strange error with ASP.NET Web API application. It runs fine locally, but when we deployed to Kubernetes cluster, it crashed as soon as it starts. And no error message was thrown. So, I pulled the application to my local and it crashed as well no matter how I run it, dotnet cli, Docker Desktop, Visual Studio debug. The only one that runs fine is the version from the repo. At this point, there are only two possibilities, either the environment is the issue or the application is the issue, so I decided to deploy it to a different environment and it's still not working, so it must be something with the application. Since it is the application, I tried to change the log level to Trace to get more information but no new error message that provides a hint on what's going on. Memory dump didn't work as the collector didn't have enough time to collect before the application crashed. At the end, I decided to approach this the hard way. So, in my local, there are ...

 It never caused a problem for me to implement JwtBearer token validator, but this time it is really take my time to troubleshoot what's going on. Long story short, there's a breaking change going to .NET 8 and on top of that, the default package version doesn't solve the issue. Here's how I implement my service: services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme) .AddJwtBearer(JwtBearerDefaults.AuthenticationScheme, options => ...removed for brevity); services.AddAuthorization(); ... app.UseAuthentication(); app.UseAuthorization(); But checking the bearer token, it was a completely valid token. I retrieved the token using a quick custom middleware. app.Use(async (context, next) => { await next.Invoke(); Debug.WriteLine(context.Request.Headers.Authorization); }); app.UseAuthentication(); ... Then I validate the token in  https://jwt.io . The error that I received contains: IDX14100: JWT is not well formed, there are no dots (.). Th...

Swashbuckle CLI was able to output schema of my API before, but this time, it throws this error message: System.InvalidOperationException: A type named 'StartupProduction' or 'Startup' could not be found in assembly I used top level statement with minimal API on .NET 8 and nothing is changed on that, so I was not able to find anything to do with Startup type. After I investigate further by commenting line by line, I found out that the issue is on my switch statement. So it looks like the following: return config.Section?.Key switch { Value1 => services.AddSingleton<Handler1>(), Value2 => services.AddSingleton<Handler2>(), _ => throw new InvalidOperationException(); } Problem is the Section is pulled from appsettings.json and when the CLI runs, it doesn't have value, so it never returned the services object. Changing the above to the following fixed the issue: return config.Section == null ? services : config.Section.Key switch { Value1 =...

I realized that I don't have the Layout option under Styles menu in the Editor. I found out later that I have a bare minimum theme.json. And adding appearanceTools: true field cause it to show up. My theme.json became: { "version": 3, "$schema": "https://schemas.wp.org/wp/6.6/theme.json", "settings": { "appearanceTools": true } }

 I'm working on a custom WordPress theme and I saw a very helpful plugin called "Create Block Theme" which is supposed to help developer create the theme. So, for starting, I tried to edit one of my templates, but when I hit "Save Changes" under "Save Changes to Theme" section, I expected it to overwrite my template html file, but it didn't. I was checking permissions and potential bugs, but seems like everything is good. After playing around a little, apparently, I need to hit "Save" first, so it records the customization in the database and then click the "Save Changes to Theme" will use the value from the database and modified the html file itself.

Often my branches piled up and I need a way to automatically delete them based on how many days ago. I can't find an easy way online, so I ended up writing my own scripts in PowerShell and Bash. In this repo, I have the script to clean up branches that are 90 days or older based on last committed date. Repo:  https://github.com/nik-yo/DeleteGitBranchesByDaysAgo

 Due to the distributed nature of Apache Spark, when writing result, we can't specify name for the result file. This makes the result file hard to predict which I need for my process orchestration. In my case, I need to write the result to S3 and I finally found a way to do this within a reasonable amount of time by utilizing aws wrangler, Panda, and optionally Arrow. I basically feed Spark dataframe to aws wrangler and have it write to S3 using a specific name. Here's link to my sample:  https://github.com/nik-yo/PySparkFilename

I'm working on a demo where I need to subscribe my application to Kafka locally in Docker Desktop. I have 3 use cases: Connecting from a different container in Docker desktop, so in the same network as the Kafka container. Connecting from the application running on the host, so outside of Docker desktop for debugging purposes. Connecting from a pod inside Minikube running in Docker Desktop. Same Docker Network On the first case, I actually need to connect AKHQ container to Kafka, my Kafka container env variable for advertised listeners looks like the following: environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092 , ... (removed for brevity) Since AKHQ running in Docker desktop as well, it can use kafka:29092. From the Host (my computer) Outside of Docker Network Next is my application that runs outside of Docker desktop, since it won't resolve the kafka host, it has to use the 2nd entry of the advertised listener. In my case, I had to change the port from 9092 t...

 In my worker app, I attempted to add multiple hosted services as follow: builder.Services .addSingleton(HostedService1) .addSingleton(HostedService2) .addSingleton(HostedService3); All the hosted services are added, but when the application run, only 1 is executing. Thanks to Stephen Cleary, apparently issue with synchronous call.  https://blog.stephencleary.com/2020/05/backgroundservice-gotcha-startup.html . I ended up using Task.Run for code that executes for a long time. Inside the ExecuteAsync: await Task.Run(async () => await LongRunningProcess());

I was looking for a way to use two git accounts in a single machine. Apparently, there are multiple ways to do that: Use different protocols for different accounts. One account can use HTTP, another account uses SSH. Use different SSH keys. One per account. Use HTTP and PAT. This might be GitHub specific but per my experience with PAT in Azure DevOps, this is not feasible as PAT has expiration and needs to be renewed.  https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-your-personal-account/managing-multiple-accounts I ended up using different protocols since that will save me effort in configuring one of the accounts. First, I need to create an ssh key.  https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#generating-a-new-ssh-key . Since I'm in Windows, I can use Git Bash. Launch Git Bash Run: ssh-keygen -t ed25519 -C "your_email@example.com...

 This is what I heard from my junior after he moved to a different company. They found an issue with data transfer cost due to internal ALB and NAT Gateway in the same subnet. Apparently, the internal application sends data to the ALB and it's being processed by the NAT gateway as well. I'm not exactly sure how it works, but it was a bad networking. They removed the NAT gateway and just let the ALB managed the traffic and save cost. 

 Ok, this is a rookie issue. I had virtual environment created, activated and the package installed for that virtual environment, but somehow I bumped into this error: Import could not be resolved [Pylance] Turns out to be the interpreter is pointing to the wrong one in my VS code bottom right. Changing it to the one in virtual environment fixed that.

 I have Python 3.12 installed on my Windows 10 machine. I tried to install a package using Pip. Apparently, the package contains Cython and needs to be compiled. However, the compilation failed with the following message: Cannot open include file: 'io.h': No such file or directory Ok, not a problem, I just go to Visual Studio Installer and install Desktop development with C++ package. That fixed the first issue. But installing the package still failed. This time the error message is: 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.41.34120\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2 To fix the above, I had to downgrade python to 3.10 and the package is installed properly.

 Most of the time we have to type fast, especially in today's world where speed is life. So, we are bound to mistype. In Linux, one of the commonly used command is "ls", so to "train" user to correct that, an "sl" command is created. SL stands for Steam Locomotive. Check it out in your Linux distro, search, install, and run the command and it will show a moving locomotive.

My outlook somehow keeps reminding me on old meetings that occurred weeks before. Dismissing all or each one doesn't work. It keeps coming back. I tried the suggestion to remove cache, clear reminders and none works. Finally, the only one that works for me is to open up the details of each one of the old meetings from the reminder and then dismiss them. They never showed up in the reminder ever since.

 I was looking for a budget friendly way to receive email on my custom domain. So, let say, I own example.com and I want to receive email on receive@example.com.  As I did my research, I found various way on doing it: Just forward it . My domain name vendor apparently comes with free email forwarder, so I forward it to my non-custom domain email such as gmail. Just forward it (DIY) . This is also a very cheap alternative and low cost. One way is to forward it through AWS SES. One such project is:  https://github.com/arithmetric/aws-lambda-ses-forwarder Receive it through hosting . I thought about this especially when I already paid for hosting service, usually it comes with mail server for free. Of course, we can always subscribe to some email service, but it will cost more but it has more features too.

 I have an old Samsung Galaxy Note 4 that was not turning on for a long time now. Out of curiosity, I read about it came back to life by putting it in the freezer, so I gave it a try. I took out the battery, put the phone in a ziploc bag and put in the freezer for at least 8 hours (so I can sleep or work through it). I also make sure the battery is charged separately since I have a battery charger. After 8 hours, I take it out, put the battery in, and surprised that it turns on. However, it won't turn on anymore after I turn it off, so I placed it into the freezer the second time and it works again. My guess is the freezer probably takes some humidity out from the components and allows it to work better.

 I finally got some time to get back to mobile development after many years. And Android has a new way to create an app with Jetpack Compose. At a glance, it is amazing, I managed to create a complex app much faster than using XAML, yep, you read that right, that's how I used to do it. All is well until I encountered infinite loop when trying to remove item in a mutableList displayed using LazyColumn on a button click. Basically, the button click somehow causing a recomposition and then the recomposition retrigger the button click event again and again. But it only happened when I remove an item, adding an item is fine. Here's the example of initial code: data class Pet(var timestamp: Instant, var name: String) @Composable fun Screen() { val pets = remember { mutableStateListOf<Pet>() } fun addPet() { pets.add(Pet(Clock.System.now(), "Pochi")) if (pets.count() > 5) { pets.removeAt(0) } } addPet() ...

One day, mapped drives to my NAS stopped working and the NAS itself just disappeared from my network. My first thought was either the NAS broke or my router. But my router seems fine, so I first checked my NAS by directly connecting to it via ethernet cable to my laptop (using USB converter). I also downloaded the Synology Assistant software which helps a lot in finding whether there's Synology NAS in the network. The Synology Assistant can be found in Synology Download Center under Desktop Utilities. Synology NAS model required to find the right software. https://www.synology.com/en-us/support/download My NAS was working well, so I decided to reboot my router. After the router reboots, I detached the NAS from the laptop and connect it back to the router. And my NAS is discoverable in the network again. However, it happened again when I transferred a large amount of files. Probably it overwhelms the router as I use an old Netgear Wifi 5 router. There's also a possibility that f...

 I was trying to copy only certain files within a directory to Azure Storage Account instead of the whole directory content. The files that I tried to copy are those ends with .zip, .tag.gz, and .py. AzCopy support wildcard on the source, so I would like to do something like this: azcopy copy C:\{directory}\[*.zip|*.tar.gz|*.py] ... I found out later that there's --include-pattern option, so this works: azcopy copy C:\{directory}\* --include-pattern *.zip;*.tar.gz;*.py

I'm trying to spin up a Redis Cache in Azure and placed it in my Virtual Network. However, it threw an error and the error message was not helping but I eventually figured it out. Here's my Bicep template which threw an error: resource cacheSubnet 'Microsoft.Network/virtualNetworks/subnets@2023-11-01' existing = { name: 'mySubnet' } resource redis 'Microsoft.Cache/redis@2023-08-01' = { name: 'myRedis' location: resourceGroup().location properties: { enableNonSslPort: true publicNetworkAccess: 'Disabled' sku: { capacity: 1 family: 'P' name: 'Premium' } subnetId: cacheSubnet.id } Apparently, I needed the parent field on the subnet reference, so I ended with the following template which successfully launched my Redis cache. resource vNet 'Microsoft.Network/virtualNetworks@2023-11-01' existing = { name: 'myVNet' } resource cacheSubnet 'Microsoft.Network/virtualNetworks/subnets@2023-11-01' existing = { n...