Learn how to programmatically upload CSV files through the API for data ingestion using the standard Linux command Curl or an HTTP client written in Go.

Although you can select “Upload and Index” a file from the Splunk GUI to upload and index a file, there are advantages to having an API endpoint that allows the same functionality.

There are a few CLI functions that can be used to perform one-time bulk loads of data however we are not going to cover CLI examples here, we are going to focus on using the Splunk API.

Scenarios when this method might…

ECR Scan Details

I didn’t realize how many security vulnerabilities my Docker image had until AWS ECR scanned the image. Thankfully I was able to remove all of them by moving to distroless images.

Initially, I started with a very popular Linux distribution as a base image thinking that was a good starting point, resulting in 6 critical and 538 other vulnerabilities on a 318MB image.

Excel at using CloudWatch Logs Insights by sending in structured JSON logs.

We are going to cover:

  • Why structured logs are important
  • How does JSON fit in
  • Web server logging example with GoLang
  • CloudWatch Log Insights walkthrough

Why are structured logs important?

It is about saving time in the future and improving analytics.

I have started outputting logs in JSON format because I don’t have time for anything else. I found that producing structured logs can save countless hours or days later on and takes little to no upfront effort. So I have decided to write logs for the computers, not the humans.

Anyone who…

Learn how to use SSH to traverse network boundaries and proxy SSH connections through other hosts.

Early in my career, I was fortunate enough to be introduced to SSH and all the extra things you can do with it besides opening a terminal on a remote host.

One of the features I find myself using frequently is the ProxyCommand. The man page states:

Specifies the command to use to connect to the server

Using this directive in your ~/.ssh/config file you can route SSH connections to remote hosts through other machines on the network. …

In late 2020, Docker announced changes to their resource consumption limits and pricing models. Now, since the original announcement was made, there has been changes based on community feedback, see the links below.

Docker links:

As a consumer of Docker Hub, the download limits and pricing changes present new challenges such as:

  1. What pricing model do I pick if I only pull in images like DataDog in AWS ECS tasks for side car containers?
  2. What pricing model do I pick if my CI/CD tool pulls in images during tests?
  3. How do I manage Docker Hub credentials across my organization? …

Michael Gale

Cloud Architect and Go enthusiast

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store