52 posts tagged with "aws" | DevOpStar

52 posts tagged with "aws"

  September 16, 2024     11 min read

Why You Need to Care About Apache Iceberg Optimization

So you've started using Apache Iceberg and you're loving the features it offers. But have you thought about the cost and performance implications of using Iceberg? In this post we'll look at the new optimization features in AWS Glue that help you manage Iceberg tables.

awsathenaglueiceberg
  August 03, 2024     6 min read

Let's Try - Amazon Data Firehose Delivery to Apache Iceberg

A guide to setting up an Iceberg table in Amazon Athena, creating an Amazon Data Firehose delivery stream, and delivering data to the Iceberg table using a Preview feature in Amazon Data Firehose.

awsfirehoseglueiceberg
  July 27, 2024     17 min read

Federating S3 Access Grants with IAM Identity Center Trusted Identity Propagation

Explore how to federate S3 Access Grants using IAM Identity Center Trusted Identity Propagation in this in-depth guide. Learn to set up a secure and scalable solution for managing user and group-based permissions to S3 data across multiple client applications and AWS services. This post covers the necessary prerequisites, setting up an OAuth 2.0 authorizer, deploying S3 Access Grants and subsequent AWS IAM Identity Center applications with Terraform, and running a simple client application to interface with the federated authentication. Follow along to enhance your understanding of modern AWS security practices and ensure secure data access management.

awss3iamsecurityfederation
  July 11, 2024     7 min read

Let's Try - AWS Glue Apache Iceberg Statistics

Explore the newly supported AWS Glue table statistics feature for Apache Iceberg tables. This post provides a high-level overview of how to generate column-level aggregation statistics to significantly accelerate query performance. Follow along to set up your environment, create and load an Iceberg table, and generate valuable statistics that can optimize query execution.

awsathenaglueiceberg
  April 25, 2024     4 min read

Let's Try - CodeBuild Managed GitHub Action Runners

Let's try out the new AWS CodeBuild feature that allows you to run GitHub Actions inside AWS accounts using the same, familiar GitHub Actions workflows.

awscicdgithub
  March 16, 2024     5 min read

Using Claude 3 Haiku as an Eye of Sauron

Using the new Anthropic Claude 3 Haiku model hosted by Amazon Bedrock to watch a security camera feed and alert when something interesting happens.

awsbedrockclaudellm
  January 28, 2024     10 min read

S3 Batching - Drastically Simplifying Your IoT Architecture

IoT Core, Kinesis Data Streams, Firehose, IoT Rules and Lambda... the list of services that AWS Solution architects will recommend you use to build your IoT data ingestion pipelines. But what if I told you that you could replace all of these services with a single S3 bucket? Without sacrificing any abilities to process, analyze, or visualize your data.

awss3iotgreengrassathena
  December 11, 2023     11 min read

Let's Try - AWS S3 Access Grants

In this post, we'll look at the new AWS S3 Access Grants service and compare it to the traditional methods of managing access to S3 data such as S3 Access Points or IAM/Bucket policies. If you've struggle with massive policy documents in the past, or you're looking for a way to grant access to federated identities, then this post is for you!

awss3iamsecurity
  November 18, 2023     9 min read

Let's Try - AWS Glue Automatic Compaction for Apache Iceberg

In this post we'll take a look at how to use the new automatic compaction feature in AWS Glue, and how it can help you optimize your Iceberg tables. Use my handy helper script to create an Iceberg table, load sample data into it, and test out this new feature in a hands-on way.

awsathenaglueiceberg
  November 12, 2023     4 min read

Amazon Managed Grafana - Entra ID SAML Group Attribute

Walkthrough of how a groups attribute can be customized for Entra ID SAML integration with Amazon Managed Grafana, allowing for group based access control to Grafana.

awsgrafanaentra
  October 15, 2023     17 min read

Building a Garmin Dashboard - Automating Data Persistence with AWS SAM and Lambda

Automating the daily download of Garmin Connect data to S3 using AWS SAM and Lambda. The eventual goal is to build a dashboard to visualize our health data using AWS.

awslambdaaws-sams3
  September 25, 2023     9 min read

Let's Try - Amazon Athena Partition Projections

A quick look at how to use partition projections in Amazon Athena accompanied by some hands on examples that cover the basics through to more advanced use cases.

awsathenaglue
  July 28, 2023     5 min read

Vacuuming Amazon Athena Iceberg with AWS Step Functions

Resolving ICEBERG_VACUUM_MORE_RUNS_NEEDED errors by using AWS Step Functions to retry vacuuming Amazon Athena Iceberg tables until they are clean.

awsathenaicebergstep-functions
  July 25, 2023     15 min read

Let's Try - AWS Greengrass Testing Framework

A quick look at the AWS Greengrass Testing Framework by adding it into an existing Greengrass component to efficiently test how a component behaves when actually deployed.

awsiotgreengrasstesting
  June 25, 2023     9 min read

Small AWS Greengrass v2 tips for big observability gains

Improve your AWS Greengrass v2 observability by trying some of these simple tips. We explore the aws.greengrass.LocalDebugConsole and aws.greengrass.LogManager components.

awsiotgreengrass
  April 25, 2023     16 min read

Deploying Containers to AWS Greengrass v2 - A Comprehensive Guide

A complete guide to deploying containers to Greengrass v2

awsiotgreengrasscicdcontainers
  February 25, 2023     7 min read

Automatic API Key rotation for Amazon Managed Grafana

Amazon Managed Grafana has an unfortunate limitation where API keys created have a maximum expiration of 30 days - making continuous deployments difficult without manual intervention. This post will show you how to write some simple Terraform to automatically rotate an API key that can be used in your CI/CD pipeline.

awsgrafanaterraform
  February 05, 2023     12 min read

Sort AWS Config and Security Hub emails with Power Automate

Microsoft Power Automate can simplify the process of sorting incoming emails from AWS Config and AWS Security Hub. The tool can automate the sorting, categorizing and prioritizing of alerts, freeing up your time to focus on other important tasks.

security-hubawsaws-configpower-automate
  January 22, 2023     11 min read

GitHub Actions - AWS IoT build status light

Be notified when a GitHub Action build fails by illuminating an LED connected to a Raspberry Pi, controlled over AWS IoT

awsiotgithubcicd
  January 08, 2023     8 min read

Automate AWS OIDC Role Changes with GitHub Configurable OIDC Claims

A Demonstration of how you could utilize GitHub's configurable OIDC Claims to mostly automate AWS OIDC role changes.

githuboidcaws
  December 06, 2022     10 min read

Migrate to Amazon CodeCatalyst - GatsbyJS example

Building a simple Amazon CodeCatalyst workflow to deploy a GatsbyJS static site to s3. This guide is meant to facilitate how you might migrate an existing CI/CD pipeline to Amazon CodeCatalyst

awscodecatalystcicd
  September 22, 2022     11 min read

Using GitHub Actions for Greengrass v2 Continuous Deployment

Learn how to set up a Continuous deployment pipeline for AWS IoT Greengrass v2 from GitHub actions. Leverage the power of the Greengrass Development Kit - Command Line Interface (GDK CLI) to streamline your workflow and deploy more often with more confidence.

awsiotgreengrasscicd
  September 21, 2022     9 min read

Setting up an AWS IoT Greengrass v2 Learning environment

Learn how to set up a simple AWS IoT Greengrass v2 Learning environment that can be used in a wider range of tutorials to get you fully educated on how to use Greengrass on projects.

awsiotgreengrasscicd
  August 01, 2022     9 min read

Setup GitHub Codespaces with AWS IAM Roles Anywhere

Setup GitHub Codespaces with AWS IAM Roles Anywhere and authenticate your development environment with AWS securely without the need to have long-lived access tokens and ad-hoc user accounts.

codespacesawsgithubiam
  June 09, 2022     6 min read

Exporting Amazon WorkMail content programmatically

Export data from Amazon WorkMail programmatically using CloudFormation and the AWS CLI. No longer do you have to download some shady tool from the internet to export your data.

awsworkmail
  July 27, 2021     4 min read

LinkedIn Professional - Deploying a Serverless project

Learn how to deploy a simple python project to AWS Lambda using the Serverless Framework. We will use the LinkedIn Professional Jargon bot as a sample project.

awsserverlesslambda
  July 26, 2021     5 min read

LinkedIn Professional - Using LinkedIn's API

In this post we go through creating a small function that posts artifically generated status messages on LinkedIn. These messages are generated by taking trends from the ThoughtWorks technology radar and running sentences into DeepAI's text generator.

awsserverlesslambda
  December 16, 2020     8 min read

Feeding my cat with AWS IoT GreengrassV2

In this post we explore the new AWS IoT Greengrass V2 and create a recipe and component for controlling an IoT cat feeder.

awsiotgreengrass
  September 18, 2020     7 min read

Fighting API Abusers - Chaotic Good

When the services I maintain day to day come under attack by a bad actor I decide to take it personally. In this post I outline how you can fight back against sites that might be high-jacking your API for their own use uses.

awscloudformationwafapigateway
  August 22, 2020     5 min read

Amplify Flutter - Cat Image App

With the developer preview launch of AWS Amplify Flutter comes all the magic of Amplify Auth, storage, and analytics to the world of Flutter cross-platform development. Join me in this post as I stand up a simple cat photo app with this brand-new support

amplifyawsflutter
  May 16, 2020     9 min read

AWS IoT - ESP32-CAM Setup

Example for publishing images from an ESP32-CAM to AWS IoT over MQTT

iotawsespressif
  April 15, 2020     6 min read

Dog Bark Detector - Frontend

Building a Dog Bark Detector to help understand what my dogs do while I'm not home. In this post we cover how to deploy a simple frontend that leverages AppSync resolvers for DynamoDB and build a simple Vue dashboard for interacting with the bark events.

awsserverlesslambdaappsyncdynamodbs3
  April 14, 2020     9 min read

Dog Bark Detector - Serverless Audio Processing

Building a Dog Bark Detector to help understand what my dogs do while I'm not home. In this post we cover how to extract and process audio from video clips using a Serverless lambda processor and FFMPEG layers.

awsserverlesslambda
  April 13, 2020     11 min read

Dog Bark Detector - Machine Learning Model

Building a Dog Bark Detector to help understand what my dogs do while I'm not home. In this post we cover the training and deployment of a Machine Learning module using the UrbanSound8K dataset.

awsmachine-learningsagemaker
  March 31, 2020     7 min read

Kubeflow on EKS - Cognito Authentication

Kubeflow can better equips your Data science team with a self service access to all the resources they might need to build out Machine learning pipelines and applications. Pair this with Cognito and you have a secure way to work on data projects from anywhere in the world collaboratively.

kuberneteskubeflowawseks
  March 14, 2020     6 min read

Easy AWS IoT with ESP-IDF

Developing simple AWS IoT projects using the Espressif IoT Development Framework (ESP-IDF) and PlatformIO

awsiotespressif
  March 09, 2020     7 min read

Fighting COVID-19 with AWS CDK

Deploy Folding@Home on Amazon Fargate quick and easily with AWS CDK

awscdkfargate
  March 01, 2020     11 min read

So, you want to build an IoT Project?

Are you a beginner to developing for The Internet of Things? Perhaps you're an expert but are curious about what new things there are to focus on in? Let's chat about what it means to develop IoT projects in 2020.

iotaws
  February 24, 2020     22 min read

Certified Solutions Architect Professional - Study Notes

I recently passed my AWS Certified Solutions Architect Professional exam and along my journey of studying I scribbled down a bunch of notes I took. I thought it would be a good idea to share these random bits of information so that they can be used as flash cards / bite sized chunks of information

aws
  February 12, 2020     14 min read

The A to Z of Internet of Things (AWS)

Join us as we go through the A to Z of IoT from an AWS perspective. We won't be using buzzwords or hand-waving, just talking about the awesome services and how they can be used more effectively.

iotawsgreengrassdevice-defenderalexarobomaker
  February 08, 2020     9 min read

Amplify AppSync - AWS X-Ray Integration

AWS AppSync is a highly flexible API service backed with GraphQL. Recently X-Ray support was launched for AppSync, so we go through the setup process and some general usage patterns for this excellent service.

xrayappsyncamplifyaws
  January 25, 2020     16 min read

Alexa Gadgets Introduction - Voice Controlled Cat Feeder

When Amazon Alexa launched in 2014 as the first major home assistant platform I was captivated by the opportunities it offered someone like me. By the end of this article, you will have everything you need to build your gadget by using the source code I've provided to build this simple Alexa-powered cat feeder.

alexaawsiot
  December 25, 2019     11 min read

Greengrass - Secure Tunneling

AWS IoT Secure Tunneling is a managed proxy meant for devices positioned behind secure firewalls on remote sites. A secure tunnel can be created using temporary credentials allowing access to the device on configurable ports. The secure tunneling process requires a bidirectional link to be established before communication can proceed.

awsgreengrassiot
  December 14, 2019     9 min read

Greengrass - Docker Deployments

AWS IoT Greengrass recently got Docker support; a much needed addition to the suite of edge tooling available at our disposal. As with most new IoT Greengrass feature, it can be a bit bumped to get setup with. I would know, I fought through the setup process on the day of release and ran into a couple quirks that I figure I'll share with the world in the form of an unofficial guide.

awsgreengrassiot
  November 24, 2019     9 min read

Greengrass - Device Defender - Detect

Dive into Detect; a sub feature of Device Defender Detect that allows you to craft your own detection profiles using both cloud-side and device-side metrics.

awsdevice-defenderiotgreengrass
  November 24, 2019     6 min read

Greengrass - Device Setup

Deploy a learning environment called Greener Grass that effectively boostraps all the AWS IoT and Greengrass configuration for you. This allows you to be able to learn about the advanced IoT features in a friendly & repeatable way.

awsgreengrassiot
  November 24, 2019     5 min read

Greengrass - Device Setup - Raspberry Pi

For this deployment we'll be describing how to setup Greengrass Core on a Raspberry Pi. This is part of a bigger post called Greener Grass.

awsgreengrassiot
  August 25, 2019     7 min read

Building Selfie2Anime - Iterating on an Idea

The development of Selfie2Anime has had its ups and downs. This post is an overview of the iterations we've made along this journey so far and what the plans are for the future

awsgangkelambdas3serverless
  July 14, 2019     3 min read

Amazon EventBridge - Slack Notification on Event

WA Animals Amazon EventBridge rule to alert Slack with a Serverless lambda when someone signs in to AWS Account

awseventbridgelambdaserverless
  January 24, 2019     11 min read

Managing AWS Fargate with CloudFormation Nested Stacks

Fargate is an AWS offering that allows developers to worry even less about the systems their code runs on. Using CloudFormation you can orchestrate scalable containerised services entirely as code.

awsci-cdcloudformationfargate
  January 12, 2019     8 min read

Create a Private VPN using AWS IoT Button, SNS & CloudFormation

Deploy a Private VPN to EC2 by Triggering a CloudFormation deploy using your AWS IoT Button. Finally have your VPN Credentials sent to you via SNS.

awscloudformationiotsns
  January 06, 2019     5 min read

Deploying a Private VPN to AWS EC2 using CloudFormation

Deploy your very own Private PPTP VPN to AWS EC2 using CloudFormation. Learn how to deploy using AWS CLI or the CloudFormation GUI

awscloudformation
devopstar

DevOpStar by Nathan Glover | 2025