So you've started using Apache Iceberg and you're loving the features it offers. But have you thought about the cost and performance implications of using Iceberg? In this post we'll look at the new optimization features in AWS Glue that help you manage Iceberg tables.
A guide to setting up an Iceberg table in Amazon Athena, creating an Amazon Data Firehose delivery stream, and delivering data to the Iceberg table using a Preview feature in Amazon Data Firehose.
Explore how to federate S3 Access Grants using IAM Identity Center Trusted Identity Propagation in this in-depth guide. Learn to set up a secure and scalable solution for managing user and group-based permissions to S3 data across multiple client applications and AWS services. This post covers the necessary prerequisites, setting up an OAuth 2.0 authorizer, deploying S3 Access Grants and subsequent AWS IAM Identity Center applications with Terraform, and running a simple client application to interface with the federated authentication. Follow along to enhance your understanding of modern AWS security practices and ensure secure data access management.
Explore the newly supported AWS Glue table statistics feature for Apache Iceberg tables. This post provides a high-level overview of how to generate column-level aggregation statistics to significantly accelerate query performance. Follow along to set up your environment, create and load an Iceberg table, and generate valuable statistics that can optimize query execution.
Let's try out the new AWS CodeBuild feature that allows you to run GitHub Actions inside AWS accounts using the same, familiar GitHub Actions workflows.
Using the new Anthropic Claude 3 Haiku model hosted by Amazon Bedrock to watch a security camera feed and alert when something interesting happens.
IoT Core, Kinesis Data Streams, Firehose, IoT Rules and Lambda... the list of services that AWS Solution architects will recommend you use to build your IoT data ingestion pipelines. But what if I told you that you could replace all of these services with a single S3 bucket? Without sacrificing any abilities to process, analyze, or visualize your data.
In this post, we'll look at the new AWS S3 Access Grants service and compare it to the traditional methods of managing access to S3 data such as S3 Access Points or IAM/Bucket policies. If you've struggle with massive policy documents in the past, or you're looking for a way to grant access to federated identities, then this post is for you!
In this post we'll take a look at how to use the new automatic compaction feature in AWS Glue, and how it can help you optimize your Iceberg tables. Use my handy helper script to create an Iceberg table, load sample data into it, and test out this new feature in a hands-on way.
Walkthrough of how a groups attribute can be customized for Entra ID SAML integration with Amazon Managed Grafana, allowing for group based access control to Grafana.
Automating the daily download of Garmin Connect data to S3 using AWS SAM and Lambda. The eventual goal is to build a dashboard to visualize our health data using AWS.
A quick look at how to use partition projections in Amazon Athena accompanied by some hands on examples that cover the basics through to more advanced use cases.
Resolving ICEBERG_VACUUM_MORE_RUNS_NEEDED errors by using AWS Step Functions to retry vacuuming Amazon Athena Iceberg tables until they are clean.
A quick look at the AWS Greengrass Testing Framework by adding it into an existing Greengrass component to efficiently test how a component behaves when actually deployed.
Improve your AWS Greengrass v2 observability by trying some of these simple tips. We explore the aws.greengrass.LocalDebugConsole and aws.greengrass.LogManager components.
A complete guide to deploying containers to Greengrass v2
Amazon Managed Grafana has an unfortunate limitation where API keys created have a maximum expiration of 30 days - making continuous deployments difficult without manual intervention. This post will show you how to write some simple Terraform to automatically rotate an API key that can be used in your CI/CD pipeline.
Microsoft Power Automate can simplify the process of sorting incoming emails from AWS Config and AWS Security Hub. The tool can automate the sorting, categorizing and prioritizing of alerts, freeing up your time to focus on other important tasks.
Be notified when a GitHub Action build fails by illuminating an LED connected to a Raspberry Pi, controlled over AWS IoT
A Demonstration of how you could utilize GitHub's configurable OIDC Claims to mostly automate AWS OIDC role changes.
Building a simple Amazon CodeCatalyst workflow to deploy a GatsbyJS static site to s3. This guide is meant to facilitate how you might migrate an existing CI/CD pipeline to Amazon CodeCatalyst
Learn how to set up a Continuous deployment pipeline for AWS IoT Greengrass v2 from GitHub actions. Leverage the power of the Greengrass Development Kit - Command Line Interface (GDK CLI) to streamline your workflow and deploy more often with more confidence.
Learn how to set up a simple AWS IoT Greengrass v2 Learning environment that can be used in a wider range of tutorials to get you fully educated on how to use Greengrass on projects.
Setup GitHub Codespaces with AWS IAM Roles Anywhere and authenticate your development environment with AWS securely without the need to have long-lived access tokens and ad-hoc user accounts.
Export data from Amazon WorkMail programmatically using CloudFormation and the AWS CLI. No longer do you have to download some shady tool from the internet to export your data.
Learn how to deploy a simple python project to AWS Lambda using the Serverless Framework. We will use the LinkedIn Professional Jargon bot as a sample project.
In this post we go through creating a small function that posts artifically generated status messages on LinkedIn. These messages are generated by taking trends from the ThoughtWorks technology radar and running sentences into DeepAI's text generator.
In this post we explore the new AWS IoT Greengrass V2 and create a recipe and component for controlling an IoT cat feeder.
When the services I maintain day to day come under attack by a bad actor I decide to take it personally. In this post I outline how you can fight back against sites that might be high-jacking your API for their own use uses.
With the developer preview launch of AWS Amplify Flutter comes all the magic of Amplify Auth, storage, and analytics to the world of Flutter cross-platform development. Join me in this post as I stand up a simple cat photo app with this brand-new support
Example for publishing images from an ESP32-CAM to AWS IoT over MQTT
Building a Dog Bark Detector to help understand what my dogs do while I'm not home. In this post we cover how to deploy a simple frontend that leverages AppSync resolvers for DynamoDB and build a simple Vue dashboard for interacting with the bark events.
Building a Dog Bark Detector to help understand what my dogs do while I'm not home. In this post we cover how to extract and process audio from video clips using a Serverless lambda processor and FFMPEG layers.
Building a Dog Bark Detector to help understand what my dogs do while I'm not home. In this post we cover the training and deployment of a Machine Learning module using the UrbanSound8K dataset.
Kubeflow can better equips your Data science team with a self service access to all the resources they might need to build out Machine learning pipelines and applications. Pair this with Cognito and you have a secure way to work on data projects from anywhere in the world collaboratively.
Developing simple AWS IoT projects using the Espressif IoT Development Framework (ESP-IDF) and PlatformIO
Deploy Folding@Home on Amazon Fargate quick and easily with AWS CDK
Are you a beginner to developing for The Internet of Things? Perhaps you're an expert but are curious about what new things there are to focus on in? Let's chat about what it means to develop IoT projects in 2020.
I recently passed my AWS Certified Solutions Architect Professional exam and along my journey of studying I scribbled down a bunch of notes I took. I thought it would be a good idea to share these random bits of information so that they can be used as flash cards / bite sized chunks of information
Join us as we go through the A to Z of IoT from an AWS perspective. We won't be using buzzwords or hand-waving, just talking about the awesome services and how they can be used more effectively.
AWS AppSync is a highly flexible API service backed with GraphQL. Recently X-Ray support was launched for AppSync, so we go through the setup process and some general usage patterns for this excellent service.
When Amazon Alexa launched in 2014 as the first major home assistant platform I was captivated by the opportunities it offered someone like me. By the end of this article, you will have everything you need to build your gadget by using the source code I've provided to build this simple Alexa-powered cat feeder.
AWS IoT Secure Tunneling is a managed proxy meant for devices positioned behind secure firewalls on remote sites. A secure tunnel can be created using temporary credentials allowing access to the device on configurable ports. The secure tunneling process requires a bidirectional link to be established before communication can proceed.
AWS IoT Greengrass recently got Docker support; a much needed addition to the suite of edge tooling available at our disposal. As with most new IoT Greengrass feature, it can be a bit bumped to get setup with. I would know, I fought through the setup process on the day of release and ran into a couple quirks that I figure I'll share with the world in the form of an unofficial guide.
Deploy a learning environment called Greener Grass that effectively boostraps all the AWS IoT and Greengrass configuration for you. This allows you to be able to learn about the advanced IoT features in a friendly & repeatable way.
Dive into Detect; a sub feature of Device Defender Detect that allows you to craft your own detection profiles using both cloud-side and device-side metrics.
For this deployment we'll be describing how to setup Greengrass Core on a Raspberry Pi. This is part of a bigger post called Greener Grass.
The development of Selfie2Anime has had its ups and downs. This post is an overview of the iterations we've made along this journey so far and what the plans are for the future
WA Animals Amazon EventBridge rule to alert Slack with a Serverless lambda when someone signs in to AWS Account
Fargate is an AWS offering that allows developers to worry even less about the systems their code runs on. Using CloudFormation you can orchestrate scalable containerised services entirely as code.
Deploy a Private VPN to EC2 by Triggering a CloudFormation deploy using your AWS IoT Button. Finally have your VPN Credentials sent to you via SNS.
Deploy your very own Private PPTP VPN to AWS EC2 using CloudFormation. Learn how to deploy using AWS CLI or the CloudFormation GUI