Big Data Archives - Page 8 of 23

How Ontraport reduced data processing cost by 80% with AWS Glue

This post is written in collaboration with Elijah Ball from Ontraport. Customers are implementing data Read More →

Introducing Apache Airflow version 2.6.3 support on Amazon MWAA

Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is a managed orchestration service for Apache Read More →

Perform Amazon Kinesis load testing with Locust

Building a streaming data solution requires thorough testing at the scale it will operate in Read More →

Monitor data pipelines in a serverless data lake

AWS serverless services, including but not limited to AWS Lambda, AWS Glue, AWS Fargate, Amazon Read More →

Configure SAML federation for Amazon OpenSearch Serverless with Okta

Modern applications apply security controls across many systems and their subsystems. Keeping all of these Read More →

Perform time series forecasting using Amazon Redshift ML and Amazon Forecast

Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud. Tens of Read More →

Configure cross-Region table access with the AWS Glue Catalog and AWS Lake Formation

Today’s modern data lakes span multiple accounts, AWS Regions, and lines of business in organizations. Read More →

Create an Apache Hudi-based near-real-time transactional data lake using AWS DMS, Amazon Kinesis, AWS Glue streaming ETL, and data visualization using Amazon QuickSight

With the rapid growth of technology, more and more data volume is coming in many Read More →

Estimating Scope 1 Carbon Footprint with Amazon Athena

Today, more than 400 organizations have signed The Climate Pledge, a commitment to reach net-zero Read More →

How FIS ingests and searches vector data for quick ticket resolution with Amazon OpenSearch Service

This post was co-written by Sheel Saket, Senior Data Science Manager at FIS, and Rupesh Read More →

Amazon Kinesis Data Streams on-demand capacity mode now scales up to 1 GB/second ingest capacity

Amazon Kinesis Data Streams is a serverless data streaming service that makes it easy to Read More →

Empower your Jira data in a data lake with Amazon AppFlow and AWS Glue

In the world of software engineering and development, organizations use project management tools like Atlassian Read More →

Migrate your existing SQL-based ETL workload to an AWS serverless ETL infrastructure using AWS Glue

Data has become an integral part of most companies, and the complexity of data processing Read More →

A side-by-side comparison of Apache Spark and Apache Flink for common streaming use cases

Apache Flink and Apache Spark are both open-source, distributed data processing frameworks used widely for Read More →

Extend your data mesh with Amazon Athena and federated views

Amazon Athena is a serverless, interactive analytics service built on the Trino, PrestoDB, and Apache Read More →

Simplify external object access in Amazon Redshift using automatic mounting of the AWS Glue Data Catalog

Amazon Redshift is a petabyte-scale, enterprise-grade cloud data warehouse service delivering the best price-performance. Today, Read More →

Five actionable steps to GDPR compliance (Right to be forgotten) with Amazon Redshift

The GDPR (General Data Protection Regulation) right to be forgotten, also known as the right Read More →

Use AWS Glue DataBrew recipes in your AWS Glue Studio visual ETL jobs

AWS Glue Studio is now integrated with AWS Glue DataBrew. AWS Glue Studio is a Read More →

Find the best Amazon Redshift configuration for your workload using Redshift Test Drive

Amazon Redshift is a widely used, fully managed, petabyte-scale cloud data warehouse. Tens of thousands Read More →

Near-real-time analytics using Amazon Redshift streaming ingestion with Amazon Kinesis Data Streams and Amazon DynamoDB

Amazon Redshift is a fully managed, scalable cloud data warehouse that accelerates your time to Read More →

End-to-end development lifecycle for data engineers to build a data integration pipeline using AWS Glue

Data is a key enabler for your business. Many AWS customers have integrated their data Read More →