How to improve security for Amazon S3 buckets Computer Engineer

Overview

Since 2006, Amazon Web Services (AWS) has provided object storage to make web-scale computing easier. This service enables organizations of any size or industry to store large troves of data for websites, mobile applications, disaster recovery, and whatever else the business requires.

Recently, Hobby Lobby, a prolific American arts and crafts enterprise had 138GB of data, including payment card info and physical addresses, plucked out of an open Amazon S3 bucket. Events like this impact the entire organization—including developers, who are responsible for remediating misconfigurations.

This article explores how to avoid post-deployment headaches by increasing the security of your Amazon S3 buckets and the objects stored within during the early phases of development.

The shared responsibility model

As with all cloud environments, you’re responsible for what you store in it. This is part of the shared responsibility model—meaning that the cloud service provider (CSP) is responsible for the overall security of the infrastructure that runs all of the services, but the user is responsible for securing any data or objects within that environment.

Seems a little vague, right? The level of responsibility assigned to customers and CSPs based on the type of cloud service being consumed across software as a service (SaaS), platform as a service (PaaS), and infrastructure as a service (IaaS). Here’s an example of the shared responsibility model for AWS:

As a user of Amazon S3, it is your responsibility to consider the following security requirements:

• Define the least privileged access to the bucket and continuously review those permissions across all the buckets
• Enable encryption
• Enable data recovery to help meet compliance requirements
• Enable protection of overwritten objects
• Define tags for better labeling, collecting, and organizing resources available within your AWS environment
• Enable “Block Public Access” for buckets that should never be public
• Ensure Amazon S3 buckets are enforcing secure sockets layer (SSL) to secure data in transit
• Ensure the logging access is enabled to track access requests
• Validate that the information being stored is safe and does not contain malicious code hidden as malware or ransomware

You may have a couple of questions: How do you audit and validate that these security policies and controls are implemented across all your Amazon S3 buckets? Do users have visibility of any changes that might occur within this and other cloud services that would indicate a policy and/or compliance violation? How can you remediate a violation without disrupting your workflow? How do you integrate an extra layer of security to scan for malicious content coming into your Amazon S3 bucket?

These are great questions to ask, and we have some insights for you. Let’s dive into some tools and solutions that can provide you with maximum visibility into your cloud risk posture as well as remediation advice.

What’s “shift left”?

All of our suggestions enable your security approach to “shift left.” Essentially, this means moving your security scans, audits, or thingamajigs to the front of your pipeline. The benefits of catching security issues at the onset are huge: it helps save time, money, and it reduces risks to the business. By introducing security checks and validation at the first step in the infrastructure build process (IaC templates), you can reduce friction for the development and operations team—who doesn’t want that?!

This all sounds great in theory, but you may be wondering what exactly are you looking for, or what should you be keeping an eye on? Misconfigurations are the number one risk to cloud environments. Therefore, you should pay extra attention to monitoring for any possible errors.

This may sound like a lot of work that eats time from building, but there’s cloud security posture management (CSPM) tools that can help monitor for misconfigurations in real-time across all your Amazon S3 buckets and other AWS services. It’s ideal that the CSPM you choose embraces the “shift-left” security approach by integrating into the infrastructure as code (IaC) with AWS CloudFormation templates. This will allow you to identify and detect any possible misconfigurations in the earliest stage of development.

Okay, now to the good stuff—here’s how a CSPM tool such as Trend Micro Cloud One™ can help you shift-left and effectively secure your Amazon S3 buckets so it’s smooth sailing from build time to run time.

FYI—If you are interested in creating your environment, here is a Git repository with CloudFormation to automate this process for you: https://github.com/fernandostc/AWS_IaC_pipeline_with_Security

Integrated Development Environment (IDE) security plugin

You can streamline the entire audit process by choosing a CSPM that uses an IDE security plug-in. This is designed to quickly get real-time feedback for developers in the IaC template, so they can scan and fix issues in their current IDE workspace as early as possible. By doing so, developers can prevent misconfigurations across different AWS services and build in accordance with the AWS Well-Architected Framework.

Below is an example of a template scanning report generated by a security plugin, which shows potential risks found in your pipeline. This helps you identify what needs to be fixed before you start building with it in production.

Link to VSCode plugin: Cloud Conformity Template Scanner Extension

How do template scanners work?

Think of template scanning as body scanners at the airport—it’s similar in the way it provides enhanced visibility into any risks or threats that may not be caught with the human eye. Template scanning is especially necessary if you use open source code repositories to build (which 90% of developers do, according to Gartner).

Template scanners use powerful APIs within your CSPM tool to provide automated, real-time checks every time you push a new template. It also shares the results with developers and cloud architects, so they can investigate any potential issues before production. Auto-remediation allows you to continue building at lightning speed. For example, you can configure settings so that if the scan finds an “Extreme” or “High-risk” issue for example, the CSPM will stop the deployment of the new infrastructure and notify the development team through Slack, Jira, ServiceNow, PagerDuty, and other third-party notification tools.

The example below shows how you can detect any misconfiguration on Amazon S3 buckets during the CI/CD pipeline before you build the CloudFormation template in your AWS account:

Introducing Trend Micro Cloud One™ – Conformity

As part of the Trend Micro Cloud One™ platform, Trend Micro Cloud One™ – Conformity is a CSPM solution that seamlessly integrates into your CI/CD pipeline to detect misconfiguration in multiple CSPs. It’s designed to overcome any visibility or security risk challenges by running auto-checks against hundreds of cloud infrastructure configuration best practices and compliance standards including PCI-DSS, HIPAA, HITRUST, NIST-800-53, and more. The solution also ensures fast remediation by providing instant alerts and remediation steps when critical misconfigurations are detected.

With Conformity, organizations receive a comprehensive picture of security and compliance risks across all cloud environments. Below is an example of the insight Conformity provides your Amazon S3 buckets after checking them against the AWS Well-Architected Framework, compliance standards, and other best practice guidelines.

Reminder: Double-check what’s going in your Amazon S3 buckets

But wait—there’s more! As we mentioned earlier, it’s critical to scan what is going into your bucket as well. By scanning any file before its uploaded, you can prevent malware from entering your organization and impacting downstream workflows or infecting external web applications.

Trend Micro Cloud One™ – File Storage Security complements Conformity by making sure the files going inside the bucket are safe as well as helping you stay compliant by keeping your files and data within your AWS account during scanning.

Below is a diagram showing the journey of a file through File Storage Security before receiving the stamp of approval.

Next, it’s important to have a post-scan game plan for flagged files. By setting up post-scan actions, File Storage Security will automate the incident and response use cases for possible malicious objects. Post-scan actions can include quarantining the file in your account but away from your application or terminating the file outright. Here is an example of scan results of a file—the “malicious” tag will trigger the appropriate post-scan action.

You also have centralized visibility about the number of objects you have scanned and how many those files have been recognized as malicious. Here is an example of such on the dashboard in Trend Micro Cloud One:

By automating the file scanning process, you’re eliminating the possibility of human interaction which in return increases the level of security and compliance within your Amazon S3 buckets. Some other benefits of File Storage Security are:

• Improve file reputation: Block bad files using Trend Micro anti-malware signatures on all types of malware, including viruses, Trojan, spyware, and more
• Variant protection: Look out for obfuscated or polymorphic variants of malware via fragments of previously seen malware and detection algorithms
• Extensive flexibility: Trusted scanning support for all file sizes and types, including .BIN, .EXE, .JPEG, .MP4, .PDF, .TXT, .ZIP, and more

Demo: Happy Path

In this real-life example, we’ll put all the pieces together and show you how Conformity and File Storage Security can apply to your web applications using AWS services.

James Beswick, principal developer advocate for the AWS Serverless Team, demonstrated how to create a new web application called Happy Path using AWS Lambda, Amazon S3 buckets, AWS Step Functions, and other AWS services.  Check out the backend architecture below or view the Git repository here. 

Source: https://aws.amazon.com/blogs/compute/using-serverless-backends-to-iterate-quickly-on-web-apps-part-2/

Imagine you have a fully automated IaC pipeline to build the Happy Path architecture and want to ensure its as secure as possible before its deployed. You can improve this environment by integrating Conformity (CSPM with IDE plugin) to check how it stacks up against the AWS Well-Architected Framework and if its meeting compliance. This will give you real-time feedback about any possible updates or improvements you can implement on your cloud infrastructure. See below:

To take Happy Path to the next level security-wise, you need to integrate additional protection for the files being uploaded into the Amazon S3 bucket. Unlike other AWS services, Trend Micro Cloud One solutions, Conformity and File Storage Security, detect misconfigurations of the Amazon S3 buckets, and the files uploaded within it before an event is triggered by Lambda. This is important because if the file is unscanned before moved elsewhere by a Lambda function, it could spread malicious malware throughout your entire infrastructure.

The grey box in the diagram below demonstrates where you could integrate File Storage Security so that any new object is automatically scanned and if tagged as malicious (or your tag of choice) will be moved to a quarantine bucket. Alternatively, if the file is determined to be safe, it will move to a promote bucket. By integrating File Storage Security into the Happy Path architecture, you can ensure that your application is processing safe documents and objects before they’re shared with your customer or partners.