How to monitor AWS S3 cloud services

What is AWS S3?

Amazon Simple Storage Service or AWS S3 is a service offered by Amazon Web Services that provides object storage through a web service interface. Amazon S3 can be employed to store any type of object, which allows for uses like storage for Internet applications, backup and recovery, disaster recovery, data archives, data lakes for analytics, and hybrid cloud storage. The basic storage units of Amazon S3 are objects which are organized into buckets. Each object is identified by a unique, user-assigned key.

AWS S3 instances monitoring

This package automatically configures the ServicePilot Agent in order to monitor all AWS-S3 instances in an AWS region. Bucket size and object count is obtained at a 24 hour interval. API usage filters are collected every minute.

The monitoring statistics gathered in this way include:

• Per S3 bucket size and object count
• Per S3 bucket filter API usage

Requirements

• Network Flows - It is is necessary to ensure network connectivity between ServicePilot and the monitored device.

  • TCP/443 (HTTPS): Between ServicePilot Agent and AWS CloudWatch API servers

Installation

Before adding a resource to monitor, make certain that all pre-requisites are in place.

Note the AWS S3 monitoring documentation.

Follow the documentation to add a resource to ServicePilot.

Key field notes

1. General tab:

   1. Service IP address for ServicePilot Host licensing: An IP address of the AWS CloudWatch API service (monitoring.*.amazonaws.com) must be specified here if you plan to use a ServicePilot Host license for this resource
   2. Region: AWS region name
   3. Access key: AWS access key with read permissions to S3 CloudWatch data
   4. Secret key: AWS secret key with read permissions to S3 CloudWatch data
   5. Use HTTP Proxy: Use an HTTP Proxy to access AWS CloudWatch API service
   6. HTTP Proxy: HTTP Proxy to access AWS CloudWatch API service

2. Monitoring Options tab:

   1. Storage: Obtain daily S3 bucket size and number of objects
   2. Connection: Obtain per bucket, per filter S3 API usage. Note that these statistics are chargable by AWS.

3. In the Policies tab, specify the policy or policies to apply to the resource

License

Notes

AWS access keys are needed for programmatic calls to AWS. See AWS credentials to generate keys and make sure they have the permissions required to query AWS CloudWatch APIs for S3. AWS managed AmazonS3ReadOnlyAccess and CloudWatchReadOnlyAccess permissions policies may be used.

CloudWatch metrics are not available instantly via the CloudWatch API. There will be a delay until the metrics become available based on your monitoring subscription level. CloudWatch API usage incurs cost — see GetMetricData Pricing.

How to install a aws-s3 resource?

1. Use your ServicePilot OnPremise installation or a SaaS account.
2. Add a new aws-s3 resource via the web interface (/prmviews or /prmresources) or via API (/prmpackages page), the default ServicePilot agent or another agent will be provisioned automatically.

Details of the aws-s3 package are located in the /prmpackages page of the software.

Benefits

ServicePilot enables you to deliver IT services faster and more securely with automated discovery and advanced monitoring features.

By correlating the technology AWS S3 with APM and infrastructure monitoring, ServicePilot is able to provide a more comprehensive view of an organization's IT environment.

This allows IT teams to quickly identify and diagnose issues that may be impacting application performance, and take corrective action before end-users are affected.

Start with a free trial of our SaaS solution. Explore ourplans or contact us to find what works best for you.