Explore updates to Bedrock FedRAMP, Meta Llama 3.1, SRT Support, and DMS Introduction In the ever-evolving landscape of cloud computing, AWS continues to innovate, delivering new features and updates that enhance the capabilities and reliability of their services. This month brings a host of exciting developments across various AWS offerings. From improved data replication for […]
Introduction In this blog post we will introduce vector databases and some of the algorithms used for indexing and show examples of how to work with AWS OpenSearch vector database using python and LangChain library. Vector databases A vector database is a type of database that stores high-dimensional vectors for fast retrieval and similarity search. […]
First things first, what is data quality monitoring? Data quality monitoring for machine learning can generally be thought of from two perspectives. One perspective is that of traditional data-engineering. This type of monitoring is concerned with the “physical” characteristics of the data and ensuring they are what you expect them to be. It involves criteria […]
Introduction Data pipelines in AWS orchestrate the movement and transformation of data across various AWS services. The core objective of these pipelines is to enable efficient data processing, analysis, and storage, ensuring that data is available where and when it is needed. Maintaining high data quality throughout this process is critical; it ensures reliability, accuracy, […]
The final installment of our blog series on AWS testing methodologies focuses on integration testing. This crucial phase ensures that all components of your application work together seamlessly in a live environment, simulating real-world usage with production code and test data. Below is an outline designed to guide the creation of a comprehensive and informative […]
A deep dive into functional testing for AWS development Introduction In our exploration of advanced testing techniques for AWS development, we’ve delved into powerful tools like moto for unit testing and pytest.mark.parametrize for enhancing test coverage and efficiency. Building on this foundation, we turn our focus to a pivotal tool that bridges the gap between […]
Leveraging Moto and Pytest Introduction In the world of AWS development, ensuring the reliability, efficiency, and correctness of your cloud-based applications is paramount. As cloud solutions grow increasingly complex, so too does the challenge of effectively testing these systems. Traditional testing methods often fall short in the face of AWS’s vast and intricately interconnected services. […]
Introduction In the realm of distributed computing and batch processing, operational challenges frequently arise that necessitate innovative solutions. A particular challenge we encountered involved a scenario where multiple jobs within our AWS environment were generating tens of thousands of files and storing them in an Amazon S3 bucket. Subsequently, a specific job was tasked with […]
Introduction In the digital era, where data is king, the choice of table format for data storage and processing is crucial. Common file formats like CSV, Avro, and Parquet have long been the go-to solutions in various data handling scenarios. However, with the evolving needs of big data and cloud computing, newer and more efficient […]
From hiring decisions to loan approvals and even healthcare recommendations, machine learning (ML) impacts our lives daily. Fairness and explainability are crucial in this context. Fairness means data is balanced, and model predictions are fair across groups. Checking for fairness ensures that negative outcomes are fair across all groups, such as age or gender. Explainability […]
- 1
- 2