Steve McGhee

Are you going to Amazon Web Services (AWS) re:invent in few weeks? I’m excited to join Antony Vance, Principal Engineer at Intel, and Jon Handler,Senior Principal Solutions Architect at AWS, where we’ll demystify GenAI and empower you to build and deploy cutting-edge applications. In our session, we’ll explore the Open Platform for Enterprise AI (OPEA), a collaborative framework that simplifies GenAI deployments. You'll also learn how to optimize Retrieval Augmented Generation (RAG) pipelines on AWS, leveraging the power of Intel® Xeon® processors and the OpenSearch Project.​ Register for this session to learn how Intel and OPEA are shaping the future of AI.​ Register here: https://lnkd.in/gsUUt6n7 ​ #IAmIntel #AWS #reInvent #GenAI  

40

3 Comments

Folks I've spoken to about Amazon have likely heard me talk about the importance of the PR/FAQ process, a.k.a. "the working backwards process". This blog posts gives a very nice example of a PR/FAQ. https://lnkd.in/edTeuVRQ. Note that the blog doesn't capture the presumably 100s of reviews/discussions/rewrites that this doc represents, which is a part of the real value. When the process is done it all looks so simple, but getting a large organization to make a decision to something simple and do it in a BIG way is the challenge the fun of it all.

63

3 Comments

This is a notable increase in the number of S3 buckets that a single account can have, and I could see this being a welcome improvement for folks that use a lot of S3 buckets. It should be mentioned that with this change comes the possibility of a new fee, $0.02 per additional general purpose bucket (when you have more than 2,000 S3 buckets in your account). So you still probably want to be smart about the number of S3 buckets you create. #aws #s3 #cloudstorage #awspricing https://lnkd.in/eQFNd48c

3

AWS re:Invent is less than two weeks away. Want to accelerate your AI initiatives? Let's connect in Vegas and discuss how Meibel enables your engineering team to build Explainable AI into your products and experiences.

60

✈️ Wrapping up at AWS Re:invent today. Spent half the day yesterday walking the exhibit floor. The main purpose was to observe traffic hot spots and booth placement on the floor, to get at sense of which areas folks are interested in, and how important of a partner these companies were to AWS. Here were my high-level observations: -- startups that had the most traffic/attention: Anthropic, Unstructured, some of the code gen players like Poolside, Codieum, etc. Glean’s booth had a massive amount of traffic. Poolside had a massive booth relative to their company stage (maybe owing to the amount of compute they spend with AWS). The Anthropic booth was very nice and centrally placed on the floor. They seem increasingly tied to the hip with Amazon. -- startups that weren’t getting a lot of traffic: MLOps, RAG-as-a-service, vector DBs, “build you own agent” platforms. Not sure if this reflects overall market interest or simply the attendee profile at this conference. Also could be that their booth positioning wasn’t that great – many of these startups may not be major AWS customers or could even compete with AWS in certain areas. -- overall, I felt there was still a massive discrepancy in crowd size and interest between incumbents and startups. One reason might be that most people who come to Re:invent come from larger enterprises and are either less familiar with startups or don’t align well with startup’s ICPs. This was noticeably different from other AI conferences I’ve been to such as Databricks + ai summit or NeurIPS. In terms of specific things from AWS: -- Bedrock got a lot of new upgrades including multimodal processing, graphRAG, and structured data retrieval. Also, announced some features around multi-agent orchestration, distillation, and prompt caching. It appears that they’re increasingly consolidating the middle layer. -- They’re also pushing Bedrock Guardrails a lot as it was mentioned during the keynote and by separately by folks I spoke with.  For those unfamiliar, the Guardrail platform does a few things such as using automated reasoning to check against factual errors and hallucination, and filtering toxicity and PII. As these things get rolled out into production, I see trust and safety becoming increasingly important so I see why they’re pushing this. -- They also announced some cool things around RPA with Amazon Q. They showed a demo where Q Business was able to create an automated workflow simply by watching a video recording of how it was done. Under the hood, this is probably leveraging Computer Use recently announced by Anthropic and incorporated into Claude 3.5. My overall feeling was that the middle layer is increasingly going to be owned by the CSPs like Amazon as they continuing to push up the stack. I expect acquisitions to happen in the next few years in this layer, but perhaps not at the valuations that investors/founders were originally hoping for.

108

14 Comments

On Wednesday, GitHub and JFrog announced a collaboration to integrate their platforms, making it simpler for developers and their support teams to handle source code and binaries across both services. This features single sign-on, role mapping, code tracing from source to binary packages across platforms, and a unified project architecture. Later, a unified dashboard will display GitHub and JFrog’s source- and binary-focused security scan findings. #github #jfrog #githubcopilot #eltrys ELTRYS. GitHub JFrog

5

1 Comment

Sometimes small changes are a big deal. S3's per-account bucket limit has meant that buckets have been more of a human provisioning thing, starkly different from objects, which tend to be interacted with primarily from code. For years, customers have been asking us to increase this limit -- and in the storage innovation talk at re:Invent this year, i'd tell a bit of a story about why it was a more complicated bit of work than you might expect. With this limit increase, buckets have become a much more programmatic construct. We hear customers talking about using buckets, with these limit increases, as a way of managing data sharing with their own customers and using them to more effectively isolate data for specific jobs and teams. If larger numbers of buckets opens up a new pattern for you, I'd love to hear more about it. These launches, like object consistency, S3 conditionals, and big bucket limit increases have a tendency to fly a little below radar. But they are the ones that I actually hear the most excitement back from developers about, because they really broaden S3 as a programmatic interface to your data, and a building block for really incredible systems. I'm so impressed with the hard and careful work on the team that went in to getting this one out. https://lnkd.in/gA2cWATM

245

8 Comments

Well done #AWS for promptly fixing an S3 billing edge-case! Interesting S3 cost issue in this blog: "How an empty S3 bucket can make your AWS bill explode". In brief: external unauthorised (denied) requests to a new empty S3 bucket were being charged - $1,300 within two days. The author had inadvertently chosen the same bucket name that was the placeholder default configuration in a popular open-source tool!    https://lnkd.in/eNEqedR8 That blog was published on 29th April 2024. On 13th May, AWS fixed this problem: "Amazon S3 will no longer charge for several HTTP error codes". https://lnkd.in/eMXuhhu8 To quote that announcement: "bucket owners will never incur request or bandwidth charges for requests that return an HTTP 403 (Access Denied) error response if initiated from outside their individual AWS account or AWS Organization."

6

This is the initial infrastructure Tradescoop was built on. A few things have changed, some additions and removals; for example, we’ve gone from MongoDB on Digital Ocean to Mongo Atlas, Redis cloud to Memorystore, but the general appraoch remains. Simplicity. Shouts to Karl Obinna Amalu, CSM for cleaning it up and adding color ✨ #gcp #infrastructure #devops

19

1 Comment

Primitives or abstractions? I had the chance to pose this question to Matt Garman, CEO of AWS, at re:Invent last week. His response was clear: AWS is doubling down on both. They’re investing in innovating and owning key primitives such as foundation models, infrastructure, and chips, while also building powerful abstractions—services that enable developers to run diverse workloads and create innovative applications with industry-leading price-performance. Against the backdrop of thousands of attendees buzzing about all things AI, AWS once again delivered one of the best industry conferences. Here are my top 5 insights from the event.

18

4 Comments

First story on what we have been up to at Augment. Much more to come! https://lnkd.in/gC6gbADA

370

64 Comments

It was somewhat scary to take on the role of WW Head of EPIC at AWS. I had to say goodbye to my stable role as WW Business Development Manager of RDS/Aurora (the fastest growing service in AWS history) to move to this radically different, untested role. It was a totally new role I'd created, and I had no idea if and how it would work out! I had to embrace uncertainty. Do you enjoy uncertainty or avoid it like the plague? It turns out that embracing uncertainty can be really good for you. We all have different relationships with it, but in the business world, we tend to index towards trying to make things as predictable as possible. We do things like strategic planning, forecasting, modeling, predictive analysis, etc. to make things more “certain”—and there are obvious advantages to that. Yet the reality is that life (and work) are filled with uncertainties…and probably more than we want! If we can learn to embrace the inevitable uncertainties that come at us, we will be healthier, more successful, and happier along the way. That was actually one of the main points in my talk on "Emotionally Intelligent Leadership for an AI-powered Future" at AWS ExecLeaders London this week. If there is one thing we know about the AI future—it’s that no one really knows what it's going to look like. As best-selling author and business guru Jim Collins said, "Uncertainty is chronic, instability is permanent, disruption is common, and we can neither predict nor govern events. There will be no new normal, there will only be a continuous series of not normal episodes, defying prediction, and unforeseen by most of us until they happen." Welcome to the world of continuous uncertainty, instability, and disruption! Learn more in this article, How Embracing Uncertainty Can Improve Your Life: https://lnkd.in/gPNv4-_R How do you feel about uncertainty and ambiguity? What has helped you have a healthier relationship with these realities? I'd love see in the comments below. #uncertainty #ambiguity, #complexity #success #emotionalintelligence #epicleadership #ai

124

4 Comments

Large Language Models (#LLMs) are evolving rapidly, and the demand for running them at scale is skyrocketing. #Ollama is stepping up to meet this need with its official container image on Docker Hub, excelling in local #deployments. However, there’s a catch for containerized environments: downloading #models requires the ollama serve command to be running within the #container. This extra step complicates automated deployments via #CI #CD #pipelines, making seamless model access challenging. To take your #model to production, it needs to be #container-ready—a feature Ollama has yet to fully support. The gap in #AIOps for smooth deployments gets blurred with the inference services available as an easy way to take it to production. Goal is to overcome obstacles and deliver solutions 😉 so here is to get you started on having your (O)#llama #production ready in a private #deployment from your #local to #cloud all in #oneshot . Enjoy Medium: https://lnkd.in/g9jqBQ8d dev.to: https://lnkd.in/gi_gGwam #AI #LLM #Ollama #llama3 #phi3 #mistral #S6overlay #AIOps #CI #CD #oneshot #container #docker #podmanEnjoy

11

AWS Lambda pioneered the serverless revolution. This document is like a time machine to its origins. Serverless is hugely popular, but in my opinion still heavily underutilized as a computing paradigm. I've personally had great success building applications on serverless and have recommended it for most applications.

7

Wow! One of my favorite episodes yet! What an amazing opportunity to be joined by Kristina D. for an in-depth look at optimizing Kubernetes for sustainability! Check out this episode and subscribe wherever you get your podcasts! #ncast #sustainability #karpenter #cloudoptimization #kubernetes

18

1 Comment

Can you trust AI? 🤔 For most companies the answer is no - at least using current foundational models out of the box. ❌ Problems like sensitive data leakage and toxic content are major barriers to implementing production genAI solutions. The route towards trust lies in system architecture and the good news is the field is advancing rapidly, including increasingly easily accessible toolkits of solutions to increase trust in AI systems. 🔥 This is the hot area for new releases from major AI platform players (along with AI security), including AWS (https://lnkd.in/e45S-ehS), DataRobot (https://lnkd.in/eCWvppWR), and Databricks (https://lnkd.in/e6A7cTcH). 👨‍⚖️ Guardrail or judge models - leveraging a second AI model to police the content coming out of an original - are probably the most promising scalable solution. For many the best route is likely to be a new class of foundational models specifically designed for this task such as Llama Guard and Databrick’s new judge LLM. There is also the ability to fine tune a model (think a discriminator model from a GAN architecture) or use a general purpose foundational model. 👮‍♂️ Classic filtering techniques for sensitive data can also be applied to the result of an LLM, whether for PII, company IP, or inappropriate content. This can include regex, classic PII models (such as Amazon Comprehend), etc. 🚻 Human in the loop techniques can also be used, but are far more scalable when used to calibrate automated solutions such as those described above rather than manually verifying outputs in realtime. What’s still missing? 🔎 🎯 While techniques to improve accuracy and reduce hallucination such as RAG and AI agents have made great strides, there remains a big gap in conveying a model’s confidence in its output to end users. 💡One idea is to leverage the probability distributions used for top-p (nucleus sampling) at the token level instead at greater aggregations to convey confidence levels for entire model outputs or significant segments (e.g. each step in a chain of thought structure or AI agents framework). #ai, #artificialintelligence, #GenAI, #GenerativeAI, #llm, #datascience, #machinelearning, #mlops, #bigdata

5

1 Comment

The Site Reliability Engineering: How Google Runs Production System book is arguably one of the most prevalent and well-known SRE resources out there, but there are many other great options for folks looking to expand on their learning. We recently spoke to Google's Reliability Advocate, Steve McGhee, in our Humans of Reliability interview series. In addition to his interesting anecdotes on the early days of SRE at Google, and his journey to becoming a Reliability Advocate, he also shared a handful of his favorite SRE resources, which we compiled here into a list: https://lnkd.in/gpUVhyRa

16

*Piling on!* Data scientists build the models, not software engineers. Software architects figure out how to enable the deployment of AI capabilities to users… I’m gonna keep shouting from the roof tops: Groundbreaking potential with limitations and challenges!!! Data scientists build the models, not software engineers. Software architects figure out how to enable the deployment of AI capabilities to users… We’ve got to respect the data scientists and data engineers… they are the ones that actually know when and how these should be used. We need to be professionally “married.” “AI washing” our industry puts us at risk. We need to stop posing that we are the AI experts and listen to those who are. </rant> #SoftwareArchitecture #ArchAITecture #GenAI #DataScience

17

5 Comments