Podcasts about amazon sqs

On this episode of The Cloud Pod, the team sits to talk about AWS's new patching policies, the general availability of Azure OpenAI, and the role of addressing IM or access management challenges in ensuring the seamless transition to the Cloud. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

[00:00.000 --> 00:04.560] All right, so I'm here with 52 weeks of AWS[00:04.560 --> 00:07.920] and still continuing to do developer certification.[00:07.920 --> 00:11.280] I'm gonna go ahead and share my screen here.[00:13.720 --> 00:18.720] All right, so we are on Lambda, one of my favorite topics.[00:19.200 --> 00:20.800] Let's get right into it[00:20.800 --> 00:24.040] and talk about how to develop event-driven solutions[00:24.040 --> 00:25.560] with AWS Lambda.[00:26.640 --> 00:29.440] With Serverless Computing, one of the things[00:29.440 --> 00:32.920] that it is going to do is it's gonna change[00:32.920 --> 00:36.000] the way you think about building software[00:36.000 --> 00:39.000] and in a traditional deployment environment,[00:39.000 --> 00:42.040] you would configure an instance, you would update an OS,[00:42.040 --> 00:45.520] you'd install applications, build and deploy them,[00:45.520 --> 00:47.000] load balance.[00:47.000 --> 00:51.400] So this is non-cloud native computing and Serverless,[00:51.400 --> 00:54.040] you really only need to focus on building[00:54.040 --> 00:56.360] and deploying applications and then monitoring[00:56.360 --> 00:58.240] and maintaining the applications.[00:58.240 --> 01:00.680] And so with really what Serverless does[01:00.680 --> 01:05.680] is it allows you to focus on the code for the application[01:06.320 --> 01:08.000] and you don't have to manage the operating system,[01:08.000 --> 01:12.160] the servers or scale it and really is a huge advantage[01:12.160 --> 01:14.920] because you don't have to pay for the infrastructure[01:14.920 --> 01:15.920] when the code isn't running.[01:15.920 --> 01:18.040] And that's really a key takeaway.[01:19.080 --> 01:22.760] If you take a look at the AWS Serverless platform,[01:22.760 --> 01:24.840] there's a bunch of fully managed services[01:24.840 --> 01:26.800] that are tightly integrated with Lambda.[01:26.800 --> 01:28.880] And so this is another huge advantage of Lambda,[01:28.880 --> 01:31.000] isn't necessarily that it's the fastest[01:31.000 --> 01:33.640] or it has the most powerful execution,[01:33.640 --> 01:35.680] it's the tight integration with the rest[01:35.680 --> 01:39.320] of the AWS platform and developer tools[01:39.320 --> 01:43.400] like AWS Serverless application model or AWS SAM[01:43.400 --> 01:45.440] would help you simplify the deployment[01:45.440 --> 01:47.520] of Serverless applications.[01:47.520 --> 01:51.960] And some of the services include Amazon S3,[01:51.960 --> 01:56.960] Amazon SNS, Amazon SQS and AWS SDKs.[01:58.600 --> 02:03.280] So in terms of Lambda, AWS Lambda is a compute service[02:03.280 --> 02:05.680] for Serverless and it lets you run code[02:05.680 --> 02:08.360] without provisioning or managing servers.[02:08.360 --> 02:11.640] It allows you to trigger your code in response to events[02:11.640 --> 02:14.840] that you would configure like, for example,[02:14.840 --> 02:19.200] dropping something into a S3 bucket like that's an image,[02:19.200 --> 02:22.200] Nevel Lambda that transcribes it to a different format.[02:23.080 --> 02:27.200] It also allows you to scale automatically based on demand[02:27.200 --> 02:29.880] and it will also incorporate built-in monitoring[02:29.880 --> 02:32.880] and logging with AWS CloudWatch.[02:34.640 --> 02:37.200] So if you look at AWS Lambda,[02:37.200 --> 02:39.040] some of the things that it does[02:39.040 --> 02:42.600] is it enables you to bring in your own code.[02:42.600 --> 02:45.280] So the code you write for Lambda isn't written[02:45.280 --> 02:49.560] in a new language, you can write things[02:49.560 --> 02:52.600] in tons of different languages for AWS Lambda,[02:52.600 --> 02:57.600] Node, Java, Python, C-sharp, Go, Ruby.[02:57.880 --> 02:59.440] There's also custom run time.[02:59.440 --> 03:03.880] So you could do Rust or Swift or something like that.[03:03.880 --> 03:06.080] And it also integrates very deeply[03:06.080 --> 03:11.200] with other AWS services and you can invoke[03:11.200 --> 03:13.360] third-party applications as well.[03:13.360 --> 03:18.080] It also has a very flexible resource and concurrency model.[03:18.080 --> 03:20.600] And so Lambda would scale in response to events.[03:20.600 --> 03:22.880] So you would just need to configure memory settings[03:22.880 --> 03:24.960] and AWS would handle the other details[03:24.960 --> 03:28.720] like the CPU, the network, the IO throughput.[03:28.720 --> 03:31.400] Also, you can use the Lambda,[03:31.400 --> 03:35.000] AWS Identity and Access Management Service or IAM[03:35.000 --> 03:38.560] to grant access to what other resources you would need.[03:38.560 --> 03:41.200] And this is one of the ways that you would control[03:41.200 --> 03:44.720] the security of Lambda is you have really guardrails[03:44.720 --> 03:47.000] around it because you would just tell Lambda,[03:47.000 --> 03:50.080] you have a role that is whatever it is you need Lambda to do,[03:50.080 --> 03:52.200] talk to SQS or talk to S3,[03:52.200 --> 03:55.240] and it would specifically only do that role.[03:55.240 --> 04:00.240] And the other thing about Lambda is that it has built-in[04:00.560 --> 04:02.360] availability and fault tolerance.[04:02.360 --> 04:04.440] So again, it's a fully managed service,[04:04.440 --> 04:07.520] it's high availability and you don't have to do anything[04:07.520 --> 04:08.920] at all to use that.[04:08.920 --> 04:11.600] And one of the biggest things about Lambda[04:11.600 --> 04:15.000] is that you only pay for what you use.[04:15.000 --> 04:18.120] And so when the Lambda service is idle,[04:18.120 --> 04:19.480] you don't have to actually pay for that[04:19.480 --> 04:21.440] versus if it's something else,[04:21.440 --> 04:25.240] like even in the case of a Kubernetes-based system,[04:25.240 --> 04:28.920] still there's a host machine that's running Kubernetes[04:28.920 --> 04:31.640] and you have to actually pay for that.[04:31.640 --> 04:34.520] So one of the ways that you can think about Lambda[04:34.520 --> 04:38.040] is that there's a bunch of different use cases for it.[04:38.040 --> 04:40.560] So let's start off with different use cases,[04:40.560 --> 04:42.920] web apps, I think would be one of the better ones[04:42.920 --> 04:43.880] to think about.[04:43.880 --> 04:46.680] So you can combine AWS Lambda with other services[04:46.680 --> 04:49.000] and you can build powerful web apps[04:49.000 --> 04:51.520] that automatically scale up and down.[04:51.520 --> 04:54.000] And there's no administrative effort at all.[04:54.000 --> 04:55.160] There's no backups necessary,[04:55.160 --> 04:58.320] no multi-data center redundancy, it's done for you.[04:58.320 --> 05:01.400] Backends, so you can build serverless backends[05:01.400 --> 05:05.680] that lets you handle web, mobile, IoT,[05:05.680 --> 05:07.760] third-party applications.[05:07.760 --> 05:10.600] You can also build those backends with Lambda,[05:10.600 --> 05:15.400] with API Gateway, and you can build applications with them.[05:15.400 --> 05:17.200] In terms of data processing,[05:17.200 --> 05:19.840] you can also use Lambda to run code[05:19.840 --> 05:22.560] in response to a trigger, change in data,[05:22.560 --> 05:24.440] shift in system state,[05:24.440 --> 05:27.360] and really all of AWS for the most part[05:27.360 --> 05:29.280] is able to be orchestrated with Lambda.[05:29.280 --> 05:31.800] So it's really like a glue type service[05:31.800 --> 05:32.840] that you're able to use.[05:32.840 --> 05:36.600] Now chatbots, that's another great use case for it.[05:36.600 --> 05:40.760] Amazon Lex is a service for building conversational chatbots[05:42.120 --> 05:43.560] and you could use it with Lambda.[05:43.560 --> 05:48.560] Amazon Lambda service is also able to be used[05:50.080 --> 05:52.840] with voice IT automation.[05:52.840 --> 05:55.760] These are all great use cases for Lambda.[05:55.760 --> 05:57.680] In fact, I would say it's kind of like[05:57.680 --> 06:01.160] the go-to automation tool for AWS.[06:01.160 --> 06:04.160] So let's talk about how Lambda works next.[06:04.160 --> 06:06.080] So the way Lambda works is that[06:06.080 --> 06:09.080] there's a function and there's an event source,[06:09.080 --> 06:10.920] and these are the core components.[06:10.920 --> 06:14.200] The event source is the entity that publishes events[06:14.200 --> 06:19.000] to AWS Lambda, and Lambda function is the code[06:19.000 --> 06:21.960] that you're gonna use to process the event.[06:21.960 --> 06:25.400] And AWS Lambda would run that Lambda function[06:25.400 --> 06:29.600] on your behalf, and a few things to consider[06:29.600 --> 06:33.840] is that it really is just a little bit of code,[06:33.840 --> 06:35.160] and you can configure the triggers[06:35.160 --> 06:39.720] to invoke a function in response to resource lifecycle events,[06:39.720 --> 06:43.680] like for example, responding to incoming HTTP,[06:43.680 --> 06:47.080] consuming events from a queue, like in the case of SQS[06:47.080 --> 06:48.320] or running it on a schedule.[06:48.320 --> 06:49.760] So running it on a schedule is actually[06:49.760 --> 06:51.480] a really good data engineering task, right?[06:51.480 --> 06:54.160] Like you could run it periodically to scrape a website.[06:55.120 --> 06:58.080] So as a developer, when you create Lambda functions[06:58.080 --> 07:01.400] that are managed by the AWS Lambda service,[07:01.400 --> 07:03.680] you can define the permissions for the function[07:03.680 --> 07:06.560] and basically specify what are the events[07:06.560 --> 07:08.520] that would actually trigger it.[07:08.520 --> 07:11.000] You can also create a deployment package[07:11.000 --> 07:12.920] that includes application code[07:12.920 --> 07:17.000] in any dependency or library necessary to run the code,[07:17.000 --> 07:19.200] and you can also configure things like the memory,[07:19.200 --> 07:23.200] you can figure the timeout, also configure the concurrency,[07:23.200 --> 07:25.160] and then when your function is invoked,[07:25.160 --> 07:27.640] Lambda will provide a runtime environment[07:27.640 --> 07:30.080] based on the runtime and configuration options[07:30.080 --> 07:31.080] that you selected.[07:31.080 --> 07:36.080] So let's talk about models for invoking Lambda functions.[07:36.360 --> 07:41.360] In the case of an event source that invokes Lambda function[07:41.440 --> 07:43.640] by either a push or a pool model,[07:43.640 --> 07:45.920] in the case of a push, it would be an event source[07:45.920 --> 07:48.440] directly invoking the Lambda function[07:48.440 --> 07:49.840] when the event occurs.[07:50.720 --> 07:53.040] In the case of a pool model,[07:53.040 --> 07:56.960] this would be putting the information into a stream or a queue,[07:56.960 --> 07:59.400] and then Lambda would pull that stream or queue,[07:59.400 --> 08:02.800] and then invoke the function when it detects an events.[08:04.080 --> 08:06.480] So a few different examples would be[08:06.480 --> 08:11.280] that some services can actually invoke the function directly.[08:11.280 --> 08:13.680] So for a synchronous invocation,[08:13.680 --> 08:15.480] the other service would wait for the response[08:15.480 --> 08:16.320] from the function.[08:16.320 --> 08:20.680] So a good example would be in the case of Amazon API Gateway,[08:20.680 --> 08:24.800] which would be the REST-based service in front.[08:24.800 --> 08:28.320] In this case, when a client makes a request to your API,[08:28.320 --> 08:31.200] that client would get a response immediately.[08:31.200 --> 08:32.320] And then with this model,[08:32.320 --> 08:34.880] there's no built-in retry in Lambda.[08:34.880 --> 08:38.040] Examples of this would be Elastic Load Balancing,[08:38.040 --> 08:42.800] Amazon Cognito, Amazon Lex, Amazon Alexa,[08:42.800 --> 08:46.360] Amazon API Gateway, AWS CloudFormation,[08:46.360 --> 08:48.880] and Amazon CloudFront,[08:48.880 --> 08:53.040] and also Amazon Kinesis Data Firehose.[08:53.040 --> 08:56.760] For asynchronous invocation, AWS Lambda queues,[08:56.760 --> 09:00.320] the event before it passes to your function.[09:00.320 --> 09:02.760] The other service gets a success response[09:02.760 --> 09:04.920] as soon as the event is queued,[09:04.920 --> 09:06.560] and if an error occurs,[09:06.560 --> 09:09.760] Lambda will automatically retry the invocation twice.[09:10.760 --> 09:14.520] A good example of this would be S3, SNS,[09:14.520 --> 09:17.720] SES, the Simple Email Service,[09:17.720 --> 09:21.120] AWS CloudFormation, Amazon CloudWatch Logs,[09:21.120 --> 09:25.400] CloudWatch Events, AWS CodeCommit, and AWS Config.[09:25.400 --> 09:28.280] But in both cases, you can invoke a Lambda function[09:28.280 --> 09:30.000] using the invoke operation,[09:30.000 --> 09:32.720] and you can specify the invocation type[09:32.720 --> 09:35.440] as either synchronous or asynchronous.[09:35.440 --> 09:38.760] And when you use the AWS service as a trigger,[09:38.760 --> 09:42.280] the invocation type is predetermined for each service,[09:42.280 --> 09:44.920] and so you have no control over the invocation type[09:44.920 --> 09:48.920] that these events sources use when they invoke your Lambda.[09:50.800 --> 09:52.120] In the polling model,[09:52.120 --> 09:55.720] the event sources will put information into a stream or a queue,[09:55.720 --> 09:59.360] and AWS Lambda will pull the stream or the queue.[09:59.360 --> 10:01.000] If it first finds a record,[10:01.000 --> 10:03.280] it will deliver the payload and invoke the function.[10:03.280 --> 10:04.920] And this model, the Lambda itself,[10:04.920 --> 10:07.920] is basically pulling data from a stream or a queue[10:07.920 --> 10:10.280] for processing by the Lambda function.[10:10.280 --> 10:12.640] Some examples would be a stream-based event service[10:12.640 --> 10:17.640] would be Amazon DynamoDB or Amazon Kinesis Data Streams,[10:17.800 --> 10:20.920] and these stream records are organized into shards.[10:20.920 --> 10:24.640] So Lambda would actually pull the stream for the record[10:24.640 --> 10:27.120] and then attempt to invoke the function.[10:27.120 --> 10:28.800] If there's a failure,[10:28.800 --> 10:31.480] AWS Lambda won't read any of the new shards[10:31.480 --> 10:34.840] until the failed batch of records expires or is processed[10:34.840 --> 10:36.160] successfully.[10:36.160 --> 10:39.840] In the non-streaming event, which would be SQS,[10:39.840 --> 10:42.400] Amazon would pull the queue for records.[10:42.400 --> 10:44.600] If it fails or times out,[10:44.600 --> 10:46.640] then the message would be returned to the queue,[10:46.640 --> 10:49.320] and then Lambda will keep retrying the failed message[10:49.320 --> 10:51.800] until it's processed successfully.[10:51.800 --> 10:53.600] If the message will expire,[10:53.600 --> 10:56.440] which is something you can do with SQS,[10:56.440 --> 10:58.240] then it'll just be discarded.[10:58.240 --> 11:00.400] And you can create a mapping between an event source[11:00.400 --> 11:02.960] and a Lambda function right inside of the console.[11:02.960 --> 11:05.520] And this is how typically you would set that up manually[11:05.520 --> 11:07.600] without using infrastructure as code.[11:08.560 --> 11:10.200] All right, let's talk about permissions.[11:10.200 --> 11:13.080] This is definitely an easy place to get tripped up[11:13.080 --> 11:15.760] when you're first using AWS Lambda.[11:15.760 --> 11:17.840] There's two types of permissions.[11:17.840 --> 11:20.120] The first is the event source and permission[11:20.120 --> 11:22.320] to trigger the Lambda function.[11:22.320 --> 11:24.480] This would be the invocation permission.[11:24.480 --> 11:26.440] And the next one would be the Lambda function[11:26.440 --> 11:29.600] needs permissions to interact with other services,[11:29.600 --> 11:31.280] but this would be the run permissions.[11:31.280 --> 11:34.520] And these are both handled via the IAM service[11:34.520 --> 11:38.120] or the AWS identity and access management service.[11:38.120 --> 11:43.120] So the IAM resource policy would tell the Lambda service[11:43.600 --> 11:46.640] which push event the sources have permission[11:46.640 --> 11:48.560] to invoke the Lambda function.[11:48.560 --> 11:51.120] And these resource policies would make it easy[11:51.120 --> 11:55.280] to grant access to a Lambda function across AWS account.[11:55.280 --> 11:58.400] So a good example would be if you have an S3 bucket[11:58.400 --> 12:01.400] in your account and you need to invoke a function[12:01.400 --> 12:03.880] in another account, you could create a resource policy[12:03.880 --> 12:07.120] that allows those to interact with each other.[12:07.120 --> 12:09.200] And the resource policy for a Lambda function[12:09.200 --> 12:11.200] is called a function policy.[12:11.200 --> 12:14.160] And when you add a trigger to your Lambda function[12:14.160 --> 12:16.760] from the console, the function policy[12:16.760 --> 12:18.680] will be generated automatically[12:18.680 --> 12:20.040] and it allows the event source[12:20.040 --> 12:22.820] to take the Lambda invoke function action.[12:24.400 --> 12:27.320] So a good example would be in Amazon S3 permission[12:27.320 --> 12:32.120] to invoke the Lambda function called my first function.[12:32.120 --> 12:34.720] And basically it would be an effect allow.[12:34.720 --> 12:36.880] And then under principle, if you would have service[12:36.880 --> 12:41.880] S3.AmazonEWS.com, the action would be Lambda colon[12:41.880 --> 12:45.400] invoke function and then the resource would be the name[12:45.400 --> 12:49.120] or the ARN of actually the Lambda.[12:49.120 --> 12:53.080] And then the condition would be actually the ARN of the bucket.[12:54.400 --> 12:56.720] And really that's it in a nutshell.[12:57.560 --> 13:01.480] The Lambda execution role grants your Lambda function[13:01.480 --> 13:05.040] permission to access AWS services and resources.[13:05.040 --> 13:08.000] And you select or create the execution role[13:08.000 --> 13:10.000] when you create a Lambda function.[13:10.000 --> 13:12.320] The IAM policy would define the actions[13:12.320 --> 13:14.440] of Lambda functions allowed to take[13:14.440 --> 13:16.720] and the trust policy allows the Lambda service[13:16.720 --> 13:20.040] to assume an execution role.[13:20.040 --> 13:23.800] To grant permissions to AWS Lambda to assume a role,[13:23.800 --> 13:27.460] you have to have the permission for IAM pass role action.[13:28.320 --> 13:31.000] A couple of different examples of a relevant policy[13:31.000 --> 13:34.560] for an execution role and the example,[13:34.560 --> 13:37.760] the IAM policy, you know,[13:37.760 --> 13:39.840] basically that we talked about earlier,[13:39.840 --> 13:43.000] would allow you to interact with S3.[13:43.000 --> 13:45.360] Another example would be to make it interact[13:45.360 --> 13:49.240] with CloudWatch logs and to create a log group[13:49.240 --> 13:51.640] and stream those logs.[13:51.640 --> 13:54.800] The trust policy would give Lambda service permissions[13:54.800 --> 13:57.600] to assume a role and invoke a Lambda function[13:57.600 --> 13:58.520] on your behalf.[13:59.560 --> 14:02.600] Now let's talk about the overview of authoring[14:02.600 --> 14:06.120] and configuring Lambda functions.[14:06.120 --> 14:10.440] So really to start with, to create a Lambda function,[14:10.440 --> 14:14.840] you first need to create a Lambda function deployment package,[14:14.840 --> 14:19.800] which is a zip or jar file that consists of your code[14:19.800 --> 14:23.160] and any dependencies with Lambda,[14:23.160 --> 14:25.400] you can use the programming language[14:25.400 --> 14:27.280] and integrated development environment[14:27.280 --> 14:29.800] that you're most familiar with.[14:29.800 --> 14:33.360] And you can actually bring the code you've already written.[14:33.360 --> 14:35.960] And Lambda does support lots of different languages[14:35.960 --> 14:39.520] like Node.js, Python, Ruby, Java, Go,[14:39.520 --> 14:41.160] and.NET runtimes.[14:41.160 --> 14:44.120] And you can also implement a custom runtime[14:44.120 --> 14:45.960] if you wanna use a different language as well,[14:45.960 --> 14:48.480] which is actually pretty cool.[14:48.480 --> 14:50.960] And if you wanna create a Lambda function,[14:50.960 --> 14:52.800] you would specify the handler,[14:52.800 --> 14:55.760] the Lambda function handler is the entry point.[14:55.760 --> 14:57.600] And a few different aspects of it[14:57.600 --> 14:59.400] that are important to pay attention to,[14:59.400 --> 15:00.720] the event object,[15:00.720 --> 15:03.480] this would provide information about the event[15:03.480 --> 15:05.520] that triggered the Lambda function.[15:05.520 --> 15:08.280] And this could be like a predefined object[15:08.280 --> 15:09.760] that AWS service generates.[15:09.760 --> 15:11.520] So you'll see this, like for example,[15:11.520 --> 15:13.440] in the console of AWS,[15:13.440 --> 15:16.360] you can actually ask for these objects[15:16.360 --> 15:19.200] and it'll give you really the JSON structure[15:19.200 --> 15:20.680] so you can test things out.[15:21.880 --> 15:23.900] In the contents of an event object[15:23.900 --> 15:26.800] includes everything you would need to actually invoke it.[15:26.800 --> 15:29.640] The context object is generated by AWS[15:29.640 --> 15:32.360] and this is really a runtime information.[15:32.360 --> 15:35.320] And so if you needed to get some kind of runtime information[15:35.320 --> 15:36.160] about your code,[15:36.160 --> 15:40.400] let's say environmental variables or AWS request ID[15:40.400 --> 15:44.280] or a log stream or remaining time in Millies,[15:45.320 --> 15:47.200] like for example, that one would return[15:47.200 --> 15:48.840] the number of milliseconds that remain[15:48.840 --> 15:50.600] before your function times out,[15:50.600 --> 15:53.300] you can get all that inside the context object.[15:54.520 --> 15:57.560] So what about an example that runs a Python?[15:57.560 --> 15:59.280] Pretty straightforward actually.[15:59.280 --> 16:01.400] All you need is you would put a handler[16:01.400 --> 16:03.280] inside the handler would take,[16:03.280 --> 16:05.000] that it would be a Python function,[16:05.000 --> 16:07.080] it would be an event, there'd be a context,[16:07.080 --> 16:10.960] you pass it inside and then you return some kind of message.[16:10.960 --> 16:13.960] A few different best practices to remember[16:13.960 --> 16:17.240] about AWS Lambda would be to separate[16:17.240 --> 16:20.320] the core business logic from the handler method[16:20.320 --> 16:22.320] and this would make your code more portable,[16:22.320 --> 16:24.280] enable you to target unit tests[16:25.240 --> 16:27.120] without having to worry about the configuration.[16:27.120 --> 16:30.400] So this is always a really good idea just in general.[16:30.400 --> 16:32.680] Make sure you have modular functions.[16:32.680 --> 16:34.320] So you have a single purpose function,[16:34.320 --> 16:37.160] you don't have like a kitchen sink function,[16:37.160 --> 16:40.000] you treat functions as stateless as well.[16:40.000 --> 16:42.800] So you would treat a function that basically[16:42.800 --> 16:46.040] just does one thing and then when it's done,[16:46.040 --> 16:48.320] there is no state that's actually kept anywhere[16:49.320 --> 16:51.120] and also only include what you need.[16:51.120 --> 16:55.840] So you don't want to have a huge sized Lambda functions[16:55.840 --> 16:58.560] and one of the ways that you can avoid this[16:58.560 --> 17:02.360] is by reducing the time it takes a Lambda to unpack[17:02.360 --> 17:04.000] the deployment packages[17:04.000 --> 17:06.600] and you can also minimize the complexity[17:06.600 --> 17:08.640] of your dependencies as well.[17:08.640 --> 17:13.600] And you can also reuse the temporary runtime environment[17:13.600 --> 17:16.080] to improve the performance of a function as well.[17:16.080 --> 17:17.680] And so the temporary runtime environment[17:17.680 --> 17:22.280] initializes any external dependencies of the Lambda code[17:22.280 --> 17:25.760] and you can make sure that any externalized configuration[17:25.760 --> 17:27.920] or dependency that your code retrieves are stored[17:27.920 --> 17:30.640] and referenced locally after the initial run.[17:30.640 --> 17:33.800] So this would be limit re-initializing variables[17:33.800 --> 17:35.960] and objects on every invocation,[17:35.960 --> 17:38.200] keeping it alive and reusing connections[17:38.200 --> 17:40.680] like an HTTP or database[17:40.680 --> 17:43.160] that were established during the previous invocation.[17:43.160 --> 17:45.880] So a really good example of this would be a socket connection.[17:45.880 --> 17:48.040] If you make a socket connection[17:48.040 --> 17:51.640] and this socket connection took two seconds to spawn,[17:51.640 --> 17:54.000] you don't want every time you call Lambda[17:54.000 --> 17:55.480] for it to wait two seconds,[17:55.480 --> 17:58.160] you want to reuse that socket connection.[17:58.160 --> 18:00.600] A few good examples of best practices[18:00.600 --> 18:02.840] would be including logging statements.[18:02.840 --> 18:05.480] This is a kind of a big one[18:05.480 --> 18:08.120] in the case of any cloud computing operation,[18:08.120 --> 18:10.960] especially when it's distributed, if you don't log it,[18:10.960 --> 18:13.280] there's no way you can figure out what's going on.[18:13.280 --> 18:16.560] So you must add logging statements that have context[18:16.560 --> 18:19.720] so you know which particular Lambda instance[18:19.720 --> 18:21.600] is actually occurring in.[18:21.600 --> 18:23.440] Also include results.[18:23.440 --> 18:25.560] So make sure that you know it's happening[18:25.560 --> 18:29.000] when the Lambda ran, use environmental variables as well.[18:29.000 --> 18:31.320] So you can figure out things like what the bucket was[18:31.320 --> 18:32.880] that it was writing to.[18:32.880 --> 18:35.520] And then also don't do recursive code.[18:35.520 --> 18:37.360] That's really a no-no.[18:37.360 --> 18:40.200] You want to write very simple functions with Lambda.[18:41.320 --> 18:44.440] Few different ways to write Lambda actually would be[18:44.440 --> 18:46.280] that you can do the console editor,[18:46.280 --> 18:47.440] which I use all the time.[18:47.440 --> 18:49.320] I like to actually just play around with it.[18:49.320 --> 18:51.640] Now the downside is that if you don't,[18:51.640 --> 18:53.800] if you do need to use custom libraries,[18:53.800 --> 18:56.600] you're not gonna be able to do it other than using,[18:56.600 --> 18:58.440] let's say the AWS SDK.[18:58.440 --> 19:01.600] But for just simple things, it's a great use case.[19:01.600 --> 19:06.080] Another one is you can just upload it to AWS console.[19:06.080 --> 19:09.040] And so you can create a deployment package in an IDE.[19:09.040 --> 19:12.120] Like for example, Visual Studio for.NET,[19:12.120 --> 19:13.280] you can actually just right click[19:13.280 --> 19:16.320] and deploy it directly into Lambda.[19:16.320 --> 19:20.920] Another one is you can upload the entire package into S3[19:20.920 --> 19:22.200] and put it into a bucket.[19:22.200 --> 19:26.280] And then Lambda will just grab it outside of that S3 package.[19:26.280 --> 19:29.760] A few different things to remember about Lambda.[19:29.760 --> 19:32.520] The memory and the timeout are configurations[19:32.520 --> 19:35.840] that determine how the Lambda function performs.[19:35.840 --> 19:38.440] And these will affect the billing.[19:38.440 --> 19:40.200] Now, one of the great things about Lambda[19:40.200 --> 19:43.640] is just amazingly inexpensive to run.[19:43.640 --> 19:45.560] And the reason is that you're charged[19:45.560 --> 19:48.200] based on the number of requests for a function.[19:48.200 --> 19:50.560] A few different things to remember would be the memory.[19:50.560 --> 19:53.560] Like so if you specify more memory,[19:53.560 --> 19:57.120] it's going to increase the cost timeout.[19:57.120 --> 19:59.960] You can also control the memory duration of the function[19:59.960 --> 20:01.720] by having the right kind of timeout.[20:01.720 --> 20:03.960] But if you make the timeout too long,[20:03.960 --> 20:05.880] it could cost you more money.[20:05.880 --> 20:08.520] So really the best practices would be test the performance[20:08.520 --> 20:12.880] of Lambda and make sure you have the optimum memory size.[20:12.880 --> 20:15.160] Also load test it to make sure[20:15.160 --> 20:17.440] that you understand how the timeouts work.[20:17.440 --> 20:18.280] Just in general,[20:18.280 --> 20:21.640] anything with cloud computing, you should load test it.[20:21.640 --> 20:24.200] Now let's talk about an important topic[20:24.200 --> 20:25.280] that's a final topic here,[20:25.280 --> 20:29.080] which is how to deploy Lambda functions.[20:29.080 --> 20:32.200] So versions are immutable copies of a code[20:32.200 --> 20:34.200] in the configuration of your Lambda function.[20:34.200 --> 20:35.880] And the versioning will allow you to publish[20:35.880 --> 20:39.360] one or more versions of your Lambda function.[20:39.360 --> 20:40.400] And as a result,[20:40.400 --> 20:43.360] you can work with different variations of your Lambda function[20:44.560 --> 20:45.840] in your development workflow,[20:45.840 --> 20:48.680] like development, beta, production, et cetera.[20:48.680 --> 20:50.320] And when you create a Lambda function,[20:50.320 --> 20:52.960] there's only one version, the latest version,[20:52.960 --> 20:54.080] dollar sign, latest.[20:54.080 --> 20:57.240] And you can refer to this function using the ARN[20:57.240 --> 20:59.240] or Amazon resource name.[20:59.240 --> 21:00.640] And when you publish a new version,[21:00.640 --> 21:02.920] AWS Lambda will make a snapshot[21:02.920 --> 21:05.320] of the latest version to create a new version.[21:06.800 --> 21:09.600] You can also create an alias for Lambda function.[21:09.600 --> 21:12.280] And conceptually, an alias is just like a pointer[21:12.280 --> 21:13.800] to a specific function.[21:13.800 --> 21:17.040] And you can use that alias in the ARN[21:17.040 --> 21:18.680] to reference the Lambda function version[21:18.680 --> 21:21.280] that's currently associated with the alias.[21:21.280 --> 21:23.400] What's nice about the alias is you can roll back[21:23.400 --> 21:25.840] and forth between different versions,[21:25.840 --> 21:29.760] which is pretty nice because in the case of deploying[21:29.760 --> 21:32.920] a new version, if there's a huge problem with it,[21:32.920 --> 21:34.080] you just toggle it right back.[21:34.080 --> 21:36.400] And there's really not a big issue[21:36.400 --> 21:39.400] in terms of rolling back your code.[21:39.400 --> 21:44.400] Now, let's take a look at an example where AWS S3,[21:45.160 --> 21:46.720] or Amazon S3 is the event source[21:46.720 --> 21:48.560] that invokes your Lambda function.[21:48.560 --> 21:50.720] Every time a new object is created,[21:50.720 --> 21:52.880] when Amazon S3 is the event source,[21:52.880 --> 21:55.800] you can store the information for the event source mapping[21:55.800 --> 21:59.040] in the configuration for the bucket notifications.[21:59.040 --> 22:01.000] And then in that configuration,[22:01.000 --> 22:04.800] you could identify the Lambda function ARN[22:04.800 --> 22:07.160] that Amazon S3 can invoke.[22:07.160 --> 22:08.520] But in some cases,[22:08.520 --> 22:11.680] you're gonna have to update the notification configuration.[22:11.680 --> 22:14.720] So Amazon S3 will invoke the correct version each time[22:14.720 --> 22:17.840] you publish a new version of your Lambda function.[22:17.840 --> 22:21.800] So basically, instead of specifying the function ARN,[22:21.800 --> 22:23.880] you can specify an alias ARN[22:23.880 --> 22:26.320] in the notification of configuration.[22:26.320 --> 22:29.160] And as you promote a new version of the Lambda function[22:29.160 --> 22:32.200] into production, you only need to update the prod alias[22:32.200 --> 22:34.520] to point to the latest stable version.[22:34.520 --> 22:36.320] And you also don't need to update[22:36.320 --> 22:39.120] the notification configuration in Amazon S3.[22:40.480 --> 22:43.080] And when you build serverless applications[22:43.080 --> 22:46.600] as common to have code that's shared across Lambda functions,[22:46.600 --> 22:49.400] it could be custom code, it could be a standard library,[22:49.400 --> 22:50.560] et cetera.[22:50.560 --> 22:53.320] And before, and this was really a big limitation,[22:53.320 --> 22:55.920] was you had to have all the code deployed together.[22:55.920 --> 22:58.960] But now, one of the really cool things you can do[22:58.960 --> 23:00.880] is you can have a Lambda function[23:00.880 --> 23:03.600] to include additional code as a layer.[23:03.600 --> 23:05.520] So layer is basically a zip archive[23:05.520 --> 23:08.640] that contains a library, maybe a custom runtime.[23:08.640 --> 23:11.720] Maybe it isn't gonna include some kind of really cool[23:11.720 --> 23:13.040] pre-trained model.[23:13.040 --> 23:14.680] And then the layers you can use,[23:14.680 --> 23:15.800] the libraries in your function[23:15.800 --> 23:18.960] without needing to include them in your deployment package.[23:18.960 --> 23:22.400] And it's a best practice to have the smaller deployment packages[23:22.400 --> 23:25.240] and share common dependencies with the layers.[23:26.120 --> 23:28.520] Also layers will help you keep your deployment package[23:28.520 --> 23:29.360] really small.[23:29.360 --> 23:32.680] So for node, JS, Python, Ruby functions,[23:32.680 --> 23:36.000] you can develop your function code in the console[23:36.000 --> 23:39.000] as long as you keep the package under three megabytes.[23:39.000 --> 23:42.320] And then a function can use up to five layers at a time,[23:42.320 --> 23:44.160] which is pretty incredible actually,[23:44.160 --> 23:46.040] which means that you could have, you know,[23:46.040 --> 23:49.240] basically up to a 250 megabytes total.[23:49.240 --> 23:53.920] So for many languages, this is plenty of space.[23:53.920 --> 23:56.620] Also Amazon has published a public layer[23:56.620 --> 23:58.800] that includes really popular libraries[23:58.800 --> 24:00.800] like NumPy and SciPy,[24:00.800 --> 24:04.840] which does dramatically help data processing[24:04.840 --> 24:05.680] in machine learning.[24:05.680 --> 24:07.680] Now, if I had to predict the future[24:07.680 --> 24:11.840] and I wanted to predict a massive announcement,[24:11.840 --> 24:14.840] I would say that what AWS could do[24:14.840 --> 24:18.600] is they could have a GPU enabled layer at some point[24:18.600 --> 24:20.160] that would include pre-trained models.[24:20.160 --> 24:22.120] And if they did something like that,[24:22.120 --> 24:24.320] that could really open up the doors[24:24.320 --> 24:27.000] for the pre-trained model revolution.[24:27.000 --> 24:30.160] And I would bet that that's possible.[24:30.160 --> 24:32.200] All right, well, in a nutshell,[24:32.200 --> 24:34.680] AWS Lambda is one of my favorite services.[24:34.680 --> 24:38.440] And I think it's worth everybody's time[24:38.440 --> 24:42.360] that's interested in AWS to play around with AWS Lambda.[24:42.360 --> 24:47.200] All right, next week, I'm going to cover API Gateway.[24:47.200 --> 25:13.840] All right, see you next week.If you enjoyed this video, here are additional resources to look at:Coursera + Duke Specialization: Building Cloud Computing Solutions at Scale Specialization: https://www.coursera.org/specializations/building-cloud-computing-solutions-at-scalePython, Bash, and SQL Essentials for Data Engineering Specialization: https://www.coursera.org/specializations/python-bash-sql-data-engineering-dukeAWS Certified Solutions Architect - Professional (SAP-C01) Cert Prep: 1 Design for Organizational Complexity:https://www.linkedin.com/learning/aws-certified-solutions-architect-professional-sap-c01-cert-prep-1-design-for-organizational-complexity/design-for-organizational-complexity?autoplay=trueEssentials of MLOps with Azure and Databricks: https://www.linkedin.com/learning/essentials-of-mlops-with-azure-1-introduction/essentials-of-mlops-with-azureO'Reilly Book: Implementing MLOps in the EnterpriseO'Reilly Book: Practical MLOps: https://www.amazon.com/Practical-MLOps-Operationalizing-Machine-Learning/dp/1098103017O'Reilly Book: Python for DevOps: https://www.amazon.com/gp/product/B082P97LDW/O'Reilly Book: Developing on AWS with C#: A Comprehensive Guide on Using C# to Build Solutions on the AWS Platformhttps://www.amazon.com/Developing-AWS-Comprehensive-Solutions-Platform/dp/1492095877Pragmatic AI: An Introduction to Cloud-based Machine Learning: https://www.amazon.com/gp/product/B07FB8F8QP/Pragmatic AI Labs Book: Python Command-Line Tools: https://www.amazon.com/gp/product/B0855FSFYZPragmatic AI Labs Book: Cloud Computing for Data Analysis: https://www.amazon.com/gp/product/B0992BN7W8Pragmatic AI Book: Minimal Python: https://www.amazon.com/gp/product/B0855NSRR7Pragmatic AI Book: Testing in Python: https://www.amazon.com/gp/product/B0855NSRR7Subscribe to Pragmatic AI Labs YouTube Channel: https://www.youtube.com/channel/UCNDfiL0D1LUeKWAkRE1xO5QSubscribe to 52 Weeks of AWS Podcast: https://52-weeks-of-cloud.simplecast.comView content on noahgift.com: https://noahgift.com/View content on Pragmatic AI Labs Website: https://paiml.com/

Slovenský startup CloudTalk, který vyvíjí SaaS řešení pro call centra, založili v roce 2018 Martin Malych a Viktor Vaněk a od té doby zažívají neuvěřitelnou jízdu. Letos se zařadili mezi top 100 nejrychleji rostoucích softwarových platforem světa, z 10 zaměstnanců vyrostli na 250, zákazníků mají rovnou 10x tolik a patří mezi ně i taková jména jako DHL, Revolut nebo Mercedes.

Pull your podcast player out of instant retrieval, because we're discussing re:Invent 2021 as well as the weeks before it. Lots of announcements; big, small, weird, awesome, and anything in between. We had fun with this episode and hope you do too. Find us at melb.awsug.org.au or as @AWSMelb on Twitter. News Finally in Sydney AWS Snowcone SSD is now available in the US East (Ohio), US West (San Francisco), Asia Pacific (Singapore), Asia Pacific (Sydney) and AWS Asia Pacific (Tokyo) regions Amazon EC2 M6i instances are now available in 5 additional regions Serverless Introducing Amazon EMR Serverless in preview Announcing Amazon Kinesis Data Streams On-Demand Announcing Amazon Redshift Serverless (Preview) Introducing Amazon MSK Serverless in public preview Introducing Amazon SageMaker Serverless Inference (preview) Simplify CI/CD Configuration for AWS Serverless Applications and your favorite CI/CD system – General Availability Amazon AppStream 2.0 launches Elastic fleets, a serverless fleet type AWS Chatbot now supports management of AWS resources in Slack (Preview) Lambda AWS Lambda now supports partial batch response for SQS as an event source AWS Lambda now supports cross-account container image pulling from Amazon Elastic Container Registry AWS Lambda now supports mTLS Authentication for Amazon MSK as an event source AWS Lambda now logs Hyperplane Elastic Network Interface (ENI) ID in AWS CloudTrail data events Step Functions AWS Step Functions Synchronous Express Workflows now supports AWS PrivateLink Amplify Introducing AWS Amplify Studio AWS Amplify announces the ability to override Amplify-generated resources using CDK AWS Amplify announces the ability to add custom AWS resources to Amplify-created backends using CDK and CloudFormation AWS Amplify UI launches new Authenticator component for React, Angular, and Vue AWS Amplify announces the ability to export Amplify backends as CDK stacks to integrate into CDK-based pipelines AWS Amplify expands its Notifications category to include in-app messaging (Developer Preview) AWS Amplify announces a redesigned, more extensible GraphQL Transformer for creating app backends quickly Containers Fargate Announcing AWS Fargate for Amazon ECS Powered by AWS Graviton2 Processors ECS Amazon ECS now adds container instance health information Amazon ECS has improved Capacity Providers to deliver faster Cluster Auto Scaling Amazon ECS-optimized AMI is now available as an open-source project Amazon ECS announces a new integration with AWS Distro for OpenTelemetry EKS Amazon EKS on AWS Fargate now Supports the Fluent Bit Kubernetes Filter Amazon EKS adds support for additional cluster configuration options using AWS CloudFormation Visualize all your Kubernetes clusters in one place with Amazon EKS Connector, now generally available AWS Karpenter v0.5 Now Generally Available AWS customers can now find, subscribe to, and deploy third-party applications that run in any Kubernetes environment from AWS Marketplace Other Amazon ECR announces pull through cache repositories AWS App Mesh now supports ARM64-based Envoy Images EC2 & VPC Instances New – EC2 Instances (G5) with NVIDIA A10G Tensor Core GPUs | AWS News Blog Announcing new Amazon EC2 G5g instances powered by AWS Graviton2 processors Introducing Amazon EC2 R6i instances Introducing two new Amazon EC2 bare metal instances Amazon EC2 Mac Instances now support hot attach and detach of EBS volumes Amazon EC2 Mac Instances now support macOS Monterey Announcing Amazon EC2 M1 Mac instances for macOS Announcing preview of Amazon Linux 2022 Elastic Beanstalk supports AWS Graviton-based Amazon EC2 instance types Announcing preview of Amazon EC2 Trn1 instances Announcing new Amazon EC2 C7g instances powered by AWS Graviton3 processors Announcing new Amazon EC2 Im4gn and Is4gen instances powered by AWS Graviton2 processors Introducing the AWS Graviton Ready Program Introducing Amazon EC2 M6a instances AWS Compute Optimizer now offers enhanced infrastructure metrics, a new feature for EC2 recommendations AWS Compute Optimizer now offers resource efficiency metrics Networking AWS price reduction for data transfers out to the internet Amazon Virtual Private Cloud (VPC) customers can now create IPv6-only subnets and EC2 instances Application Load Balancer and Network Load Balancer end-to-end IPv6 support AWS Transit Gateway introduces intra-region peering for simplified cloud operations and network connectivity Amazon Virtual Private Cloud (VPC) announces IP Address Manager (IPAM) to help simplify IP address management on AWS Amazon Virtual Private Cloud (VPC) announces Network Access Analyzer to help you easily identify unintended network access Introducing AWS Cloud WAN Preview Introducing AWS Direct Connect SiteLink Other Recover from accidental deletions of your snapshots using Recycle Bin Amazon EBS Snapshots introduces a new tier, Amazon EBS Snapshots Archive, to reduce the cost of long-term retention of EBS Snapshots by up to 75% Amazon CloudFront now supports configurable CORS, security, and custom HTTP response headers Amazon EC2 now supports access to Red Hat Knowledgebase Amazon EC2 Fleet and Spot Fleet now support automatic instance termination with Capacity Rebalancing AWS announces a new capability to switch license types for Windows Server and SQL Server applications on Amazon EC2 AWS Batch introduces fair-share scheduling Amazon EC2 Auto Scaling Now Supports Predictive Scaling with Custom Metrics Dev & Ops New services Measure and Improve Your Application Resilience with AWS Resilience Hub | AWS News Blog Scalable, Cost-Effective Disaster Recovery in the Cloud | AWS News Blog Announcing general availability of AWS Elastic Disaster Recovery AWS announces the launch of AWS AppConfig Feature Flags in preview Announcing Amazon DevOps Guru for RDS, an ML-powered capability that automatically detects and diagnoses performance and operational issues within Amazon Aurora Introducing Amazon CloudWatch Metrics Insights (Preview) Introducing Amazon CloudWatch RUM for monitoring applications' client-side performance IaC AWS announces Construct Hub general availability AWS Cloud Development Kit (AWS CDK) v2 is now generally available You can now import your AWS CloudFormation stacks into a CloudFormation stack set You can now submit multiple operations for simultaneous execution with AWS CloudFormation StackSets AWS CDK releases v1.126.0 - v1.130.0 with high-level APIs for AWS App Runner and hotswap support for Amazon ECS and AWS Step Functions SDKs AWS SDK for Swift (Developer Preview) AWS SDK for Kotlin (Developer Preview) AWS SDK for Rust (Developer Preview) CICD AWS Proton now supports Terraform Open Source for infrastructure provisioning AWS Proton introduces Git management of infrastructure as code templates AWS App2Container now supports Jenkins for setting up a CI/CD pipeline Other Amazon CodeGuru Reviewer now detects hardcoded secrets in Java and Python repositories EC2 Image Builder enables sharing Amazon Machine Images (AMIs) with AWS Organizations and Organization Units Amazon Corretto 17 Support Roadmap Announced Amazon DevOps Guru now Supports Multi-Account Insight Aggregation with AWS Organizations AWS Toolkits for Cloud9, JetBrains and VS Code now support interaction with over 200 new resource types AWS Fault Injection Simulator now supports Amazon CloudWatch Alarms and AWS Systems Manager Automation Runbooks. AWS Device Farm announces support for testing web applications hosted in an Amazon VPC Amazon CloudWatch now supports anomaly detection on metric math expressions Introducing Amazon CloudWatch Evidently for feature experimentation and safer launches New – Amazon CloudWatch Evidently – Experiments and Feature Management | AWS News Blog Introducing AWS Microservice Extractor for .NET Security AWS Secrets Manager increases secrets limit to 500K per account AWS CloudTrail announces ErrorRate Insights AWS announces the new Amazon Inspector for continual vulnerability management Amazon SQS Announces Server-Side Encryption with Amazon SQS-managed encryption keys (SSE-SQS) AWS WAF adds support for Captcha AWS Shield Advanced introduces automatic application-layer DDoS mitigation Security Hub AWS Security Hub adds support for AWS PrivateLink for private access to Security Hub APIs AWS Security Hub adds three new FSBP controls and three new partners SSO Manage Access Centrally for CyberArk Users with AWS Single Sign-On Manage Access Centrally for JumpCloud Users with AWS Single Sign-On AWS Single Sign-On now provides one-click login to Amazon EC2 instances running Microsoft Windows AWS Single Sign-On is now in scope for AWS SOC reporting Control Tower AWS Control Tower now supports concurrent operations for detective guardrails AWS Control Tower now supports nested organizational units AWS Control Tower now provides controls to meet data residency requirements Deny services and operations for AWS Regions of your choice with AWS Control Tower AWS Control Tower introduces Terraform account provisioning and customization Data Storage & Processing Databases Relational databases Announcing Amazon RDS Custom for SQL Server New Multi-AZ deployment option for Amazon RDS for PostgreSQL and for MySQL; increased read capacity, lower and more consistent write transaction latency, and shorter failover time (Preview) Amazon RDS now supports cross account KMS keys for exporting RDS Snapshots Amazon Aurora supports MySQL 8.0 Amazon RDS on AWS Outposts now supports backups on AWS Outposts Athena Amazon Athena adds cost details to query execution plans Amazon Athena announces cross-account federated query New and improved Amazon Athena console is now generally available Amazon Athena now supports new Lake Formation fine-grained security and reliable table features Announcing Amazon Athena ACID transactions, powered by Apache Iceberg (Preview) Redshift Announcing preview for write queries with Amazon Redshift Concurrency Scaling Amazon Redshift announces native support for SQLAlchemy and Apache Airflow open-source frameworks Amazon Redshift simplifies the use of other AWS services by introducing the default IAM role Announcing Amazon Redshift cross-region data sharing (preview) Announcing preview of SQL Notebooks support in Amazon Redshift Query Editor V2 Neptune Announcing AWS Graviton2-based instances for Amazon Neptune AWS releases open source JDBC driver to connect to Amazon Neptune MemoryDB Amazon MemoryDB for Redis now supports AWS Graviton2-based T4g instances and a 2-month Free Trial Database Migration Service AWS Database Migration Service now supports parallel load for partitioned data to S3 AWS Database Migration Service now supports Kafka multi-topic AWS Database Migration Service now supports Azure SQL Managed Instance as a source AWS Database Migration Service now supports Google Cloud SQL for MySQL as a source Introducing AWS DMS Fleet Advisor for automated discovery and analysis of database and analytics workloads (Preview) AWS Database Migration Service now offers a new console experience, AWS DMS Studio AWS Database Migration Service now supports Time Travel, an improved logging mechanism Other Database Activity Streams now supports Graviton2-based instances Amazon Timestream now offers faster and more cost-effective time series data processing through scheduled queries, multi-measure records, and magnetic storage writes Amazon DynamoDB announces the new Amazon DynamoDB Standard-Infrequent Access table class, which helps you reduce your DynamoDB costs by up to 60 percent Achieve up to 30% better performance with Amazon DocumentDB (with MongoDB compatibility) using new Graviton2 instances S3 Amazon S3 on Outposts now delivers strong consistency automatically for all applications Amazon S3 Lifecycle further optimizes storage cost savings with new actions and filters Announcing the new Amazon S3 Glacier Instant Retrieval storage class - the lowest cost archive storage with milliseconds retrieval Amazon S3 Object Ownership can now disable access control lists to simplify access management for data in S3 Amazon S3 Glacier storage class is now Amazon S3 Glacier Flexible Retrieval; storage price reduced by 10% and bulk retrievals are now free Announcing the new S3 Intelligent-Tiering Archive Instant Access tier - Automatically save up to 68% on storage costs Amazon S3 Event Notifications with Amazon EventBridge help you build advanced serverless applications faster Amazon S3 console now reports security warnings, errors, and suggestions from IAM Access Analyzer as you author your S3 policies Amazon S3 adds new S3 Event Notifications for S3 Lifecycle, S3 Intelligent-Tiering, object tags, and object access control lists Glue AWS Glue DataBrew announces native console integration with Amazon AppFlow AWS Glue DataBrew now supports custom SQL statements to retrieve data from Amazon Redshift and Snowflake AWS Glue DataBrew now allows customers to create data quality rules to define and validate their business requirements FSx Introducing Amazon FSx for OpenZFS Amazon FSx for Lustre now supports linking multiple Amazon S3 buckets to a file system Amazon FSx for Lustre can now automatically update file system contents as data is deleted and moved in Amazon S3 Announcing the next generation of Amazon FSx for Lustre file systems Backup Announcing preview of AWS Backup for Amazon S3 AWS Backup adds support for Amazon Neptune AWS Backup adds support for Amazon DocumentDB (with MongoDB compatibility) AWS Backup provides new resource assignment rules for your data protection policies AWS Backup adds support for VMware workloads Other AWS Lake Formation now supports AWS PrivateLink AWS Transfer Family adds identity provider options and enhanced monitoring capabilities Introducing ability to connect to EMR clusters in different subnets in EMR Studio AWS Snow Family now supports external NTP server configuration Announcing data tiering for Amazon ElastiCache for Redis Now execute python files and notebooks from another notebook in EMR Studio AWS Snow Family launches offline tape data migration capability AI & ML SageMaker Introducing Amazon SageMaker Canvas - a visual, no-code interface to build accurate machine learning models Announcing Fully Managed RStudio on Amazon SageMaker for Data Scientists | AWS News Blog Amazon SageMaker now supports inference testing with custom domains and headers from SageMaker Studio Amazon SageMaker Pipelines now supports retry policies and resume Announcing new deployment guardrails for Amazon SageMaker Inference endpoints Amazon announces new NVIDIA Triton Inference Server on Amazon SageMaker Amazon SageMaker Pipelines now integrates with SageMaker Model Monitor and SageMaker Clarify Amazon SageMaker now supports cross-account lineage tracking and multi-hop lineage querying Introducing Amazon SageMaker Inference Recommender Introducing Amazon SageMaker Ground Truth Plus: Create high-quality training datasets without having to build labeling applications or manage the labeling workforce on your own Amazon SageMaker Studio Lab (currently in preview), a free, no-configuration ML service Amazon SageMaker Studio now enables interactive data preparation and machine learning at scale within a single universal notebook through built-in integration with Amazon EMR Other General Availability of Syne Tune, an open-source library for distributed hyperparameter and neural architecture optimization Amazon Translate now supports AWS KMS Encryption Amazon Kendra releases AWS Single Sign-On integration for secure search Amazon Transcribe now supports automatic language identification for streaming transcriptions AWS AI for data analytics (AIDA) partner solutions Introducing Amazon Lex Automated Chatbot Designer (Preview) Amazon Kendra launches Experience Builder, Search Analytics Dashboard, and Custom Document Enrichment Other Cool Stuff In The Works – AWS Canada West (Calgary) Region | AWS News Blog Unified Search in the AWS Management Console now includes blogs, knowledge articles, events, and tutorials AWS DeepRacer introduces multi-user account management Amazon Pinpoint launches in-app messaging as a new communications channel Amazon AppStream 2.0 Introduces Linux Application Streaming Amazon SNS now supports publishing batches of up to 10 messages in a single API request Announcing usability improvements in the navigation bar of the AWS Management Console Announcing General Availability of Enterprise On-Ramp Announcing preview of AWS Private 5G AWS Outposts is Now Available in Two Smaller Form Factors Introducing AWS Mainframe Modernization - Preview Introducing the AWS Migration and Modernization Competency Announcing AWS Data Exchange for APIs Amazon WorkSpaces introduces Amazon WorkSpaces Web Amazon SQS Enhances Dead-letter Queue Management Experience For Standard Queues Introducing AWS re:Post, a new, community-driven, questions-and-answers service AWS Resource Access Manager enables support for global resource types AWS Ground Station launches expanded support for Software Defined Radios in Preview Announcing Amazon Braket Hybrid Jobs for running hybrid quantum-classical workloads on Amazon Braket Introducing AWS Migration Hub Refactor Spaces - Preview Well-Architected Framework Customize your AWS Well-Architected Review using Custom Lenses New Sustainability Pillar for the AWS Well-Architected Framework IoT Announcing AWS IoT RoboRunner, Now Available in Preview AWS IoT Greengrass now supports Microsoft Windows devices AWS IoT Core now supports Multi-Account Registration certificates on IoT Credential Provider endpoint Announcing AWS IoT FleetWise (Preview), a new service for transferring vehicle data to the cloud more efficiently Announcing AWS IoT TwinMaker (Preview), a service that makes it easier to build digital twins AWS IoT SiteWise now supports hot and cold storage tiers for industrial data New connectivity software, AWS IoT ExpressLink, accelerates IoT development (Preview) AWS IoT Device Management Fleet Indexing now supports two additional data sources (Preview) Connect Amazon Connect now enables you to create and orchestrate tasks directly from Flows Amazon Connect launches scheduled tasks Amazon Connect launches Contact APIs to fetch and update contact details programmatically Amazon Connect launches API to configure security profiles programmatically Amazon Connect launches APIs to archive and delete contact flows Amazon Connect now supports contact flow modules to simplify repeatable logic Sponsors CMD Solutions Silver Sponsors Cevo Versent

A lot of things happened in October, and we talked about them all in early November. In this episode Arjen, Guy, and JM discuss a whole bunch of cool things that were released and may be a bit harsh on everything Microsoft. News Finally in Sydney Amazon EC2 Mac instances are now available in seven additional AWS Regions Amazon MemoryDB for Redis is now available in 11 additional AWS Regions Serverless Lambda AWS Lambda now supports triggering Lambda functions from an Amazon SQS queue in a different account AWS Lambda now supports IAM authentication for Amazon MSK as an event source Step Functions Now — AWS Step Functions Supports 200 AWS Services To Enable Easier Workflow Automation | AWS News Blog AWS Batch adds console support for visualizing AWS Step Functions workflows Amplify Announcing General Availability of Amplify Geo for AWS Amplify AWS Amplify for JavaScript now supports resumable file uploads for Storage Other Accelerating serverless development with AWS SAM Accelerate | AWS Compute Blog Containers Amazon EKS Managed Node Groups adds native support for Bottlerocket AWS Fargate now supports Amazon ECS Windows containers Announcing the general availability of cdk8s and support for Go | Containers Monitoring clock accuracy on AWS Fargate with Amazon ECS Amazon ECS Anywhere now supports GPU-based workloads AWS Console Mobile Application adds support for Amazon Elastic Container Service AWS Load Balancer Controller version 2.3 now available with support for ALB IPv6 targets AWS App Mesh Metric Extension is now generally available EC2 & VPC New – Amazon EC2 C6i Instances Powered by the Latest Generation Intel Xeon Scalable Processors | AWS News Blog Amazon EC2 now supports sharing Amazon Machine Images across AWS Organizations and Organizational Units Amazon EC2 Hibernation adds support for Ubuntu 20.04 LTS Announcing Amazon EC2 Capacity Reservation Fleet a way to easily migrate Amazon EC2 Capacity Reservations across instance types Amazon EC2 Auto Scaling now supports describing Auto Scaling groups using tags Amazon EC2 now offers Microsoft SQL Server on Microsoft Windows Server 2022 AMIs AWS Elastic Beanstalk supports Database Decoupling in an Elastic Beanstalk Environment AWS FPGA developer kit now supports Jumbo frames in virtual ethernet frameworks for Amazon EC2 F1 instances Amazon VPC Flow Logs now supports Apache Parquet, Hive-compatible prefixes and Hourly partitioned files Network Load Balancer now supports TLS 1.3 New – Attribute-Based Instance Type Selection for EC2 Auto Scaling and EC2 Fleet | AWS News Blog Amazon Lightsail now supports AWS CloudFormation for instances, disks and databases Dev & Ops CLI AWS Cloud Control API, a Uniform API to Access AWS & Third-Party Services | AWS News Blog Now programmatically manage alternate contacts on AWS accounts CodeGuru Amazon CodeGuru now includes recommendations powered by Infer Amazon CodeGuru announces Security detectors for Python applications and security analysis powered by Bandit Amazon CodeGuru Reviewer adds detectors for AWS Java SDK v2's best practices and features IaC AWS CDK releases v1.121.0 - v1.125.0 with features for faster development cycles using hotswap deployments and rollback control AWS CloudFormation customers can now manage their applications in AWS Systems Manager Other NoSQL Workbench for Amazon DynamoDB now enables you to import and automatically populate sample data to help build and visualize your data models Amazon Corretto October Quarterly Updates Bulk Editing of OpsItems in AWS Systems Manager OpsCenter AWS Fault Injection Simulator now supports Spot Interruptions AWS Fault Injection Simulator now injects Spot Instance Interruptions Security Firewalls AWS Firewall Manager now supports centralized logging of AWS Network Firewall logs AWS Network Firewall Adds New Configuration Options for Rule Ordering and Default Drop Backups AWS Backup Audit Manager adds compliance reports AWS Backup adds an additional layer for backup protection with the availability of AWS Backup Vault Lock Other AWS Security Hub adds support for cross-Region aggregation of findings to simplify how you evaluate and improve your AWS security posture Amazon SES now supports 2048-bit DKIM keys AWS License Manager now supports Delegated Administrator for Managed entitlements Data Storage & Processing Goodbye Microsoft SQL Server, Hello Babelfish | AWS News Blog Announcing availability of the Babelfish for PostgreSQL open source project Announcing Amazon RDS Custom for Oracle AWS announces AWS Snowcone SSD Amazon RDS Proxy now supports Amazon RDS for MySQL Version 8.0 Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) announces support for Cross-Cluster Replication Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) now comes with an improved management console AWS Transfer Family customers can now use Amazon S3 Access Point aliases for granular and simplified data access controls Amazon EMR now supports Apache Spark SQL to insert data into and update Apache Hive metadata tables when Apache Ranger integration is enabled Amazon Neptune now supports Auto Scaling for Read Replicas AWS Glue Crawlers support Amazon S3 event notifications Amazon Keyspaces (for Apache Cassandra) now supports automatic data expiration by using Time to Live (TTL) settings New – AWS Data Exchange for Amazon Redshift | AWS News Blog AI & ML SageMaker Announcing Fast File Mode for Amazon SageMaker Amazon SageMaker Projects now supports Image Building CI/CD templates Amazon SageMaker Data Wrangler now supports Amazon Athena Workgroups, feature correlation, and customer managed keys Other Amazon Kendra launches support for 34 additional languages Amazon Fraud Detector now supports event datasets AWS announces a price reduction of up to 56% for Amazon Fraud Detector machine learning fraud predictions Amazon Fraud Detector launches new ML model for online transaction fraud detection Amazon Transcribe now supports custom language models for streaming transcription Amazon Textract launches TIFF support and adds asynchronous support for receipts and invoices processing Announcing Amazon EC2 DL1 instances for cost efficient training of deep learning models Other Cool Stuff AWS IoT Core now makes it optional for customers to send the entire trust chain when provisioning devices using Just-in-Time Provisioning and Just-in-Time Registration AWS IoT SiteWise announces support for using the same asset models across different hierarchies VMware Cloud on AWS Outposts Brings VMware SDDC as a Fully Managed Service on Premises | AWS News Blog AWS Outposts adds new CloudWatch dimension for capacity monitoring Amazon Monitron launches iOS app Amazon Braket offers D-Wave's Advantage 4.1 system for quantum annealing Amazon QuickSight adds support for Pixel-Perfect dashboards Amazon WorkMail adds Mobile Device Access Override API and MDM integration capabilities Announcing Amazon WorkSpaces API to create new updated images with latest AWS drivers Computer Vision at the Edge with AWS Panorama | AWS News Blog Amazon Connect launches API to configure hours of operation programmatically New region availability and Graviton2 support now available for Amazon GameLift Sponsors CMD Solutions Silver Sponsors Cevo Versent

In this special episode, Eoin and Luciano talk about their impression on the announcements from the first day of AWS re:invent 2021. AWS Lambda now supports event filtering for Amazon SQS, Amazon DynamoDB, and Amazon Kinesis as event sources: https://aws.amazon.com/about-aws/whats-new/2021/11/aws-lambda-event-filtering-amazon-sqs-dynamodb-kinesis-sources/ Amazon CodeGuru Reviewer now detects hardcoded secrets in Java and Python repositories: https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-codeguru-reviewer-hardcoded-secrets-java-python/ Amazon ECR announces pull through cache repositories: https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-ecr-cache-repositories/ Introducing recommenders optimized to deliver personalized experiences for Media & Entertainment and Retail with Amazon Personalize: https://aws.amazon.com/about-aws/whats-new/2021/11/recommenders-optimized-personalized-media-entertainment-retail-amazon-personalize/AWS Chatbot now supports management of AWS resources in Slack (Preview): https://aws.amazon.com/about-aws/whats-new/2021/11/aws-chatbot-management-resources-slack/ Amazon CloudWatch Evidently: https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-cloudwatch-evidently-feature-experimentation-safer-launches/ AWS Migration Hub Refactor Spaces - Preview: https://aws.amazon.com/about-aws/whats-new/2021/11/aws-migration-hub-refactor-spaces/ CloudWatch Real User Monitoring: https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-cloudwatch-rum-applications-client-side-performance/ CloudWatch Metrics Insights: https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-cloudwatch-metrics-insights-preview/ AWS Karpenter: https://github.com/aws/karpenter S3 Event Notifications with EventBridge: https://aws.amazon.com/blogs/aws/new-use-amazon-s3-event-notifications-with-amazon-eventbridge/ S3 Event Notifications for S3 Lifecycle, S3 Intelligent-Tiering, object tags, and object access control lists: https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-s3-event-notifications-s3-lifecycle-intelligent-tiering-object-tags-object-access-control-lists/ Amazon Athena ACID Transactions (Preview): https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-athena-acid-apache-iceberg/ AWS Control Tower introduces Terraform account provisioning and customization: https://aws.amazon.com/about-aws/whats-new/2021/11/aws-control-tower-terraform/ Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

On The Cloud Pod this week, the team is running at half-duplex without Peter and Ryan. Plus Cloudflare R2 is here, Facebook died for a day, and AWS releases Cloud Control Plane.  A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. JumpCloud, which offers a complete platform for identity, access, and device management — no matter where your users and devices are located.  This week's highlights

Video: https://youtu.be/Cxaf8E00GMMSlides: https://docs.google.com/presentation/d/1sJSqNy-t-kVxzrWlqMTp_03nI7Zo8Znr7k0f0C6L9ig/edit?usp=sharingTimestamps:[00:00:00] Intro[00:02:17] Part 1 - Components: Code Organization for Real Apps [00:04:26] What we learned from React [00:07:46] Part 2 - Architecture: Choreography vs Orchestration [00:13:05] Retries and Timeouts [00:14:37] Part 3 - Time: React vs Temporal[00:16:34] Elevator Pitch [00:17:13] Programming Model [00:18:44] Comparing React and Temporal Principles [00:19:11] Live Demo: Amazon One Click Button [00:23:49] Talk Recap [00:24:16] React and Temporal Full Comparison [00:24:42] Conclusion: EnablementTranscript [00:00:00] Once again, I want to thank you all for tuning in and joining, React New York 2021 without further ado, I'll pass it on to Shawn. All right, so hi everyone. Hello, React new York. It is my home town in the U S and I miss everyone back in New York. I am currently based in Seattle, but I'm here to talk about React for the Backend. In 2020 I actually thought that I had given my last React talk because I was all tapped out. I had said everything I wanted to say, and then React New York came by and said, do you want to speak? And I was like, oh, I really wanted to speak for React New York. So here's my presentation about what I've been working on and what I think the parallels have been for React. And I think there's some generalizable lessons, even if you don't end up using Temporal. So, the inspiration for this talk came from Guillermo Rauch, the creator of Next.js. And he was the first person to point out that Temporal.io, does to backend and infra what React did to frontend. Temporal engine is quite complex, much like React, but the surface exposed to developers a beautiful render function and I'm a bit upset because he realized there's before me and I have been working on Temporal for a few months now. So important caveats before I start this talk. What I'm presenting to you is alpha for TypeScript. Temporal is typically a goal or Java based application, but we're developing TypeScript and hopefully launching it soon. And then finally "React for the backend" is an analogy, not a design goal.  The way I treat this is like, it's a, it's basically like crabs. And one of the most entertaining facts that I've ever found is that nature has apparently tried to evolve crabs five independent times. And in fact, there's a word in evolutionary biology for it called Carcinization. And of course, this is really good for a lot of memes. So tired convergent evolution is not uncommon, especially when species have similar selecting pressures in their environments, wired. Everything is Crab. And perhaps everything is React, because we have similar design space problems. So I'll tell a little bit of the story through three parts there's Components, and we'll tell it through the story of Uber, talk about architecture, we'll talk through the story of YouTube, and Time will tell you through the story of Amazon. So a lot to cover, I'm going to try to go really fast. Don't worry. I'll share the slides on my Twitter later on. Okay.  [00:02:17] Part 1 - Components: Code Organization for Real Apps  So part one is about components. You see this a lot on YouTube. Probably you're watching now on YouTube or live streaming. And yeah, you know, like three hour live stream and that's it. Very cool. I think we, we know how to break things down and React has really helped us be more productive by being able to break things down into the components and knowing how to compose them together in a predictable way. But there's a lot of things unanswered in things like this in, in full stack, clones of major well-known apps, which is the hard parts. What like a typical Uber trip, we'll have all these steps like search pricing match. Pick-up drop-off rating tipping, payment, email, uh, and so on and so forth. And typically the naive way of organizing all this is basically one after the other, right? Like this is search goes to pricing, goes to matching, goes to pick upgoes to dropoff goes to rating goes to tipping goes to payment, goes to email, imagine that these are all managed by separate teams and scaled independently. Then you realize, like, this is only the happy path. Then you have to throw in a whole bunch of things that can happen along the way. An Uber trip is basically a long running process with humans in the loop and humans are very, very messy by nature. So how would you write an Uber clone? good luck with a lot of data technologies that you would typically reach for just naively, because you will have to discover all these systems and all these use cases and edge cases along the way. So when people say full stack, they often really mean like this half drawn horse meme. I think this is particularly funny so I take every opportunity I can get to show it, but to be honest, a lot of us front end developers are probably the other way the half-drawn Dragon where we're frontend a very good and in the backend, we'll just like, you know, stick some stuff on Firebase and something. And in reality, if you look at the backend systems, most companies, especially at scale, go towards some form of very complex micro service, system. I don't have the chart for Uber, but Hail-0 is probably a good comparison. Netflix, Twitter, and It's not really avoidable. If you want to scale a company to any significant size, you probably have to break them up into independent services because you're going to ship your org chart anyway. [00:04:26] What we learned from React  The thing I realized as a React developer, as a front end developer, is that actually we had a pretty good run in the past seven, eight years of React in terms of the fact that front end developers know how to organize code at least in terms of the component level. So we moved from the jQuery era where everything was just kind of spaghetti all over the place to at least something more organized where event handlers are strongly tied, locally tied with renders, but essentially managed by React's runtime.  So a few key lessons from React that I personally draw [00:05:00] is that you want to have a component and a renderer model. Like, so essentially the user or the developer writes components. And then the react core team writes the render and that handles a lot of the boilerplate that you might typically forget. And this is everything to do with on mounting or having local states. And it gives you a very nice, non-leaky abstraction that you can write. Second, you can also guarantee work and correctness, which is originally what drew Jordan walk to make something like React because he was working on Facebook messenger and there was a lot of inconsistent state within Facebook manager because of the spaghetti code. So correctness, meaning that we embrace functional programming to produce a virtual DOM view is a pure function of state. If you look at the old enough React talks, you will see a lot of v = f(d), so view as a pure function of data. And finally the programming model. We like to say that it's just JavaScript. There's no custom syntax with templating syntax to learn. I think all these three lessons , there are actually a lot more, but all of these three lessons are where I'm going to focus on for this talk. And I think whenever you tackle any programming paradigm, any framework, any design question, you might want to run it through some of these ideas. So whenever I talk about React principles, I always like to bring up the fact that there's this often overlooked repo called react-basic. And it's actually in the official React organization on GitHub. And this is Sebastian Markbage, who is the tech lead of React. And he wrote down six years ago, his principles on what he thinks makes up React on a fundamental basis? No, JSX just like, what are the principles that we're designing for? We are designing for a simple, pure transformation, abstraction, composition, state, memoization. The words that he uses are very theoretical sometimes, but you feel it every single day when you write React. So there's a lot of things else apart from that, that reacts has done for front end programming. Apart from deterministic renders, we have useState with a reduction of boiler plate with unmounting, child components in con in the very careful design composition, um, side effects where, you know, we have used effect or use memo. And actually a lot of people don't know. I don't, I forget my source. I think it's Sophie Alpert, but, one-third of the React code base is actually just normalization of events across browsers. So you don't even have to worry about it. And creating synthetic events for that. They also produce a dev tool and manage a central scheduler and obviously the success of React over the past five, six years has really shown Testament to how great all these decisions have been.  If you want to learn more about the talks that I've done and my perspectives on some of these React principles, I've done three talks. One is at React Rally. The second at JSConf and the third, ,at React Summit. So you can check out my YouTube for more conversations on that. I don't have time here. Okay.  [00:07:46] Part 2 - Architecture: Choreography vs Orchestration  So that was part one where we talked about Components and the React revolution. So part two, we're going to talk about architecture. So a bit, one level higher than just components. And I'm going to motivate this question with a question of how would you write YouTube. And again, if you look on YouTube for how to write YouTube tutorials, you can get full-stack clones of YouTube, which is pretty impressive, you know, write YouTube in three hours using Firebase. That's very impressive. Unfortunately the hard parts of YouTube also come in. And there are a bunch of Googlers actually who actually went and interviewed YouTube engineers on how YouTube works on the backend. There's a bunch of work that goes on in the background. So you need to upload your file. You need to analyze for metadata. You need to split it up into chunks. You need to process these chunks in parallel, and then you need to stitch it back. And by the way, processing, you have to produce an array of formats, right? From like, 240 P to like 1400 P or something like that. And then you have to stitch all these chunks back into the continuous videos that you actually see in stream. You need to notify subscribers, you need to produce automatic captions and you need to produce thumbnails. And that is again, just the happy path. Right. So, what about all these other features? It's. For example, YouTube premiere is a scheduled release of a YouTube video or feeding into the recommendation algorithm. That must be the most craziest batch job in the world. And you need to scale this process, whatever, whatever you design for 30,000 hours of video uploaded every hour. That's the sheer amount of volume that's going on YouTube today, which is just insane. Like, like any design that you make at scale is going to break in some respect. So I think, I think that's, that's really interesting to consider. And I learned about this actually, and I thought more about this because I interviewed one of our users who is Descript (hi! I'm editing this transcript in Descript rn lol). Descript is a audio transcription platform and their entire business is transcribing audio and then making it easy for you to edit audio. I do it for my podcast every single day and millions of people use it. I think it's really cool.  So their problem was that when a user hits transcribe, it kicks off asynchronous multi-stage and parallelized process that involves reading, encoding audio, chunk splitting. external API calls, merging results that may potentially arrive out of order and then verifying their alignment. So there's a lot of [00:10:00] nuance here that can get really tricky. And if any part of the process fails, you need to try it again. So, this is typically the kind of architectures that people build up incrementally over time, as they discover all these use cases and then find holes and patch them because it's too late to rewrite something. There's a lot of decisions that goes into here. And this is normal. This is natural. I think you run into basically the eight fallacies of distributed computing, which has actually discussed or discovered back in 1994 by people at Sun Microsystems. I love these cartoons but it can be a little bit hard to read. So here's a more organized version of them. At the bare minimum, don't forget distributed computing fallacy number one, which is that the network may or may not be reliable or compute may or may not be reliable. So, what that means in practice is that when you're calling system a, B, C, D E F G you may actually need to introduce hardening layers because at every point and you cross system boundaries, you have a chance of failure and that multiplies exponentially, as you have more and more services tied up in your systems like we saw for the Uber example, like we saw for the YouTube example. You need to add in timeouts and retries. And what that means is that you need to persist the number of times you timed out, when you timed out, what jobs you timed up. So you need a database every single time, and then you need a scheduler or a timer to say when the next time is going out, I'm going to try this again. And you need to write this for every service. If the ma the maintainer for every service needs to maintain both the code and the infrastructure for this. This is a lot of how I was talking about things when I was exploring the serverless world. So here's a real life example from the AWS blog where they said that you were using dead letter queues to replay messages when such things as failures occur. This is a fine looking example until you try to scale it. And again it looks like a complete mess, complete track, and it's very hard to keep in your head, and pretty soon when you're explaining this to your CTO you look like the Pepe Silvia meme.  So the solution that I found is really to have a central orchestrator, right? Instead of every single system maintainer writing their own API hardening layer, which is a production requirements, as you find more and more of these bugs, you should centralize it with a centralized team that takes care of the orchestration of all these different services. And that's in the business, what we call choreography, which is A to B to C versus orchestration, which is a central orchestrator coordinating the dance between AB and C, and then storing both the infrastructure and the code for the scheduler and the database. So there's a really good article on this by Yan Cui in the burningmonk.com so I highly recommend checking it out where he talks about choreography versus orchestration, with real life examples that people use in AWS, but also it's not specific to any cloud. It's a architecture design pattern, which I think fundamentally, if you start off with this, it's really hard to rearchitect to this. I mean, it's, it's possible because people are doing it, but also it's a conscious, architectural choice that you might not know that you're making if you don't know about it. So, I guess a lot of my message here is to tell you that orchestration is a thing.  [00:13:05] Retries and Timeouts  Also, so you want to declaratively put into your framework retries and timeouts, so for example, this is actually our API. You want to be able to say, all right, here's the default retry policy. Whenever I fire off an activity, an activity is just like an external API call, for example. So when I fire up an activity, I want it to be retried every second. If it fails, I need a backoff coefficient, like exponential backoff. This is very similar to the TCP protocol so that if the endpoint is failing or getting rate limited, I don't keep retrying, and then building up a DDoS attack on myself, I actually back off and put more and more intervals in between until some maximum interval, let's say a hundred seconds. And then I give myself a maximum attempt, so I can say like, all right. I don't want any retries. I can just say have a maximum attempt of one. Or let's say, I want a linear back off and not an exponential for whatever reason. And I want to try to a maximum of five times - you want to have this all declarative so that you can tweak this as you understand your system and you scale your system. Right? So I think this is a really interesting programming model that just puts retries into the code that you write. And that's only possible when you have your centralized orchestrator, no matter what system, not just Temporal.  Okay. So the case that I'm making is really for choreography versus orchestration. And I, the analogy that I make for front end versus the back end is that it's kind of like vanilla or jQuery versus react. React has a react as the central orchestrator, orchestrating all the components. And I think that's a really interesting architectural analogy that you can make and learn from React. All right.  [00:14:37] Part 3 - Time  Part three - Time. I'm doing very good on time. I think better than I thought, which means that we'll have time for a live demo, which is really awesome. So let's talk a little bit about Temporal.  [00:14:45] What is Temporal?  What is Temporal? Temporal is the open source platform for orchestrating highly reliable mission-critical applications at scale. I love talking a little bit about the history, the reason because our CEO started at Amazon as the tech lead for what became Amazon SQS. Our [00:15:00] CTO was at Microsoft and it was the principal architect of the Durable Task Framework, which became a Microsoft's version of Durable Functions, and then finally they joined Uber and worked on Cadence, which is the open-source version of their workflow orchestration platform and Cadence became so popular that they spun out and became Temporal. And since then it's been adopted by a lot of well-known household name companies, especially in the developer world. There are a lot of people hiring for Temporal developers, which I really like to see because it's not just being used, but also it's creating jobs for people and it's becoming a desirable skillset. And most recently last week we had Netflix presenting about how they used Temporal for their CI/CD.  Temporal has three components or produces three products that are used in sync. The main star is Temporal server, which is comparable to the React runtime that you might see, then there's Devtools, which is the UI that you might want to inspect the state of things. And then the SDK is, which is what you use to code. So I think all those are really comparable to what we have in React and having been in the React world for while, like, it's really amazing to see the analogies that we have. We have exactly the same thing. For me, the really sort of the seal of approval comes from Mitchell Hashimoto who, created Hashi Corp, saying that without Temporal, we would have spent a significant amount of time rebuilding Temporal, which actually to me is the best form of validation because Mitchell is one of the best developers in distributed systems and he says it's hard and he says it does it well.   All right. Enough social proof you want actual facts? I would just give it straight to you.  [00:16:34] Elevator Pitch  So because your workloads like the YouTube encoding, or like the Uber journey and this technology was developed at Uber i s long running and it ties together multiple services. You want to standardize timeouts and retries and you want to make it easy for every team to have production grade retries and timeouts.  Because this work is so important. You must never drop any work. You must log all progress. In other words, you must use event sourcing. And then finally, because this work is so complex, you want to use generic programming languages, instead of Domain specific languages. So you want to model a dynamic, asynchronous logic, and then you want to reuse, test, version and migrated it.  So that's the pitch in one screen. But I'll just break it down for what it means, and then we'll go into a demo.  [00:17:13] Programming Model  So to me, The, the closest analogy to React is the programming model, because React spends a lot of time on API design and in the workflow orchestration world there are a lot of JSON or DAG based domain specific languages. So you, you write a bunch of JSON or you do boxes and arrows boxes and arrows boxes and arrows, sometimes you've even write XML, which is very interesting as well. What I find with all these is that they're actually really good for manipulating visually. But they get very tricky when you need to do programming language constructs, like variables, functions, loops, branching statements and all the things that we've invented in programming languages over the past few years. So if you use "just JavaScript" or "just programming languages", you have all the tooling available. You can use all the libraries that are available. You can use all the testing and code version, quality controls available. If you write your own, you have to rebuild all this dev tooling from scratch for yourself. So that's essentially what this is. Here's an example from one of the big clouds where this is their workflow orchestrator model, where you write Jason and it's really hard. It actually goes off the screen and I couldn't really fit everything on one screen. And with Temporal literal just JavaScript you call an endpoint you use that the result of that end point to call other end points, for example. It's a very simple example, but in built here is default retry policies that have been worked out. So both of these handle reliability on rails, it's just, we differ in the programming model and the engineering that it takes to maintain one of these SDKs is I'm learning. It's very, very immense. So it's really interesting. [00:18:44] Comparing React and Temporal Principles  So, again, back to the core principles that we talked about early on from React. React d ecided on using a framework, decided on correctness and decided on a programming model, and Temporal, in a very similar way. The developer writes workflows and the Temporal core team writes the orchestrator, which is Temporal server. In terms of correctness, React insists on functional programming, Temporal insists on event sourcing in deterministic workflows and then programming model, you want "just JavaScript" or just programming languages, not any custom DSL syntax.  [00:19:11] Live Demo: Amazon One Click Button  So the final example that I'm gonna motivate is which is like, I'm, I've been trying to re progressively reduce the complexity of my examples. So we met from Uber, which is like a super long running, a lot of humans in the loop to YouTube, which is not so much humans in the loop. You upload it once and then everything else takes over from there.  Now I just want to build one feature, which is a one-click buy button in React or in front end. It's actually super easy. It's a button. That's the literal simplest thing you can possibly do. You put an onclick handler. You're done. If you want to do a one-click buy, you do a setTimeout, and then say like, okay, if you want to cancel this within some window, with Amazon is 30 minutes, we can cancel it. But if you want to persist it, imagine if some person clicks, closes the browser and then changes their mind, opens the browser again, and it's gone. You're screwed. You don't have any other way to implement one-click [00:20:00] purchases. You need to implement timers on the backend to do this. I was watching this old talk from Joel Spolsky where he talks about the engineering for the one-click buy button. And I put it up on my YouTube because this is such an old talk. And I was afraid to link to the timestamp, but you can check it out as it's just a three minute video where he tells the story about how Amazon moved from shopping cart to one-click buy I mean they still have a shopping cart but it's that important because in online e-commerce actually even up to today the abandonment rate for shopping carts is 70%. So imagine if you implement this one feature, you improve your sales by I don't what's the inverse of 70%, three times. That's really amazing. So I think it's just fascinating and it's not just about Amazon. It's not about one click buy. It's about user experience. It's about making things easy and intuitive and that often involves turning synchronous things into asynchronous things and in persisting them so that they persist in the background. So I have a little demo here. I'm going to go really, really fast, but you can check out the code in temporalio/samples-node.  There's the specific path this year but it's basically a Next.js demo where I have a Next.js folder here. This is going to be pretty standard for a lot of React developers. Hopefully you're familiar with Next.js, so you can learn it. It's got some pages and an API routes where I have serverless functions that call and send signals to my workflow functions. I have also a Temporal folder where I have written my workflows and activities. The activities are just a little logs obviously, cause they don't interact with any backends, but they could. And then the workflow coordinates the states in the background of all of these.  So I can show you the code, but essentially I kick off a one-click buy with a purchase and then I set a timer and promise.race with a five second wait. So if I receive a cancel signal during that timer then that cancels if not, it goes through and the purchase is confirmed. Obviously I can. And what's fascinating about Temporal is that every single step is persisted in automatically saved. So in other words, I can sleep for 30 days. I can sleep for a year. I can sleep for five years and it doesn't matter because it's all persisted and wakes up automatically. So the compute the, this serverless function can be. The worker or a Temporal server itself can go down. You can just bring you back up again and it carries on as though nothing happened because of event sourcing. So, I'm gonna, I'm going to go ahead and run this. I think it's uh, demos I'm always stressed out about live demos. Okay. I mean, I did test it before the talk. It's just that whenever I'm streaming, like it adds an extra latency thing and that goes haywire. So, Let's see if I have this demo available. All right. So I also want to pull this out, which is the UI layer. These are the my test runs. But I have here at one-click purchase UI, and literally, I, you know, I, I want to implement this without a shopping cart, but I want to be able to cancel within some certain amount of time. So if I click buy, uh, it clicks, it handles it's. It sends a workflow. And that workflow starts in starts in the background and it's running, right. It's waiting for the timer to proceed. So I'm going to hit the timer, uh, and you can see that a timer started and time of ended, uh, within that five second window that I specified, obviously I should make it longer if I, if I really wanted to show this along the way. Um, so, uh, this, this is, this is as is purchased, um, and we can, uh, and now we've confirmed it. Um, but if I ever want to click buy, and then I can click the. That also fires off a different workflow, uh, where it sees that it receives the cancel signal from me. Uh, so, so I signaled it to cancel. And that's a very useful model as well. So this actually shows off a lot of the core principles of Temporal, which is you kick off a workflow, you can set durable timers, you can send it human signals and you can get out data as well with queries. There's a lot of interesting elements behind that, but that's the core demo that I wanted to show off. So maybe I'll write a YouTube example and then I'll go on to an Uber example and be a billionaire.  [00:23:49] Talk Recap  So ultimately I just want it to recap again, what we covered. We covered components, we covered architecture we covered time, and these are all the three elements I wanted to compare reacts and Temporal, and explain a little bit of how we think about doing the hard parts of making clones of very popular projects. Why is it so interesting? It's a little bit like the crabs story, you know. Obviously the founders of Temporal are not front end developers. They didn't even know react at all.  [00:24:16] React and Temporal Full Comparison  But they independently evolved a lot of the same principles and this that's, I haven't even gone into like the full comparison. So we talked a little bit about deterministic functions and local state and composition, but we haven't talked about normalization and how that compares dev tools. Testing is also super interesting thing as well as the central runtime. So there's a lot here, which I think. And fascinated by, and I'm obsessed by applying the lessons from React to things that are not React. [00:24:42] Conclusion: Enablement And I think overall, when I asked my CEO, like, what is the core message that we want to deliver is actually about enablements. Like we enable people to do things that they're not formally trained to do because we wrapped it up rapid all in a central runtime or central framework. So, uh, I always loved the Alfred north Whitehead quote that [00:25:00] civilization advances by extending the numberof things that we can do without thinking about it.   So for me, my version of it is that B2B software advances by extending the number of jobs we can perform without formal training. And the message overall here is that Temporal lets backend developers or, just general full stack developers do distributed systems right? So that's it.  I blasted through that. I only took 26 minutes. Really great for me, cause I was worried that it would take 50 and I'm happy to answer any questions you can hit me up on Twitter at Swyx. You can read my long form blog posts about why Temporal and then you can join our mailing list, YouTube or Slack. Thank you. Alright, thank you very much things. So I think that was a really, really nice. And you did, uh, went through that quite quickly. Uh, when I see the comments, people love the, like the most right there, because I could fail because I could fail. It's always like that. So, uh, yeah. Um, the nice thank you for the presentation. With this talk, I think it's actually the last talk of the event and I want to thanks everyone for joining us and thanks to everyone, thanks to all the speakers, of course, for being part of this event, uh, React New York 2021 and the sponsors. Um, I think this would be a good afternoon, I guess, or good night, depending on where we are in the world. Right. Have a good one. Everyone. 

Links: Let's Encrypt's root certificate has expired, and it might break your devices: https://techcrunch.com/2021/09/21/lets-encrypt-root-expiry/ Slack was bitten by DNSSEC: https://Twitter.com/tqbf/status/1443654964556013569 Prepare For Cybersecurity Assessments From Your Customers: https://www.securitysystemsnews.com/article/prepare-for-cybersecurity-assessments-from-your-customers AWS Lambda now supports triggering Lambda functions from an Amazon SQS queue in a different account: https://aws.amazon.com/about-aws/whats-new/2021/09/aws-lambda-lambda-function-amazon-sqs-queue/ Migrating custom Landing Zone with RAM to AWS Control Tower: https://aws.amazon.com/blogs/mt/migrating-custom-landing-zone-with-ram-to-aws-control-tower/ Introducing the Ransomware Risk Management on AWS Whitepaper: https://aws.amazon.com/blogs/security/introducing-the-ransomware-risk-management-on-aws-whitepaper/ Validate IAM policies in CloudFormation templates using IAM Access Analyzer: https://aws.amazon.com/blogs/security/validate-iam-policies-in-cloudformation-templates-using-iam-access-analyzer/ Pacu: The Open Source AWS Exploitation Framework: https://rhinosecuritylabs.com/aws/pacu-open-source-aws-exploitation-framework/ TranscriptCorey: This is the AWS Morning Brief: Security Edition. AWS is fond of saying security is job zero. That means it's nobody in particular's job, which means it falls to the rest of us. Just the news you need to know, none of the fluff.Corey: This episode is sponsored in part by Thinkst Canary. This might take a little bit to explain, so bear with me. I linked against an early version of their tool, canarytokens.org, in the very early days of my newsletter, and what it does is relatively simple and straightforward. It winds up embedding credentials, files, or anything else like that that you can generate in various parts of your environment, wherever you want them to live. It gives you fake AWS API credentials, for example, and the only thing that these things do is alert you whenever someone attempts to use them. It's an awesome approach to detecting breaches. I've used something similar for years myself before I found them. Check them out. But wait, there's more because they also have an enterprise option that you should be very much aware of: canary.tools. You can take a look at this, but what it does is it provides an enterprise approach to drive these things throughout your entire environment and manage them centrally. You can get a physical device that hangs out on your network and impersonates whatever you want to. When it gets Nmap scanned, or someone attempts to log into it, or access files that it presents on a fake file store, you get instant alerts. It's awesome. If you don't do something like this, instead you're likely to find out that you've gotten breached the very hard way. So, check it out. It's one of those few things that I look at and say, “Wow, that is an amazing idea. I am so glad I found them. I love it.” Again, those URLs are canarytokens.org and canary.tools. And the first one is free because of course it is. The second one is enterprise-y. You'll know which one of those you fall into. Take a look. I'm a big fan. More to come from Thinkst Canary in the weeks ahead.Corey: Somehow we made it through an entire week without a major vendor having a headline-level security breach. You know, I could get used to this; I'll take, “It's harder for me to figure out what to talk about here,” over, “A bunch of customers are scrambling because their providers have failed them,” every time.So, let's see what the community had to say. Last week, as you're probably aware, Let's Encrypt's root certificate expiredwhich caused pain for a bunch of folks. Any device or configuration that hadn't been updated for a few years is potentially going to see things breaking. The lesson here is to be aware that certificates do expire. The antipattern is to do super-long registrations for thing, but that just makes it worse.One of the things Let's Encrypt got very right is forcing 90-day certificate rotations for client certs. When you've got to do that every three months, you know where all of your certificates are. If you've got to replace it once every ten years, you'll have no clue; that was six employees ago.In bad week news, Slack was bitten by DNSSEC when they attempted and failed to roll it out. DNSSEC is a bag of pain it's best not to bother with, as a general rule. DNS is always a bag of pain because of caching and TTL issues. In effect, Slack tried to roll out DNSSEC—probably due to a demand by some big corporate customer—had it fail, panicked and rolled back the change, and was in turn bitten by outages as a bunch of DNS resolvers had the DS key cached, but the authoritative nameservers stopped publishing it. This is a mess and a great warning to those of us who might naively assume that anything like DNSSEC that offers improved security comes without severe tradeoffs. Measure twice, cut once because mistakes are going to show.I also found a somewhat alarmist article talking about cybersecurity assessments from your customers and fine, but it brings up a good point. If you're somehow responsible for security but don't have security in your job title—which, you know, this show is aimed at—you may one day be surprised to have someone from sales pop up and ask you to fill out a form from a prospective customer. Ignore the alarm and the panic but you're going to want to get towards something approaching standardization around how you handle those.The first time you get one of these, it's a novel exercise; by the tenth, you just want to have a prepared statement you can hand them so you can move on with things. Well, those prepared statements are often called things like, “SOC 2 certifications.” There's a spectrum and where you fall on it depends upon who you work for and what you do. So, take them seriously and don't be surprised when you get one.AWS had a few interesting security-related announcements. AWS Lambda now supports triggering Lambda functions from an Amazon SQS queue in a different account. That doesn't sound like a security announcement, so why am I talking about it? Because until recently, it wasn't possible so a lot of folks scoped their IAM policies very broadly; what do you care if any random SQS queue in your own account can invoke a Lambda? With this change, suddenly internet randos can invoke Lambda functions, and you should probably go check production immediately.Announcer: Have you implemented industry best practices for securely accessing SSH servers, databases, or Kubernetes? It takes time and expertise to set up. Teleport makes it easy. It is an identity-aware access proxy that brings automatically expiring credentials for everything you need, including role-based access controls, access requests, and the audit log. It helps prevent data exfiltration and helps implement PCI and FedRAMP compliance. And best of all, Teleport is open-source and a pleasure to use. Download Teleport at goteleport.com. That's goteleport.com.Corey: Migrating custom Landing Zone with RAM to AWS Control Tower. It's worth considering the concept here because, “Using the polished thing” is usually better than building and then maintaining something yourself. You wind up off in the wilderness; then AWS shows up and acts befuddled, “Why on earth would you build things the way that we told you to build them at the time you set up your environment?” It's obnoxious and they need to stop talking and own their mistakes, but keeping things current with the accepted way of doing things is usually worth at least considering.AWS has a whitepaper on Ransomware Risk Management out and I'm honestly conflicted about it. There are gems but it talks about a pile of different services they offer to offset the risk. Some of them—like AWS Backup—are great.Others—“Use Systems Manager State Manager”—present as product pitches for products of varying quality and low adoption. On balance, it's worth reading but retain a healthy skepticism if you do. It should be noted that the points that the address and the framework they lay out is exactly how risk management folks think, and that's helpful.Validate IAM policies in CloudFormation templates using IAM Access Analyzer. I like that one quite a bit. It does what it says on the tin, and applies a bunch of more advanced linting rules than you'd find in something like cfn-lint.Note that this costs nothing for a change, even though it does communicate with AWS to run its analysis. Note that as AWS improves the Access Analyzer, findings will likely change, so be aware that this may well result in a regression should you have it installed as part of a CI/CD pipeline.And as far as tools go, if you're not a security researcher, good; you're in the right place. But that said, if you have a spare afternoon at some point, you may want to check out Pacu—that's P-A-C-U. It's an open-source AWS exploitation framework that lets you see just how insecure your AWS accounts might be. I generally leave playing with those sorts of things to security professionals, but this is a fun way to just take a quick check and see if there's a burning fire that jumps out that might arise for you down the road. And I'll talk to you more about all this stuff next week.Corey: I have been your host, Corey Quinn, and if you remember nothing else, it's that when you don't get what you want, you get experience instead. Let my experience guide you with the things you need to know in the AWS security world, so you can get back to doing your actual job. Thank you for listening to the AWS Morning Brief: Security Edition.Announcer: This has been a HumblePod production. Stay humble.

2021-09-28 Weekly News - Episode 120Watch the video version on YouTube at https://youtu.be/rXPASiVo6ks Hosts: Eric Peterson - Senior Developer for Ortus SolutionsGavin Pickin - Senior Developer for Ortus SolutionsThanks to our Sponsor - Ortus SolutionsThe makers of ColdBox, CommandBox, ForgeBox, TestBox and almost every other Box out there. A few ways  to say thanks back to Ortus Solutions: Like and subscribe to our videos on YouTube.  Sign up for a free or paid account on CFCasts, which is releasing new content every week Buy Ortus's new Book - 102 ColdBox HMVC Quick Tips and Tricks on GumRoad (http://gum.co/coldbox-tips) Patreon SupportWe have 38 patreons providing 100% of the funding for our Modernize or Die Podcasts via our Patreon site: https://www.patreon.com/ortussolutions. We are now 41% of the way to our next goal, fully funding the ForgeBox.io site.Now offering Annual Memberships, pay for the year and save 10% - great for businesses.News and EventsInto the Box 2021 - RecapGreat Success! Especially the Viral Edgardo Cabezas with his Reggaeton - Video available :DCBWire v1.0.0 - https://cbwire.ortusbooks.com/ Qb v8.7.0 & v8.7.1 Released - https://qb.ortusbooks.com/whats-new#8-7-0ForgeBox v6.0.0 Stachebox v1.0.0 - ElasticSearch, LogStash, and FileBeats viewerContentBox v5.0.0 - Using Tailwind and AlpineJS got some love from attendees and twitterMULTI-SITE!!!  New CLI installation and managerNew Ortus Github Actions for CFFormat, DocBox, Fixinator, Semantic Release https://github.com/Ortus-Solutions/fixinator-action https://github.com/Ortus-Solutions/commandbox-action https://github.com/Ortus-Solutions/semantic-release-actionNew Adobe ColdFusion Product Manager - In the Adobe Session Mark Takata had a lot of really encouraging things to say about the new #ColdFusion Product Manager who  is gearing up to take over the direction of CF! For More information:Keynote Day 1: https://youtu.be/NNYK1-NPBeI Keynote Day 2: https://youtu.be/jK8RfFLQZjo Full Session Videos coming to CFCasts soon https://intothebox.orgCommandBox v5.4.1 releasedWe are pleased to announce the release of CommandBox 5.4.1.  This is a patch release with 5 small fixes for the 5.4.0 release.   There are a few bug fixes, and a couple small enhancements in this release.https://www.ortussolutions.com/blog/commandbox-541-released Lucee-Spreadsheet Release and RenameJust released v3.0 of my spreadsheet library. Two big changes: 1) it's now "spreadsheet-cfml" instead of "lucee-spreadsheet" 2) calls can now be chained for simpler code (yeah for fluent builder patterns!)Ortus fixed the forgebox slug so builds will not fail anymore and updating ForgeBox to have failover redirects for future issues in other projectshttps://github.com/cfsimplicity/spreadsheet-cfml/New Name - https://www.forgebox.io/view/spreadsheet-cfmlOld Name - https://www.forgebox.io/view/lucee-spreadsheet Minor Stable Lucee Release Out - 5.3.8.206Mainly a CFSchedule fixhttps://github.com/lucee/Lucee/compare/5.3.8.201...5.3.8.206Modernize Or Die Podcast - SoapBox Edition - Relaunching Oct 7thLuis Majano is hosting a series of Community Members over the coming months.Interview format. Several episodes are recorded and prepped for release.https://soapbox.modernizeordie.io/ Security Alert - Cring ransomware group exploits ancient ColdFusion server (CF9)Active exploit of cf9 servers leading to ransomware attacks. This is why you should not be running software that's no longer updated! Thank for heads up from @viviotech email to their clients, including one we sharehttps://news.sophos.com/en-us/2021/09/21/cring-ransomware-group-exploits-ancient-coldfusion-server/ Ortus Webinar for October - Gavin Pickin - Building Quick APIs - the extended versionIn this session we will use ColdBox's built in REST BaseHandler, and with CBSecurity and Quick ORM we will set up a secure API using fluent query language - and you'll see how quick Quick development can be!https://www.ortussolutions.com/events/webinarsSignup coming soonSome people think October is pumpkin spice, halloween, but for us, its HACKTOBERFESTHacktoberfest 2021Support open source throughout October!Hacktoberfest encourages participation in the open source community, which grows bigger every year. Complete the 2021 challenge and earn a limited edition T-shirt.GIVING TO OPEN SOURCEOpen-source projects keep the internet humming—but they can't do it without resources. Donate and support their awesome work.TREES NOT TEESRather than receive t-shirts as swag, you can choose to have a tree planted in your name and help make Hacktoberfest 2021 more carbon neutral.To win a reward, you must sign up on the Hacktoberfest site and make four pull requests on any repositories classified with the 'hacktoberfest 'topic on GitHub or GitLab by October 31. If an Ortus Solutions repo that you want to contribute to is not marked with the `hacktoberfest` topic, please let us know so we can fix it.https://hacktoberfest.digitalocean.com/ ICYMI - Adobe ColdFusion Updates ReleasedWe are pleased to announce that we have released the updates for the following ColdFusion versions:ColdFusion (2021 release) Update 2ColdFusion (2018 release) Update 12Note: The ColdFusion (2021 release) installers have also been refreshed with this update. The new server installers bundle Update 2 and JDK 11.0.11. The ColdFusion Add-Ons and other installers are bundled with JDK 11.0.11. The refreshed installers are available at ColdFusion downloads.Biggest update here is supporting Big Sur.  (A little late with Monterey just around the corner.)https://coldfusion.adobe.com/2021/09/coldfusion-2021-and-2018-september-updates/ ICYMI - Adobe CF 2021 Docker Images finally on DockerHubAdobe has been putting their ColdFusion Docker images onto Docker hubhttps://coldfusion.adobe.com/2021/09/adobe-coldfusion-2021-docker-image-now-available-dockerhub/ https://hub.docker.com/u/adobecoldfusion ICYMI - Adobe 1 Day Workshop - Adobe ColdFusion Workshop with Damien BruyndonckxWed, September 22, 2021Wed, November 10, 202109:00 - 17:00 CEST EUROPEANJoin the Adobe ColdFusion Workshop to learn how you and your agency can leverage ColdFusion to create amazing web content. This one-day training will cover all facets of Adobe ColdFusion that developers need to build applications that can run across multiple cloud providers or on-premise.https://coldfusion-workshop.meetus.adobeevents.com/ ICYMI - Adobe Webinar for September - Head in the clouds: Installing ColdFusion on Azure and AWS with Mark TakataWed, September 22, 202109:00 - 10:00 PDTIf you've been thinking about getting your CF instance moved into the cloud but don't know where to start, this webinar will get you rolling. We will explore how to install, configure and update CF2021 installs in both Amazon AWS and Microsoft Azure and give you tips and tricks on how to get things off the ground.https://installing-coldfusion-on-azure-and-aws.meetus.adobeevents.com/ CFCasts Content Updateshttps://www.cfcasts.com Just ReleasedUp and Running with Quick 25 - Serialization 26 - Step 10 Exercise 27 - Subselects and Relationship Counts Coming this weekMore Up and Running with QuickSend your suggestions at https://cfcasts.com/supportConferences and TrainingInto the Box 2021 - HYBRID - Live in Person in Texas. and Live OnlineGreat Success! Keynote Day 1: https://youtu.be/NNYK1-NPBeI Keynote Day 2: https://youtu.be/jK8RfFLQZjo Full Session Videos coming to CFCasts soon https://intothebox.orgAdobe ColdFusion Summit 2021December 7th and 8thMark let us know the Keynote Speaker will be someone from outside of Adobe - will announce once the contract is inked. She works for a big company.VirtualRegister for Free - https://cfsummit.vconfex.com/site/adobe-cold-fusion-summit-2021/1290Blog - https://coldfusion.adobe.com/2021/09/adobe-coldfusion-summit-2021-registrations-open/ More conferencesNeed more conferences, this site has a huge list of conferences for almost any language/community.https://confs.tech/Blogs, Tweets and Videos of the WeekBlog - We are Orange - From MURA to MASAFor those that don't know, MURA is a very powerful CMS that is renowned for it's fantastic UX – especially for content managers. Previously it was offered as an open source platform but that support ended in August of this year.As seasoned MURA users – with some big clients running it – this could have presented a major problem. So, we came up with a solution… and we're proud to announce that We Are Orange has forked the project and are now supporting a new version – called MASA – continuing its lifespan as an open source platform.https://www.weareorange.eu/updates/from-mura-to-masa Blog - Ben Nadel - Exploring The Scope Of SERIALIZABLE Transaction Row-Locking In Lucee CFML 5.3.7.47Earlier this week, I looked at using SERIALIZABLE transactions to enforce unique naming constraints in MySQL and Lucee CFML. Transaction isolation isn't a topic that I'm super comfortable with. As such, I wanted to continue exploring the way in which transaction locking affects concurrent access to a given database table. More specifically, I wanted to better understand the scope of row-locking that is applied in a SERIALIZABLE transaction in MySQL 5.7.32 and Lucee CFML 5.3.7.47.https://www.bennadel.com/blog/4122-exploring-the-scope-of-serializable-transaction-row-locking-in-lucee-cfml-5-3-7-47.htm Blog - Ben Nadel - CFThread "ElapsedTime" Is Not "Processor Time" In Lucee CFML 5.3.8.201This morning, as I was trying to perform some follow-up exploration on SERIALIZABLE transactions in MySQL, I noticed something strange about the metadata exposed by the CFThread tag in Lucee CFML. Whereas Adobe ColdFusion (ACF) documents the ElapsedTime property as being "The amount of processor time that was spent handling the thread", this property looks to mean something different in Lucee CFML. It appears to be the amount of time that has elapsed since the CFThread tag was spawned; but, has nothing to do with when it ended; or, how much time it took to execute.https://www.bennadel.com/blog/4121-cfthread-elapsedtime-is-not-processor-time-in-lucee-cfml-5-3-8-201.htm Blog - Ben Nadel - Fixing Protocols In My ColdFusion Custom Tag DSL For HTML EmailsLast week, I looked at the fact that [Yahoo! Mail won't render href attributes with encoded protocols]. In that post, I created a ColdFusion user defined function (UDF) to un-encoded the https:// portion of the href attribute. After letting that approach bake in production at InVison for a while without an issues, I've decided to move the fix into my ColdFusion custom tag DSL for HTML emails. With this new implementation, I've pushed the logic down into the and custom tag implementations so that the developer can continue to use these tags without any changes to their code.https://www.bennadel.com/blog/4123-fixing-protocols-in-my-coldfusion-custom-tag-dsl-for-html-emails.htm Blog - Ben Nadel - Request Tracing Propagation When Consuming Amazon SQS Queues In Lucee CFML 5.3.8.201As I continue to explore the use of Amazon SQS queues in Lucee CFML, I have to start thinking about request tracing. In a monolithic application, request tracing is much less of a value-add since most operations happen within a single request. Once I start using a message queue to decouple steps within a larger workflow, however, I break out of that single-request boundary. As such, if I want to identify "message creation" as being related to "message consumption", I have to propagate the concept of tracing through the message queue life-cycle. I'm sure there are loads of ways to do this. So, what follows is just one possible way to propagate request tracing through Amazon SQS in a Lucee CFML 5.3.8.201 application.https://www.bennadel.com/blog/4118-request-tracing-propagation-when-consuming-amazon-sqs-queues-in-lucee-cfml-5-3-8-201.htm Tweet - Ben Nadel - Too Clever or Too Cool - Execute assignments as part of return statementsI am sure many people will hate this syntax / technique. But, I love the fact that #Lucee #CFML allows me to execute assignments as part of return statements. This allows my #ColdFusion to look more and more like my #JavaScript. Both of which are my happy place!

2021-09-14 Weekly News - Episode 119Watch the video version on YouTube at https://youtu.be/kLHvpYdI_Ds Hosts: Eric Peterson - Senior Developer for Ortus SolutionsGavin Pickin - Senior Developer for Ortus SolutionsThanks to our Sponsor - Ortus SolutionsThe makers of ColdBox, CommandBox, ForgeBox, TestBox and almost every other Box out there. A few ways  to say thanks back to Ortus Solutions: Like and subscribe to our videos on YouTube.  Sign up for a free or paid account on CFCasts, which is releasing new content every week Become an ITB Sponsor - https://www.intothebox.org/#sponsors-2021  Buy Ortus's new Book - 102 ColdBox HMVC Quick Tips and Tricks on GumRoad (http://gum.co/coldbox-tips) Patreon SupportWe have 38 patreons providing 100% of the funding for our Modernize or Die Podcasts via our Patreon site: https://www.patreon.com/ortussolutions. We are now 41% of the way to our next goal, fully funding the ForgeBox.io site.Now offering Annual Memberships, pay for the year and save 10% - great for businesses.News and EventsSpecial Into the Box Announcement - ITB 2021 is now Hybrid!!!!We will still be doing live in person, but with Covid numbers rising sharply, we are now offering online streaming tickets to all of those not willing or able to attend in person.Check your tickets now!If you sign up for a 1 year subscription to CFCasts by 9/22 you will get a free virtual ticket to ITBNo Podcast September 21stWith everything happening next week with the Into the Box Conference - we will not be live streaming a podcast that week. We will be back, with a full ITB Recap on September 28th, after the conference. Adobe ColdFusion Updates ReleasedWe are pleased to announce that we have released the updates for the following ColdFusion versions: ColdFusion (2021 release) Update 2 ColdFusion (2018 release) Update 12 Note: The ColdFusion (2021 release) installers have also been refreshed with this update. The new server installers bundle Update 2 and JDK 11.0.11. The ColdFusion Add-Ons and other installers are bundled with JDK 11.0.11. The refreshed installers are available at ColdFusion downloads.https://coldfusion.adobe.com/2021/09/coldfusion-2021-and-2018-september-updates/ Adobe CF 2021 Docker Images finally on DockerHubI'm happy to be able to report that Adobe has finally started putting their ColdFusion Docker images onto Docker hub, for now at least CF2021, which I found was put there last week. We can expect more in time, I'm sure. For more info, read on.https://coldfusion.adobe.com/2021/09/adobe-coldfusion-2021-docker-image-now-available-dockerhub/ https://hub.docker.com/u/adobecoldfusion Adobe 1 Day Workshop - Adobe ColdFusion Workshop with Damien BruyndonckxWed, September 22, 202109:00 - 17:00 CEST EUROPEANJoin the Adobe ColdFusion Workshop to learn how you and your agency can leverage ColdFusion to create amazing web content. This one-day training will cover all facets of Adobe ColdFusion that developers need to build applications that can run across multiple cloud providers or on-premise.https://coldfusion-workshop.meetus.adobeevents.com/ Adobe Webinar for September - Head in the clouds: Installing ColdFusion on Azure and AWS with Mark TakataWed, September 22, 202109:00 - 10:00 PDTIf you've been thinking about getting your CF instance moved into the cloud but don't know where to start, this webinar will get you rolling. We will explore how to install, configure and update CF2021 installs in both Amazon AWS and Microsoft Azure and give you tips and tricks on how to get things off the ground.https://installing-coldfusion-on-azure-and-aws.meetus.adobeevents.com/ CFCasts Content Updateshttps://www.cfcasts.com Just ReleasedUp and Running with Quick Step 8 Exercise 23-BelongsToMany Relationship Step 9 Exercise Coming this weekMore Up and Running with QuickSend your suggestions at https://cfcasts.com/supportConferences and TrainingInto the Box 2021 - HYBRID - Live in Person in Texas. and Live OnlineSeptember 23rd and 24th.Speakers and Schedule Released!!https://intothebox.orghttps://itb2021.eventbrite.comAdobe ColdFusion Summit 2021December 7th and 8thVirtualRegister for Free - https://cfsummit.vconfex.com/site/adobe-cold-fusion-summit-2021/1290Blog - https://coldfusion.adobe.com/2021/09/adobe-coldfusion-summit-2021-registrations-open/ More conferencesNeed more conferences, this site has a huge list of conferences for almost any language/community.https://confs.tech/Blogs, Tweets and Videos of the WeekBlog - Adobe - Adobe ColdFusion Updates ReleasedWe are pleased to announce that we have released the updates for the following ColdFusion versions:ColdFusion (2021 release) Update 2ColdFusion (2018 release) Update 12Note: The ColdFusion (2021 release) installers have also been refreshed with this update. The new server installers bundle Update 2 and JDK 11.0.11. The ColdFusion Add-Ons and other installers are bundled with JDK 11.0.11. The refreshed installers are available at ColdFusion downloads.https://coldfusion.adobe.com/2021/09/coldfusion-2021-and-2018-september-updates/ Blog - Ben Nadel - Producing And Consuming Amazon SQS Messages In Lucee CFML 5.3.8.201A year ago, I shed some light on my quest to merge microservices back into my ColdFusion monolith. For my size team and the type of domain boundaries that we have, a monolith just makes the most sense. And while I've been thrilled with the progress my team has made in its re-consolidation efforts, there's one more microservice that I've had my eye on for a long time. This microservice has eluded my grasp, however, because it interacts with a message queue. And, to date, all parts of the legacy platform that deal with message queues live outside of the monolith. As such, if I ever want to pull this microservice back into the monolith, I have to figure out how to use message queues in ColdFusion. To start with, I wanted to see if I could simply produce and consume Amazon SQS (Simple Queue Service) messages in Lucee CFML 5.3.8.201.https://www.bennadel.com/blog/4110-producing-and-consuming-amazon-sqs-messages-in-lucee-cfml-5-3-8-201.htm Blog - Ben Nadel - Separation Of Concerns When Consuming Amazon SQS Queues In Lucee CFML 5.3.8.201Last week, I started to explore the consumption of Amazon SQS (Simple Queue Service) in a Lucee CFML application. That first post was a low-level look at the mechanics of using the AWS (Amazon Web Services) Java SDK to add and remove messages to and from a given queue, respectively. Today, I wanted to step back and start thinking about the separation of concerns in the larger ColdFusion application context.https://www.bennadel.com/blog/4111-separation-of-concerns-when-consuming-amazon-sqs-queues-in-lucee-cfml-5-3-8-201.htm Blog - Ben Nadel - Checking CFLock Acquisition Success In Lucee CFML 5.3.8.201Yesterday, in my post about considering the separation of concerns when consuming Amazon SQS queues in Lucee CFML, I created a demo in which I synchronized long-polling requests through the use of an exclusive CFLock tag. For the demo, I had logging in place to trace the workflow of each request; and, as I was putting it together, I noticed that the CFLock tag - at least in Lucee CFML - has an optional result attribute. This result attribute can be used to check whether or not the template was able to successfully access a given lock.https://www.bennadel.com/blog/4112-checking-cflock-acquisition-success-in-lucee-cfml-5-3-8-201.htm Blog - Charlie Arehart - Adobe ColdFusion 2021 Docker image now available on dockerhubI'm happy to be able to report that Adobe has finally started putting their ColdFusion Docker images onto Docker hub, for now at least CF2021, which I found was put there last week. We can expect more in time, I'm sure. For more info, read on.This is a follow-up to my post in July reporting how Adobe had announced then their plans to create a Dockerhub repo for ColdFusion, and how they had already started to post images to the Amazon ECR repo for ColdFusion.https://coldfusion.adobe.com/2021/09/adobe-coldfusion-2021-docker-image-now-available-dockerhub/ Blog - Charlie Arehart - Finding default/initial CF admin config (neo-*.xml) files, now at cfmlrepo.comHave you ever wished to obtain a copy of one CF's neo-*.xml files (like neo-cron.xml), for the purpose of setting yours back to its defaults? Folks sometimes need to do that to recover from certain problems.I've seen the problem raised often enough that when I saw someone raising it this weekend, I decided to solve it by creating a new folder in the cfmlrepo.com site, at least for CF2021 and CF2018 (for now), offering there the initial versions of all the neo-*.xml files for those two editions.https://www.carehart.org/blog/client/index.cfm/2021/9/13/finding_default_cf_admin_config_files_now_at_cfmlrepo_dot_com/ Tweet - James Moberg - Non Breaking Space Highlighter for VS CodeThe "Non-breaking space highlighter" @code extension has been beneficial in identifying non-ASCII spaces.  ow.ly/MyZT50G5IZj (I made the mistake of using @evernote for #cfml code snippets not knowing that it would add lots of UTF-8 chars to preserve spacing.)https://twitter.com/gamesover/status/1435303007726587904https://twitter.com/gamesoverBlog - Java - Introducing the Free Java LicenseOracle is making the industry leading Oracle JDK available for free, including all quarterly security updates.  This includes commercial and production use.https://blogs.oracle.com/java/post/free-java-license https://www.oracle.com/java/technologies/javase/jdk-faqs.html CFML JobsSeveral positions available on https://www.getcfmljobs.com/Listing over 190 ColdFusion positions from 85 companies across 111 locations in 5 Countries.9 new jobs listed (3 possible duplicate positions)Full-Time - ColdFusion Web Developer for SaaS Provider at Remote - United States Sep 14https://www.getcfmljobs.com/jobs/index.cfm/united-states/ColdFusion-Web-Developer-for-SaaS-Provider-at-Remote/11337 Full-Time - Sr. ColdFusion Developer at Alexandria, VA - United States Sep 14https://www.getcfmljobs.com/jobs/index.cfm/united-states/Sr-ColdFusion-Developer-at-Alexandria-VA/11336 Full-Time - ColdFusion Developer | 4 to 6 years | Pune at Pune, Maharash.. - India Sep 13https://www.getcfmljobs.com/jobs/index.cfm/india/ColdFusion-Developer-4-to-6-years-Pune-at-Pune-Maharashtra/11335 Full-Time - ColdFusion Developer | 4 to 6 years | Pune at Pune, Maharash.. - India Sep 13https://www.getcfmljobs.com/jobs/index.cfm/india/ColdFusion-Developer-4-to-6-years-Pune-at-Pune-Maharashtra/11334 Full-Time - ColdFusion Developer at Herndon, VA - United States Sep 11https://www.getcfmljobs.com/jobs/index.cfm/united-states/ColdFusion-Developer-at-Herndon-VA/11333 Full-Time - ColdFusion Developer at Herndon, VA - United States Sep 10https://www.getcfmljobs.com/jobs/index.cfm/united-states/ColdFusion-Developer-at-Herndon-VA/11332 Full-Time - ColdFusion Developer at Ocean Isle Beach - United States Sep 09https://www.getcfmljobs.com/viewjob.cfm?jobid=11331 Full-Time - ColdFusion Developer at Oklahoma City, OK - United States Sep 08https://www.getcfmljobs.com/jobs/index.cfm/united-states/ColdFusion-Developer-at-Oklahoma-City-OK/11330 Full-Time - ColdFusion Developer at Oklahoma City, OK - United States Sep 08https://www.getcfmljobs.com/jobs/index.cfm/united-states/ColdFusion-Developer-at-Oklahoma-City-OK/11329 ForgeBox Module of the WeekUpchunk by Michael BornChunked uploads

Video version on youtube: https://www.youtube.com/watch?v=ErO9UjccwdsMy conversation with Jaden Baptista introducing him to Temporal.Transcript[00:00:00] swyx: Hey everyone is Swyx. I've been gone for the past couple of days and you may or may not be wondering what's happened. I basically got a cold and at this conference, it's actually nice to have a conference cold. I guess, because that means that things are going back to normal, even the, not so great stuff about being meeting back in person. [00:00:22] But I have a cold and I did not feel great. And I think gets COVID. But anyway, I decided to take a little bit of a break, but I also, I'm not sure if you know, but the weekday topics are all done on the same day. And I tried to do this pattern of batching things in weekly themes and this week's theme. [00:00:43] Basically it was poorly chosen. I thought that I had enough to do some, some stuff on basically the how technologies get adopted or get their traction. But I just didn't I ended up not liking any of the other episodes or any other podcasts that I shortlisted for that particular topic. So I just ran out and I just didn't feel inspired. [00:01:06] I felt trapped in the format and didn't really know what to do with it. So I ended up not doing anything. I also had other work to catch up on, on top of the conference stuff. So that was all in my head recently. And in general, and wondering whether or not I'm going in the right direction. So if you do that, And I really am grateful. [00:01:26] Checking out this podcast, because it's like a weird experiment with no particular theme or direction apart from cause it's stuff that I like. If you are, if you have a strong opinion and if you particularly think that there's some ideas that I should just pursue further and you're just mad at me for not doing. [00:01:42] Now's the time to tell me, because I'm also a little bit direction as to where this podcast could go. That's it? I do think that there's a strong thesis for exploring or having a central place for exploring ideas. I am personally interested in and recent dictating my podcast appearances on other people's podcasts to my own feed. [00:02:03] So people who are interested in what I do can follow up directly. So here's a conversation that I had with Jen Battista on Twitter spaces that was recorded. And it's about Tim Poro. So this is the first of maybe. Three podcasts that I did in the last month or so. So I'll be dripping them out over the weekends the next couple of weeks. [00:02:23] And we'll see, we'll see where this goes. I want to get back more into writing. I still have hopes of spinning out my YouTube more seriously, but I think the creative journey. The part-time creative journey of these, where I have a day job and that should take priority over the other side of stuff. [00:02:42] I do dictates the format of the things that I can take on. So, so far the podcast thing has been really great. I actually have a lot, a long backlog of things, which I selected for sure. They just don't fit in any, any particular theme. I think that theme is really nice when I can do it. And just, maybe you don't have enough backlog to do it just yet, so maybe I might go back to not having themes anyway. [00:03:06] So here's my conversation with Jane and review stuff. Well, thank you for  [00:03:09] Jaden Baptista: joining me today, Sean. I really appreciate you taking the time are you doing today?  [00:03:14] swyx: Very good. I'm very entertained by your Twitch title called temporal nutso temporary. Very interesting.  [00:03:22] Jaden Baptista: Oh yeah. I was trying to come up with like a S a stupid clever name for the Twitch streams, despite them not really helping out with what the stream is usually ends up being about. [00:03:34] We tend to wander from topic to time.  [00:03:37] swyx: Sure sure. Yeah. Thanks for having me in a happy to chat to portal. Awesome.  [00:03:42] Jaden Baptista: Yeah. Well, let me ask you just the first quick question. You know, every, every big program that we all talk about w we really enjoy using was, was built to solve a problem, some sort of problem. [00:03:55] What problem was tempura both too soon, but what was the point of building it in the first.  [00:03:59] swyx: So, to be clear, I did not build it. Who did exactly it was built to solve the problem of The abstract problem that the category of problem, this is called is workflows anything long running that needs to take anything more than a simple request response cycle, a request response will be just like, you know, you're paying a serverless function. [00:04:20] It gets back to you in, let's say 300 milliseconds, right? That's a typical cycle, but sometimes you need to do long running work. Typically I would think this is something like video projects. So, if you kick off a job, it takes like four hours to transfer code audio file. But actually it is both longer and shorter than that. [00:04:40] So, this is actually a topic of my recent blog posts. Because even if you, so Dropbox, so box is one of our users and they use us for file transfers and normally a file transfer just feels instantaneous until you try to transfer a million files. Right. They just change that. A thousand of them. [00:04:58] So you need a solution that scales pretty nicely from like a single transfer that should feel instantaneous to you know, something that's gonna take a while. Cause, cause it's just distributed across a lot of systems and you need it to be perfect or you lose data. And if your box you cannot lose data the other super long running task is why does anything have to ever end? [00:05:17] So what if you could just model the entire journey of your customer from. Like their first contact with you to the time that they unsubscribe. And that is a single entity that you just interact with. So it then becomes easy to say things like, okay, on their seventh visit, send them a coupon. Every month charged them based on their, you know, their, their usage or the billing. [00:05:38] And all of this is encapsulated in a single function. So  [00:05:43] Jaden Baptista: yeah.   [00:05:44] swyx: Yeah, because because we have an internal basically, so. Solves, it brings together a database, a scheduler some networking, some search capability and it, and it does. And then it uses, it offers you an SDK, so you can write it in idiomatic language. [00:06:01] The, there, this is not a new problem. Our founders have been working on this for something like 20 years. The tech leads for Amazon SQS. And then in simple workflow Azure doable functions and then when they arrived at Uber, they, they built the initial version of temporal where it now power is like, Something like 400 use cases at Uber mostly driver onboarding marketing which they call communications or something like that. [00:06:25] It's just like, whenever you do something that is so fundamentally asynchronous, you could use it for a lot of things. And it tends to grow that way. So, you know, it was open source that Uber and then it was adopted by other companies like Coinbase. Airbnb Stripe and then once it lands in a, in a company, it just starts growing from there. [00:06:43] And it's just a, it's just a really interesting thing to watch when you solve a problem as well. You get a really strong developer adoption and that's, that's one of the reasons I joined it, just because you know, a bit about my, my personal history, like I typically have been on the front end and serverless side working at another fire in AWS. [00:07:00] But the thing I think that sort of doesn't do well, is anything long running. And the more time you spend studying developer businesses, the more time you understand that, actually, a lot of the money, a lot of value is in the long running stuff. So we've got to do this. And we got to do this and we're going to make it easy to do asynchronous stuff. [00:07:22] And when I say easy, I mean, easy to do production grid services. So, things like retries failures, timeouts, heartbeats you know, when, when systems go down do you have an easy way to. Bring it back up again. If systems go wrong, do you have a way to have everything tracked and audited? Do you have everything properly tracks so you can go back and see what happens, right? [00:07:43] Are you, is your as your monitoring practices? Correct. And then finally, When you need to scale when, when stuff is going really, really well, can you just throw more machines at it? Sometimes if you have the wrong system design, you can not, because you've just, you've just assume a single dependency somewhere. [00:07:57] So every part of your system needs to be horizontally, scalable, completely fault tolerant. And that's what tomorrow brings.  [00:08:03] Jaden Baptista: Awesome. I mean, I, I'm going to be going to be totally honest with you. I, before interacting with you for the first time, never heard of temper. But I ran into this problem a lot. [00:08:15] When I got first into development, I wanted to schedule things and kind of like model a workflow around that schedule instead of the other way around. And we, we ended up switching to like a weird monolithic backend just to get that done with the first projects as a developer. And it was miserable. I hated it. [00:08:36] That's actually why I got into the jam sag in the first place. Cause I hated those monolithic backs. So much that we tried to use this, solve this problem. But I'm curious, one thing that we ran into and by the way, anybody who's listening, if you have questions for Sean about temporal feel free to ask it to be a speaker or in the chat in Twitch. [00:08:52] I'm watching both of them, but my, my first question I've run into situations where that schedule is not set right. So, you know, perhaps with your Uber onboarding flow, maybe there's a set set of steps that you've, you've got that you're going to use to onboard this user, but that might change on a lot of different platforms. [00:09:15] Like, for example, the one I was working on we didn't know what email we would be sending a month down the line because it completely depends on what happened in the meantime. How, how would temporal handle something like that? Is there built in flexibility if the schedule were to change?  [00:09:32] swyx: Oh, migrations. [00:09:33] Yeah. Yeah, absolutely. In fact that's one of the key selling points. So they're the three select, the three key selling points. I'll go in reverse order. Cause that's, that's where we happen to be starting. One of the key selling points is that you want to model workflows as code and that's pretty much. [00:09:47] We're one of the only few workflow engines that do that. And by the way, there are others. That's why, that's why we do have to have our talking points straight. So when you migrate, you need migration, tooling, and you need tests, right. To move with confidence. Would it surprise you to learn that most people, when they set up their asynchronous workflows have no idea. [00:10:06] They, they just they're there. They're happy that it works. That's about it. And then when it, when they moved,  [00:10:14] Jaden Baptista: sorry,   [00:10:16] swyx: when they, when they made changes, they they're just like, okay, should work now. You know, and then, and that's about it. That's the state of the industry is pretty sad. But I, I. Once you have the right abstraction, things just really lock into place. [00:10:29] And being the main thing about tests is that you have to be able to mock out every single possible source of determinism in time. And that's one of the reasons why we're called to Poro. We help you to handle time. And one of the pictures actually for the front end, Because we're going to launch our JavaScript SDK in October. [00:10:47] That's off the record. I, I, I didn't, I did not promise that like, we are components for time, right? So we help you mock out time. We help you test time. We help you structure asynchronous workflows in the same sort of component ties, logic that you might with a reactor view of whatever components. [00:11:03] And I think that that pitch. I'm testing it still and I'm still working on it. That pitch would probably make sense to, to front end developers who are like, yeah. Okay. You know, I have rendering solved, but like what happens to like all the asynchronous stuff? I'm still a black box  [00:11:21] Jaden Baptista: that that's kind of the, the situation that, that I ran into too. [00:11:25] Like I got all of them. That'd be awesome.  [00:11:27] swyx: What was, what was the tech stack? What did you use?  [00:11:30] Jaden Baptista: So at first we were trying to go with like a, a sort of like almost the JAMstack kind of thing. It was very bare bones. This is when I was first getting into web development and I went all gung-ho on this project with the classmate of mine that did not turn out well, but it was, it was basically just a simple JAMstack site actually hosted on Heroku. [00:11:50] Cause we didn't know any better. And we ended up trying to solve like the problem. Of sending messages back and forth. And then depending on the content of those messages doing something like a month later, like sending an email with a recap of some of those messages, or if there were no messages sent and do something else, we ended up actually having to create a PHP backend, like a second backup. [00:12:17] And then a third one later in Python. So we actually had three backends for this problem because the only tools that we could find were all over the place like I said, we didn't know what we were doing, but we were, we're all over the place trying to solve this problem. And so talking to you, it's very fascinating to me that somebody had this problem 20 years ago and decided to fix it. [00:12:37] Yeah,  [00:12:41] swyx: there's a weird, there's an interesting pipeline, I think. And you know, this is a more general career reflect. The pipeline is, is as such you work on a large web-scale company early on in your career. You face all these problems at scale early on, and then you, you leave that company and then you start the problem. [00:12:59] You start a company to solve that problem for everyone else. Right. It's kind of that, that's a lot of the the Google too. There's there's a lot of ex Googlers. So, I think now we're starting to see the emergence of the Uber and Lyft mafia who solve a lot of problems that we were in Lyft. And now they're just emerging as like, you know, independent startups get hub, you know, spawn kind of scale. [00:13:23] Lyft respons there's, there's a bunch of, there's a bunch of them. Twitter  [00:13:26] Jaden Baptista: has done that a lot too people who worked at Twitter and then spun off and did something else, like fauna comes up.  [00:13:33] swyx: Yes. Yes. So, it's interesting. So, so this is, you know, this is an export of Amazon, Microsoft and Uber. [00:13:39] Cause that's the path that, or, [00:13:44] yeah. So, so, I want to give you, yeah. So that's the other thing, right? I want to, I want to really impress upon you this point, which is that you want to, so the, the, the first, the first talking point was, was worthless. Second talking point is about orchestration that you want a single engine that handles retries, timeouts, heartbeats, all that stuff. [00:14:05] And. Anytime you call an API, it actually has a, has a probability of failure, which is like you're calling you're you're crossing network boundaries. You were you know, running into rate limits, whatever, whatever that is. Have you handled it correctly? And that error handling is something that you don't typically think about. [00:14:22] Cause when you code you're typically coding. But having it properly. Exactly. And especially if you think about this in a large company setting you have service call service, a calling service B it's called service C each of these services cannot trust the others essentially. Right? Each of each other, every time you cross the service boundary, you're essentially entering untrusted territory in some sense of the word. [00:14:43] So in order to ensure the reliability of your service or that a response very well, you start building up defenses against each other service. So around the boundaries of your service, you're building in like, okay, like, you know, if, if this call failed, like wait five minutes and we try again, make them and that kind of thing. [00:14:59] And if you run a large enough company, you realize that every team is doing this and. In their own way. And they're all solving problems, incident by incident like this, you don't have to sit down and. Design this from scratch, like you run into an incident and you're like, oh, okay. Yeah, we better put in some production hardening on this thing. [00:15:16] Right. And they're all provisioning infrastructure. It's not just about duplication of code. They're also provisioning infrastructure, right? Like you went from running. Just one little machine somewhere toward to like, okay, now I need a database because I need to store what requests I've made what tracking everything. [00:15:32] Now I need a scheduler, like a Cron job to, to retry things. And you just like build up a lot of. And so what orchestration engines do is they provide reliability on rails. You have a CTO that says, okay, I have a central platform team that does this. Everyone just like use this platform and we'll scale it for you. [00:15:50] Now, you never have to think about the production hardening between your system boundaries. Again. You can just reliably. Things and, and they'll behave properly and feel according to a, a company-wide standard that we can predict and examine this also centralizes failures. So, so in other words, like, if you, if you could like one of the fun jokes about microservices is that they turn every outage into a murderer. [00:16:13] Like who who's at fault. Right. You know, then you go through the trace and then you, you, you, that's the  [00:16:18] Jaden Baptista: one, the one I've heard is a murder mystery where you're both the detective and the murderer.  [00:16:22] swyx: Exactly, exactly. The way to solve it is to have a global view of, of. What happens? So, so yeah, I mean, there's, there's that, which, which I think is, is pretty under appreciated by a lot of people. [00:16:31] And I think it's, it's very core to what we do, but also what a lot of other people do. So we're not the only workflow engine out there. You can try in other, in different contexts Amazon step functions Apache airflow and there's a bunch of others. I don't have to go through all of them. [00:16:47] Jaden Baptista: No, there's a really long list and I've looked into a lot of these, cause like I said, I've seen this problem a lot. And I'm just wondering a lot of those other ones, there was, there was a ton of setup work with those other ones. Like I've tried step functions and I, I actually didn't even get to the point of creating the step function because before that there was so much. [00:17:11] What's the word I'm looking for. Administrative stuff, just to get to that point is how much do you have to do to get temporal working for a really small project? Because you said it will scale with you as, as you start growing bigger. So what, what's the, the investment of time that you'd have to put into it, like a hobby project  [00:17:33] swyx: That's an interesting question for us right now. [00:17:35] It depends on your skill with dev ops, essentially. [00:17:37] So   [00:17:39] Jaden Baptista: is going to help out a lot with this.  [00:17:41] swyx: We still need to do a lot of work. Maybe not coming. We still, we still need to do a lot of work on that. So essentially right now it's a, it's a single go binary. That is that's that runs four services and those four services can be scaled independently. [00:17:55] You will also need a persistence layer. So we support Cassandra my sequel. And then optionally, we also support visibility search. So, using elastic search. So if you want to run all of that you can, we're not, we're not opinionated on how you deploy that. You can try to run it on a single box but also you probably wants to have like a managed database layer. [00:18:15] So you don't, you don't have. Handle the database stuff yourself. But it is very easy to get up and running with our Docker proposed file. So that's the way that we typically get people to get started. If you run Kubernetes, we have a helm chart that you can reference. We definitely don't you'll probably want to customize a home in some way. [00:18:31] People have all sorts of other container orchestration and stuff that they might want to use. But that is essentially the story right now. What it will be in the future is that we will have a single binary that you can one-click deploy on Heroku. Right. [00:18:43] That's obviously the best developer experience. It's just not ironically something that we're optimizing for right now, because right now a lot of our users already have that knowledge. So we're better off spending time working on other stuff. So, and SDK's awesome.  [00:19:00] Jaden Baptista: Cool.  [00:19:00] So other than the JavaScript SDK, that we didn't really mention earlier what other things are in the roadmap for your temporal and by the way, anybody else who has any questions in case you just came in? So feel free to request to be a speaker and a astronomic question. But yeah, what is on your roadmap for them? [00:19:17] A little bit  [00:19:18] swyx: we I've been forbidden from publicly discussing them. So yeah, it's just the thing. Like we we look bad if we let our users down by saying we're working on something and then it doesn't pan out for whatever reason. So we just don't do that. I will say that our JavaScript and Python SDKs are highly, highly, highly in demand. [00:19:35] So, you know, Make your own conclusions there? I am also, maybe  [00:19:41] Jaden Baptista: I was thinking more like your priorities for like, not necessarily like a specific feature or anything, but like, what are you. Planning to focus on .  [00:19:52] swyx: Yeah, yeah, sure. Appreciate that question. Yeah, so, I mean, this is, this is a part of that, like getting us in front of the largest developer populations in the world is very important. [00:20:01] Our web UI is under a major rewrite right now. And that, that will hopefully. Help us be more intuitive, especially to people who are new to the system. always have this view that the docs or docs are. Dogs are a last resort. If, if at all possible use your UI to teach people as they go, instead of having to read a wall of docs. [00:20:21] And then, and then finally, so we're not generally available yet. We're just in private design partnerships with people. So we have, you know, some, some very, very well-known customers, some of them publicly listed and yeah, and, and we're basically learning how to scale. Service and scale a multitenant version of this, right? [00:20:39] It's the, you have to think about the origins of this company was an open source project running solely within Uber. So it never really had to think about You know, often organizations and like security and billing and all that. So we have to build out all of that. We have a list of about 30 things that we need to build before we are a production ready cloud. [00:20:58] But it is a very different startup from other forms of startup, because it seems like this company is like, it there's no risk of mismatch in product market fit. Like we already there in terms of. Right. You're already there. It, I, there, the reason I hesitate to say that we have product market fit is because we don't actually have a full product yet. [00:21:22] Right. Like, we need to build that product. And then we need to be able to repeatedly sell it either from enterprise sales or through a bottoms up, like just developers, trying stuff out. And yeah, there's a long way to get before we go there. But we have plenty of open source developers who are open-source users who are happy. [00:21:38] Build out. You know, we're looking forward to our production hosted cloud.  [00:21:43] Jaden Baptista: Awesome.   [00:21:44] Well, thank you for joining me, Sean today. I really appreciate you. You teach me a little bit about this because I've been looking to try and figure out what temporal is about so that I can potentially be using it in the future. [00:21:54] Once it fully launched. Yeah.  [00:22:00] swyx: You hear more stuff from me about it? There's a lot more, I haven't covered. We use event sourcing. So the way that we do reliability is that events are the source of truth. And we are able to resume from any downtime because of that, right there. That's just a fundamentally opinionated architecture, which I think a lot of people struggle with. [00:22:15] Like they eventually find it, but then they're like, oh, I have like five years of legacy work that I had to throw away now. Right. Yeah. I'll you hear more from me about, about this, for sure, but thanks for taking the time to, to check it out. Cool.  [00:22:28] Jaden Baptista: Well, maybe you can just, while we're finishing up here tell us a little bit about your social media and I have one more bonus question. [00:22:37] swyx: Can you, can you help me here? Like what, what specifically about social media follow him on Twitter at, oh yeah. Yeah. It's Swyx, it's my English and Chinese initials. SWIX but also, you know, just. Temporal, we have a mailing list or we're doing a meet up next week. A lot of people, you know, we'll have one of our users presenting airbikes and we're just trying to grow our community, you know, and, and help people solve their problems. [00:23:00] We're still cool. 30. Yeah. I mean, I'm pretty excited.  [00:23:04] Jaden Baptista: Awesome. Well, I have one little bonus question for you. I didn't let you know about beforehand, but I've been asking all of our interviewees, thus what is one tool that you use all the time that you just could not live with? Put me on the spot here. [00:23:15] swyx: Man, there's Honestly, simple notes. Very good. Oh yeah. Just the notes and notes app has to be reliable offline first and multi-platform I know there are other notes I have out there. Simple notes free. It's backed by WordPress. And it's very reliable and it just does notes, right? It doesn't do any, any rich media. [00:23:37] So you don't like, I know, I know notions the new hotness and all that, but like anything that slows it down, like notion we'll choose fancy new feature over speed and reliability. And I value speed and reliability. So I, yeah, I shout out for simple notes. I think everyone should have at least as reliable a note taking. [00:23:56] Jaden Baptista: Cool. Cool. I'll have to go. I'll have to go down and download that. Then I've been using notion forever. So anyway,  [00:24:03] swyx: it's like, like anytime you search it, it's like that spinner just like ticks you off. Like anytime you accidentally or something on the mobile app, it takes you off. Like just stuff like that. [00:24:13] Just don't do it around  [00:24:15] Jaden Baptista: and around and around. But thank you for introducing me to that. I'm going to go download that right now, actually. Well, thank you for joining me today, Sean. I really appreciate you. Teach me all about them for us. I'd love to have you back in the, in the coming weeks to tell us a little bit more about progress once those SDKs are released. [00:24:31] So let me know, but in the meantime maybe we should cap it off there. Thank you, Sean. All  [00:24:37] swyx: right. Thanks everyone.

This week on The Cloud Pod, apparently there was a machine learning conference because there is A LOT of machine learning news. For the listeners (and hosts of The Cloud Pod) who don't understand machine learning, buckle up because this will be a long episode for you.     A big thanks to this week's sponsors: Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning, and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights Amazon is acting like it's helping but really it's lying with numbers. Google is pretending the 1991 Ford Fiesta it's selling is a 2021 Mustang.  Azure got a little overexcited with the use of its naming bot. General News: Fake It Until You Make It Amazon data shows more diversity among senior leaders after the definition of “executive” loosened. Well, that's one way to do it.    Amazon's Andy Jassy is warming up for the CEO role. We hope competitors don't expect him to tread softly when he starts.   Pluralsight will acquire A Cloud Guru to address growing cloud skills gap. This is earth-shattering. Amazon Web Services: Busy As Usual Amazon Redshift Machine Learning is now generally available. There's a helpful table to explain the different machine learning products.  Amazon ECS Anywhere is now generally available. A bit disappointed that they haven't addressed the networking issue more. Introducing Amazon Kinesis Data Analytics Studio for analyzing streaming data. They're really into studios at the moment. Amazon SQS now supports a high throughput mode for FIFO Queues. This is nice.  Amazon Location Service is now generally available with new routing and satellite imagery capabilities. Just so you don't run your truck under a bridge that's too low.       Google Cloud Platform: Not A Robot In Disguise New Cloud TPU VMs make training machine learning models on TPUs easier. We told you this would be a long episode.  Google releases Log Field Analytics in Cloud Logging, a new way to search, filter and understand the structure of logs. This will make all those angry executives happy.   Google announces the generally availability of Datashare for Financial Services. Same product, different press release. Google introduces Analytics Hub, secure and scalable sharing for data and analytics.   Google announces Datastream, a serverless change data capture and replication service, is now in preview. Pretty nice feature!  Google is releasing logical replication and decoding for Cloud SQL for Postgres in Preview. A no-brainer.   Google releases Data Flow Prime, a new platform to simplify big data processing. No relation to Optimus Prime, just in case you were wondering. Google announces Dataplex in Preview, an intelligent data fabric for analytics at scale. Nice!    Azure: Crazy Naming Bot Azure has announced the general availability of its Azure ND A100 V4 Cloud GPU instances. Someone is excited about this.  Azure announces Synapse Link for Dataverse for application data analytics and predictive insights. The naming bot has gone crazy with this one.   Azure announces new infrastructure capabilities to simplify deployment and management. You can picture The Cloud Pod team flexing their muscles, can't you.    TCP Lightning Round Ryan wants to fight to the death but the others don't want to get blood on the carpet so he takes this week's point, leaving scores at Justin (9), Ryan (5), Jonathan (7).  Other headlines mentioned: Amazon QLDB supports IAM-based access policy for PartiQL queries and ledger tables  Announcing Amazon CloudWatch Resource Health Amazon SageMaker Autopilot adds automatic cross validation to improve model quality on smaller datasets by up to 35% AWS Launch Wizard adds support for SQL Server Always On Failover Cluster Instances deployed on Amazon FSx for Windows File Server Introducing AWS App Runner Integration in the AWS Toolkit for JetBrains IDEs AWS Glue DataBrew adds new nest and unnest transformations AWS Security Hub now supports bidirectional integration with Atlassian Jira Service Management   Amazon API Gateway now supports synchronous invocations of Express Workflows using REST APIs Amazon CloudWatch adds Control Plane API Usage Metrics across AWS Services  Cloud Bigtable lifts SLA to 99.999% and adds new security features for regulated industries Cloud Spanner trims entry cost by 90%, offers sharper observability and easier querying  Things Coming Up Announcing Google Cloud 2021 Summits [frequently updated] Harness Unscripted Conference — June 16–17 Google Cloud Next — Not announced yet (one site says Moscone is reserved June 28–30) Amazon re:Inforce — August 24-25 — Houston, TX Google Cloud Next 2021 — October 12–14, 2021 AWS re:Invent — November 29–December 3 — Las Vegas Oracle Open World (no details yet) 

Guest Ms. Angelica Lapastora of YoungCTO Rafi Quisumbing Angelica Lapastora is a Digital Operations Engineer - APAC at BMAT Licensing SLU, Barcelona, Spain. She specializes in creating backend services with Python, Django and AWS Lambda. She is also a Director of Marketing at PythonPH and a Python Lead at Women Who Code Manila. https://www.linkedin.com/in/angelica-lapastora-6585b2139/ https://python.ph/ MySQL, Amazon S3, Amazon Simple Notification Service (SNS), Django, PostgreSQL , Vue.js , Teaching, Communication , Team Leadership , Amazon EC2 , Amazon Dynamodb , RabbitMQ , Celery , Amazon SQS, API Development

Alexander Gallego is the founder and CEO of Vectorized. Vectorized offers a product called RedPanda, an Apache Kafka-compatible event streaming platform that’s significantly faster and easier to operate than Kafka. We talk about the increasing ubiquity of streaming platforms, what they’re used for, why Kafka is slow, and how to safely and effectively build a replacement.Previously, Alex was a Principal Software Engineer at Akamai systems and the creator of the Concord Framework, a distributed stream processing engine built in C++ on top of Apache Mesos.Apple Podcasts | Spotify | Google PodcastsHighlights7:00 - Who uses streaming platforms, and why? Why would someone use Kafka?12:30 - What would be the reason to use Kafka over Amazon SQS or Google PubSub?17:00 - What makes Kafka slow? The story behind RedPanda. We talk about memory efficiency in RedPanda which is better optimized for machines with more cores.34:00 - Other optimizations in RedPanda39:00 - WASM programming within the streaming engine, almost as if Kafka was an AWS Lambda processor.43:00 - How to convince potential customers to switch from Kafka to Redpanda?48:00 - What is the release process for Redpanda? How do they ensure that a new version isn’t broken?52:00 - What have we learnt about the state of Kafka and the use of streaming tools? Get on the email list at www.softwareatscale.dev

Der offizielle deutschsprachige Podcast rund um Amazon Web Services (AWS), für Neugierige, Cloud-Einsteiger und AWS-Experten, produziert von Dennis Traub, Developer Advocate bei AWS. Bei Fragen, Anregungen und Feedback wendet euch gerne direkt an Dennis auf Twitter (@dtraub) oder per Mail an traubd@amazon.com. In dieser Episode spricht Dennis über Amazon SQS und SNS, zwei Services zur Applikations-Integration für Microservices, verteilte Anwendungen und Serverless-Applikationen, sowie die Vorteile und Unterschiede dieser beiden scheinbar doch sehr ähnlichen Services. Links zum Thema: - Amazon Simple Queue Service (SQS) - https://aws.amazon.com/sqs/ - Amazon Simple Notofication Service (SNS) - https://aws.amazon.com/sns/ Für mehr Infos, Tipps und Tricks rund um AWS und die Cloud folgt Dennis auf: - Twitter - https://twitter.com/dtraub   - Twitch - https://www.twitch.tv/dennis_at_work  - YouTube - https://www.youtube.com/dennistraub

re:Invent arrived, and with it came a lot of announcements. Some meh, some good, some great. In this episode Arjen, Jean-Manuel, Guy, and special guest star Rob will do their best to make sense of it. Or maybe they just make it more confusing? Who knows? Our brains can't really handle the number of announcements. Which is probably also why it took far too long to edit this episode. What's New Finally in ANZ In the Works – AWS Region in Melbourne, Australia | AWS News Blog Amazon EMR now provides up to 30% lower cost and up to 15% improved performance for Spark workloads on Graviton2-based instances Amazon Aurora Serverless v1 with PostgreSQL compatibility now available in eight additional regions Amazon SageMaker Studio is now expanded to AWS regions worldwide Serverless Lambda New for AWS Lambda – 1ms Billing Granularity Adds Cost Savings | AWS News Blog New for AWS Lambda – Functions with Up to 10 GB of Memory and 6 vCPUs | AWS News Blog New for AWS Lambda – Container Image Support | AWS News Blog Using Amazon CloudWatch Lambda Insights to Improve Operational Visibility | AWS News Blog AWS Lambda now supports batch windows of up to 5 minutes for functions with Amazon SQS as an event source AWS Lambda now supports Advanced Vector Extensions 2 (AVX2) Announcing Code Signing, a trust and integrity control for AWS Lambda EventBridge AWS Systems Manager Change Calendar integrates with Amazon EventBridge to enable automated actions based on calendar state changes Amazon EventBridge adds Server-Side Encryption (SSE) and increases default quotas Step Functions Amazon API Gateway now supports integration with Step Functions StartSyncExecution for HTTP APIs AWS Step Functions now supports Synchronous Express Workflows Amplify AWS Amplify announces new Admin UI Containers ECR Amazon Elastic Container Registry Public: A New Public Container Registry | AWS News Blog Amazon ECR announces cross region replication of images Fargate New – Fully Serverless Batch Computing with AWS Batch Support for AWS Fargate | AWS News Blog ECS Introducing Amazon ECS Anywhere | Containers Amazon ECS Announces the Preview of ECS Deployment Circuit Breaker Amazon ECS Cluster Auto Scaling now supports specifying a custom instance warm-up time Amazon ECS Capacity Providers Now Support Update Functionality Amazon ECS adds support for P4d instance types Amazon ECS Cluster Auto Scaling now offers more responsive scaling AWS Copilot CLI is now Generally Available EKS Amazon EKS Anywhere – Amazon Web Services Amazon EKS Distro: The Kubernetes Distribution Used by Amazon EKS | AWS News Blog Simplify running Apache Spark jobs with Amazon EMR on Amazon EKS Amazon EKS simplifies installation and management for Kubernetes cluster add-ons Amazon EKS adds built-in logging support for AWS Fargate Amazon EKS adds support for EC2 Spot Instances in managed node groups Amazon EKS Console Now Includes Kubernetes Resources to Simplify Cluster Management EC2 & VPC EBS New – Amazon EBS gp3 Volume Lets You Provision Performance Apart From Capacity | AWS News Blog Now in Preview – Larger & Faster io2 Block Express EBS Volumes with Higher Throughput | AWS News Blog AWS announces tiered pricing for input/output operations per second (IOPS) charges for Amazon Elastic Block Store (EBS) io2 volume, reducing the cost of provisioning peak IOPS by 15% Amazon EBS reduces the minimum volume size of Throughput Optimized HDD and Cold HDD Volumes by 75% AWS Compute Optimizer now supports Amazon EBS volume recommendations Instance Types New – Use Amazon EC2 Mac Instances to Build & Test macOS, iOS, iPadOS, tvOS, and watchOS Apps | AWS News Blog New EC2 M5zn Instances – Fastest Intel Xeon Scalable CPU in the Cloud | AWS News Blog Coming Soon – Amazon EC2 G4ad Instances Featuring AMD GPUs for Graphics Workloads | AWS News Blog Coming Soon – EC2 C6gn Instances – 100 Gbps Networking with AWS Graviton2 Processors | AWS News Blog EC2 Update – D3 / D3en Dense Storage Instances | AWS News Blog New – Amazon EC2 R5b Instances Provide 3x Higher EBS Performance | AWS News Blog   Other EC2 Amazon Machine Images (AMIs) now support tag-on-create and tag-based access control Amazon EC2 Auto Scaling now supports attaching multiple network interfaces at launch AWS Announcing Windows Server version 20H2 AMIs for Amazon EC2 Simplify EC2 provisioning and viewing cloud resources in the ServiceNow CMDB with AWS Service Management Connector for ServiceNow Networking New – VPC Reachability Analyzer | AWS News Blog Introducing AWS Transit Gateway Connect to simplify SD-WAN branch connectivity AWS Global Accelerator launches custom routing Dev & Ops New services Preview: AWS Proton – Automated Management for Container and Serverless Deployments | AWS News Blog AWS announces Amazon DevOps Guru in Preview, an ML-powered cloud operations service to improve application availability for AWS workloads Preview: Amazon Lookout for Metrics, an Anomaly Detection Service for Monitoring the Health of Your Business | AWS News Blog Code New for Amazon CodeGuru – Python Support, Security Detectors, and Memory Profiling | AWS News Blog Amazon CodeGuru Reviewer announces Security Detectors to help improve code security Amazon CodeGuru Profiler adds Memory Profiling and Heap Summary Amazon CodeGuru Reviewer announces CodeQuality Detector to help manage technical debt and codebase maintainability AWS CodeArtifact now supports NuGet Tools AWS IDE Toolkit now available for AWS Cloud9 Porting Assistant for .NET adds support for .NET 5 Other Announcing Modules for AWS CloudFormation Amazon CloudWatch Synthetics now supports canary scripts in Python with Selenium framework AWS Systems Manager now supports Amazon Virtual Private Cloud (Amazon VPC) endpoint policies Security New services AWS Audit Manager Simplifies Audit Preparation | AWS News Blog SSO New – Attribute-Based Access Control with AWS Single Sign-On | AWS News Blog AWS Single Sign-On enables administrators to require users to set up MFA devices during sign-in AWS Single Sign-On adds Web Authentication (WebAuthn) support for user authentication with security keys and built-in biometric authenticators Other AWS CloudTrail provides more granular control of data event logging through advanced event selectors AWS Security Hub adds open source tool integrations with Kube-bench and Cloud Custodian AWS Transfer Family supports AWS WAF for identity provider integrations AWS Secrets Manager now supports 5000 requests per second for the GetSecretValue API operation Data Storage & Processing Aurora Introducing the next version of Amazon Aurora Serverless in preview Introducing Amazon Aurora R6g instance types, powered by AWS Graviton2 processors, in preview (includes Sydney) Babelfish for Amazon Aurora PostgreSQL is Available for Preview Amazon Aurora PostgreSQL Integrates with AWS Lambda RDS Amazon RDS for Oracle supports managed disaster recovery (DR) with Amazon RDS Cross-Region Automated Backups PostgreSQL 13 now available in Amazon RDS Database preview environment Lakes Amazon HealthLake Stores, Transforms, and Analyzes Health Data in the Cloud | AWS News Blog Announcing preview of AWS Lake Formation features: Transactions, Row-level Security, and Acceleration S3 New – Amazon S3 Replication Adds Support for Multiple Destination Buckets | AWS News Blog Amazon S3 Update – Strong Read-After-Write Consistency | AWS News Blog Amazon S3 Replication adds support for multiple destinations in the same, or different AWS Regions Amazon S3 now delivers strong read-after-write consistency automatically for all applications Amazon S3 Bucket Keys reduce the costs of Server-Side Encryption with AWS Key Management Service (SSE-KMS) Amazon S3 Replication adds support for two-way replication EMR Amazon EMR Studio makes it easier for data scientists to build and deploy code Redshift AWS announces AQUA for Amazon Redshift (preview) Amazon Redshift introduces data sharing (preview) Amazon Redshift launches RA3.xlplus nodes with managed storage Amazon Redshift announces Automatic Table Optimization Amazon Redshift now includes Amazon RDS for MySQL and Amazon Aurora MySQL databases as new data sources for federated querying (Preview) Amazon Redshift launches the ability to easily move clusters between AWS Availability Zones (AZs) DynamoDB You now can use Amazon DynamoDB with AWS Glue Elastic Views to combine and replicate data across multiple data stores by using SQL – available in limited preview You now can use a SQL-compatible query language to query, insert, update, and delete table data in Amazon DynamoDB Glue Announcing Amazon Elasticsearch Service support for AWS Glue Elastic Views Announcing AWS Glue Elastic Views Preview AWS Glue now supports workload partitioning to further improve the reliability of Spark applications Other Amazon FSx for Lustre now enables you to grow storage on your file systems with the click of a button Introducing Amazon Managed Workflows for Apache Airflow (MWAA) AI & ML Sagemaker :allthethings: Amazon SageMaker Simplifies Training Deep Learning Models With Billions of Parameters | AWS News Blog Amazon SageMaker JumpStart Simplifies Access to Pre-built Models and Machine Learning Solutions | AWS News Blog New – Store, Discover, and Share Machine Learning Features with Amazon SageMaker Feature Store | AWS News Blog New – Profile Your Machine Learning Training Jobs With Amazon SageMaker Debugger | AWS News Blog New – Amazon SageMaker Pipelines Brings DevOps Capabilities to your Machine Learning Projects | AWS News Blog Amazon SageMaker Edge Manager Simplifies Operating Machine Learning Models on Edge Devices | AWS News Blog New – Managed Data Parallelism in Amazon SageMaker Simplifies Training on Large Datasets | AWS News Blog Introducing Amazon SageMaker Data Wrangler, a Visual Interface to Prepare Data for Machine Learning | AWS News Blog Amazon SageMaker JumpStart Simplifies Access to Pre-built Models and Machine Learning Solutions | AWS News Blog New – Amazon SageMaker Clarify Detects Bias and Increases the Transparency of Machine Learning Models | AWS News Blog Amazon SageMaker Model Monitor now supports new capabilities to maintain model quality in production Introducing two new libraries for managed distributed training on Amazon SageMaker Edge New – Amazon Lookout for Equipment Analyzes Sensor Data to Help Detect Equipment Failure | AWS News Blog Amazon Lookout for Vision – New ML Service Simplifies Defect Detection for Manufacturing | AWS News Blog AWS Panorama Appliance: Bringing Computer Vision Applications to the Edge | AWS News Blog Introducing Amazon Monitron, an end-to-end system to detect abnormal equipment behavior AI Services Amazon Kendra adds Google Drive connector Amazon Kendra launches incremental learning Amazon Kendra launches connector library Announcing Amazon Forecast Weather Index – automatically include local weather to increase your forecasting model accuracy Added ML Amazon announces Amazon Neptune ML: easy, fast, and accurate predictions for graphs AWS announces Amazon Redshift ML (preview) Other Cool Stuff Regions/Zones Announcing new AWS Wavelength Zone in Las Vegas Announcing Preview of AWS Local Zones in Boston, Houston, and Miami Braket PennyLane on Braket + Progress Toward Fault-Tolerant Quantum Computing + Tensor Network Simulator | AWS News Blog Amazon Braket tensor network simulator supports 50-qubit quantum circuits Amazon Braket now supports manual qubit allocation Connect Contact Lens for Amazon Connect launches real-time contact center analytics to detect customer issues on live calls Amazon Connect Wisdom provides contact center agents the information they need to quickly solve customer issues Amazon Connect Customer Profiles for a unified view of your customers to provide more personalized service Amazon Connect Voice ID provides real-time caller authentication for more secure calls Amazon Connect Tasks makes it easy to prioritize, assign, track, and automate contact center agent tasks Amazon Connect Chat now supports Apple Business Chat (Preview) Quicksight Introducing Amazon QuickSight Q: ask questions about your data and get answers in seconds Amazon QuickSight launches new session capacity pricing options, embedding without user management and a developer portal for embedded analytics Other Announcing Unified Search in the AWS Management Console Amazon WorkSpaces Streaming Protocol now Generally Available New – SaaS Lens in AWS Well-Architected Tool | AWS News Blog The Amazon Chime SDK now supports messaging AWS Batch now has integrated Amazon Linux 2 support Nanos Amazon WorkDocs now supports Dark Mode on Android Sponsors Gold Sponsor Innablr Silver Sponsors AC3 CMD Solutions DoIT International

最新情報を "ながら" でキャッチアップ！ ラジオ感覚放送 「毎日AWS」 おはようございます、サーバーワークスの加藤です。 今日は 12/17 に出たアップデートをピックアップしてご紹介。 感想は Twitter にて「#サバワ」をつけて投稿してください！ ■ re:Invent re:CAP 開催！ 「re:Invent」を振り返ってわかる、DXの次の一手はこれだ！ https://20201221webinar.serverworks.co.jp/ re:Invent に関するサーバーワークスの情報発信はこちら ■ UPDATE PICKUP Amazon SQS が FIFO キューの高スループットモードをプレビューでサポート AWS Key Management Service が属性ベースのアクセス制御に対応 EC2 Image Builder がコンテナイメージをサポート Data Lifecycle Manager がアカウント間での EBS スナップショットのコピーを自動化 AWS Well-Architected Guidance Engine が利用可能に Amazon SageMaker が ml.P4d インスタンスをサポート Amazon Route53 が DNSSEC に対応 ■ サーバーワークスSNS Twitter / Facebook ■ サーバーワークスブログ サーバーワークスエンジニアブログ

On this week's episode, Chris introduces a new segment called "Good Idea, Terrible Idea?" as he considers introducing a read-only mode to avoid interrupting users during scheduled downtime. Steph has started a new project and explores the idea of merging separate, but similar, applications into one codebase. They also dive into micro-service environments to discuss the difficulties of integration testing and potential strategies. This episode is brought to you by: ScoutAPM (https://scoutapm.com/bikeshed) - Give Scout a try for free today and Scout will donate $5 to the open source project of your choice when you deploy Indeed (https://Indeed.com/bikeshed) - Click through and get started with a free seventy five dollar credit for your first job post Earth, Wind & Fire - September (https://youtu.be/Gs069dndIYk) ActiveRecord Persistence (https://api.rubyonrails.org/files/activerecord/lib/active_record/persistence_rb.html) Amazon SQS (https://aws.amazon.com/sqs/) Swagger (https://swagger.io/docs/) GraphQL (https://graphql.org/) VCR (https://github.com/vcr/vcr) The Bike Shed VCR episode - 189: It's Gonna Work, Definitely, No Problems Whatsoever (https://www.bikeshed.fm/189) Capybara Discoball (https://github.com/thoughtbot/capybara_discoball) React Podcast Episode - 110: Sam Selikoff on Finding a Full Stack React (https://spec.fm/podcasts/reactpodcast/X1swadgC) Become a Sponsor (https://thoughtbot.com/sponsorship) of The Bike Shed!

Bob Quillin and Karthik Gaekwad are on the Oracle developer relations team. Karthik has been on Ruby Rogues previously, and he explains how he went from the Kubernetes team to developer relations. They begin the show by explaining what Kafka is, the leading open-source event streaming platform that Oracle is compatible with. It allows cloud developers to build, publish, and subscribe models for streams of records in addition to many other functions. Systems that used to take a long time to make have become very small and simple with Kafka. Kafka stands out from other message queueing systems because of its robust nature and scalability.  Bob goes into more depth about the evolution of Kafka and the panel discusses some different use cases, concluding that Kafka works best for projects with a large amount of data coming in and for making real-time decisions. Bob and Karthik talk about other things Kafka can do beyond the message queue, such as building streams from specific patterns. They talk about when you should consider moving over to Kafka. Karthik talks about how to get started with Kafka. One of the best ways to do this is to set up a service with Oracle and to just play around with it, which won’t cost you much if you aren’t pushing a lot of data through it. Bob and Karthik talk about some of the features offered by Oracle and Kafka. While the offerings are somewhat vanilla, you get the advantage of it being an open-source driven service on top of a cloud that’s highly secure, available, and built to last. The panel discusses security within Kafka. They talk briefly about the framework Karafka and tools and resources available through Oracle for Kafka. The show concludes with the panel talking about compatibility between Kafka and Docker. Panelists John Epperson Charles Max Wood Guests Bob Quillin Karthik Gaekwad Sponsors Cloud 66 - Pain-Free Rails Deployments Try Cloud 66 Rails for FREE & get $100 of free credits with promo code RubyRogues-19 RedisGreen ____________________________ > "The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today! ____________________________________________________________ Links RR 397: Oracle with Karthik Gaekwad RR 402: Kafka and Karafka with Maciej Menfeld Kafka Redis Amazon SQS Sidekick Resque RabbitMQ Zookeeper Karafka Picks Charles Max Wood: Discord HubSpot John Epperson: Aberlour A'Bunadh The Witcher Bob Quillin: Balcones Distillery Milam and Greene Whiskeys The Overstory Karthik Gaekwad: Follow Karthik on LinkedIn, Github, and Twitter

Bob Quillin and Karthik Gaekwad are on the Oracle developer relations team. Karthik has been on Ruby Rogues previously, and he explains how he went from the Kubernetes team to developer relations. They begin the show by explaining what Kafka is, the leading open-source event streaming platform that Oracle is compatible with. It allows cloud developers to build, publish, and subscribe models for streams of records in addition to many other functions. Systems that used to take a long time to make have become very small and simple with Kafka. Kafka stands out from other message queueing systems because of its robust nature and scalability.  Bob goes into more depth about the evolution of Kafka and the panel discusses some different use cases, concluding that Kafka works best for projects with a large amount of data coming in and for making real-time decisions. Bob and Karthik talk about other things Kafka can do beyond the message queue, such as building streams from specific patterns. They talk about when you should consider moving over to Kafka. Karthik talks about how to get started with Kafka. One of the best ways to do this is to set up a service with Oracle and to just play around with it, which won’t cost you much if you aren’t pushing a lot of data through it. Bob and Karthik talk about some of the features offered by Oracle and Kafka. While the offerings are somewhat vanilla, you get the advantage of it being an open-source driven service on top of a cloud that’s highly secure, available, and built to last. The panel discusses security within Kafka. They talk briefly about the framework Karafka and tools and resources available through Oracle for Kafka. The show concludes with the panel talking about compatibility between Kafka and Docker. Panelists John Epperson Charles Max Wood Guests Bob Quillin Karthik Gaekwad Sponsors Cloud 66 - Pain-Free Rails Deployments Try Cloud 66 Rails for FREE & get $100 of free credits with promo code RubyRogues-19 RedisGreen ____________________________ > "The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today! ____________________________________________________________ Links RR 397: Oracle with Karthik Gaekwad RR 402: Kafka and Karafka with Maciej Menfeld Kafka Redis Amazon SQS Sidekick Resque RabbitMQ Zookeeper Karafka Picks Charles Max Wood: Discord HubSpot John Epperson: Aberlour A'Bunadh The Witcher Bob Quillin: Balcones Distillery Milam and Greene Whiskeys The Overstory Karthik Gaekwad: Follow Karthik on LinkedIn, Github, and Twitter

Bob Quillin and Karthik Gaekwad are on the Oracle developer relations team. Karthik has been on Ruby Rogues previously, and he explains how he went from the Kubernetes team to developer relations. They begin the show by explaining what Kafka is, the leading open-source event streaming platform that Oracle is compatible with. It allows cloud developers to build, publish, and subscribe models for streams of records in addition to many other functions. Systems that used to take a long time to make have become very small and simple with Kafka. Kafka stands out from other message queueing systems because of its robust nature and scalability.  Bob goes into more depth about the evolution of Kafka and the panel discusses some different use cases, concluding that Kafka works best for projects with a large amount of data coming in and for making real-time decisions. Bob and Karthik talk about other things Kafka can do beyond the message queue, such as building streams from specific patterns. They talk about when you should consider moving over to Kafka. Karthik talks about how to get started with Kafka. One of the best ways to do this is to set up a service with Oracle and to just play around with it, which won’t cost you much if you aren’t pushing a lot of data through it. Bob and Karthik talk about some of the features offered by Oracle and Kafka. While the offerings are somewhat vanilla, you get the advantage of it being an open-source driven service on top of a cloud that’s highly secure, available, and built to last. The panel discusses security within Kafka. They talk briefly about the framework Karafka and tools and resources available through Oracle for Kafka. The show concludes with the panel talking about compatibility between Kafka and Docker. Panelists John Epperson Charles Max Wood Guests Bob Quillin Karthik Gaekwad Sponsors Cloud 66 - Pain-Free Rails Deployments Try Cloud 66 Rails for FREE & get $100 of free credits with promo code RubyRogues-19 RedisGreen ____________________________ > "The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today! ____________________________________________________________ Links RR 397: Oracle with Karthik Gaekwad RR 402: Kafka and Karafka with Maciej Menfeld Kafka Redis Amazon SQS Sidekick Resque RabbitMQ Zookeeper Karafka Picks Charles Max Wood: Discord HubSpot John Epperson: Aberlour A'Bunadh The Witcher Bob Quillin: Balcones Distillery Milam and Greene Whiskeys The Overstory Karthik Gaekwad: Follow Karthik on LinkedIn, Github, and Twitter

Learn how easy it is to get started with a new application composed of several microservices on AWS Lambda, Amazon EKS, Amazon SQS, and Amazon DynamoDB. In this session, you first learn through live demonstration how to do basic administration, such as deploying an application and inspecting the relevant Kubernetes resources. Next, you learn some troubleshooting practices (using log files, accessing containers, etc.) and dive into advanced troubleshooting. Innovate faster with software as you learn to leverage some of the most modern services on AWS. This presentation is brought to you by New Relic, an APN Partner.

Intercom is "all in" on AWS-a strategy that's aligned with the engineering principles used to build the company. This talk covers how the company uses those principles to make engineering decisions. Learn about the evolution of Intercom's architecture, from handful of Amazon EC2 hosts to thousands of instances. Also learn how Intercom uses Amazon DynamoDB, AWS Lambda, Amazon SQS, Amazon Aurora, Amazon Redshift, Amazon Kinesis, and more. Finally, learn why Intercom decided against leveraging microservices, and why this has increased its ability to move fast and ship great products.

National Australia Bank (NAB) is migrating hundreds of sensitive, regulated financial workloads to the cloud. This session focuses on the automated approach that NAB has taken to evolve and scale its ability to meet its strict risk and regulatory requirements. Specifically, NAB describes its cloud governance framework and the automated enforcement platform (Deputy) that supports it and that brings governance and technology together to ensure that compliant workloads are deployed in the cloud. The talk dives deep into the architecture of Deputy, which uses AWS Lambda, Amazon SNS, Amazon SQS, Amazon DynamoDB, and Amazon RDS to create a cost-efficient, scalable solution.

FINRA ingests up to 100 billion trading data records daily that need to be obfuscated securely when moving them. This data replication and obfuscation supports production data analysis, setup environments for user acceptance testing (UAT), and troubleshooting production issues. With the automated, one-click solution, FINRA teams can leverage the pipelines and within hours (versus days) get data for analysis, development, and testing. The solution is serverless, leveraging AWS Step Functions, AWS Lambda, Amazon ECS/AWS Fargate, Amazon SNS, and Amazon SQS. FINRA plans to open source this by the end of this year so attendees will be able to adopt the solution.

Verizon Media had to create a better, stronger, and faster push notification system to serve the requirements of iconic Verizon brands, fulfill push notification completion time of 27 million devices in under three minutes, and consistently show the push 'toast' on all users' lock screens. Verizon decided to use Amazon DynamoDB and other AWS services such as Amazon ElastiCache and Amazon SQS in conjunction with its own Vespa search engine to power all the use cases of its brands. It also uses Kubernetes to orchestrate microservices across many Amazon EC2 instances. Join this session to learn how Verizon has been able to accomplish all of this.

Launched in fall 2017, Bird is a micromobility company that enables access to shared e-scooters and lightweight electric vehicles in 100+ locations worldwide. Join us to hear how building a modern stack on top of Amazon EKS has enabled Bird to quickly ramp up its development in order to provide business value in a stable and secure manner. Further, learn how Bird's backend utilizes native AWS services like Amazon S3 and Amazon SQS, open-source streaming systems like Kafka and Flink, and a modern microservices architecture to turn terabytes of geospatial data into the mobility revolution of the future.

In this messaging themed episode of AWS TechChat, Pete is back, and more so in person. They started the show reminiscing about messaging history, going back, looking at where we came from and how we arrived at the position we are today. More importantly, why do we use messaging and the benefits you can derive in decoupling your architecture. They then pivot to event streams, which cover both Amazon Kinesis and Amazon Managed Streaming for Apache Kafka, (Amazon MSK). They are both designed to process or analyze streaming data for specialized needs. Next, they moved to a more traditional message bus - Amazon Simple Queue Service (SQS) and Amazon MQ (Managed message broker service for ActiveMQ), both a durable pull-based messaging platform. Amazon SQS being lightweight and tightly integrated to the AWS Cloud platform and Amazon MQ supporting a variety of protocols making it a great choice for existing applications that use industry-standard protocols. Finally, they talked about push-based messaging with Amazon Simple Notification Service (SNS) and the Message Broker for AWS IoT. Both publish/subscribe (pub/sub) platform that enables you to build fan out architectures with hundreds of thousands to millions of subscribers. You now have more than a hammer to build your applications, Maslow would be proud. Speakers: Shane Baldacchino - Solutions Architect, ANZ, AWS Peter Stanski - Head of Solution Architecture, AWS Resources: Amazon CloudFront announces new Edge location in Shenzhen, China https://aws.amazon.com/about-aws/whats-new/2019/09/amazon-cloudfront-shenzhen-launch/ What is Pub/Sub Messaging? https://aws.amazon.com/pub-sub-messaging/ Amazon Kinesis Data Streams https://aws.amazon.com/kinesis/data-streams/ Amazon Managed Streaming for Apache Kafka (Amazon MSK) https://aws.amazon.com/msk/ Apache ZooKeeper https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-zookeeper.html Amazon Simple Queue Service https://aws.amazon.com/sqs/ Amazon Simple Queue Service Released https://aws.amazon.com/blogs/aws/amazon_simple_q/ Amazon MQ https://aws.amazon.com/amazon-mq/ Amazon Simple Notification Service https://aws.amazon.com/sns/ MQTT - AWS IoT https://docs.aws.amazon.com/iot/latest/developerguide/mqtt.html Message Broker for AWS IoT https://docs.aws.amazon.com/iot/latest/developerguide/iot-message-broker.html AWS Events: AWS re:Invent https://reinvent.awsevents.com/ AWSome Day Online Series https://aws.amazon.com/events/awsome-day/awsome-day-online/ AWS Modern Application Development Online Event https://aws.amazon.com/events/application/modern-app-development/ AWS Innovate on-demand https://aws.amazon.com/events/aws-innovate/

It is a MASSIVE episode of updates that Simon and Nikki do their best to cover! There is also an EXTRA SPECIAL bonus just for AWS Podcast listeners! Special Discount for Intersect Tickets: https://int.aws/podcast use discount code 'podcast' - note that tickets are limited! Chapters: 02:19 Infrastructure 03:07 Storage 05:34 Compute 13:47 Network 14:54 Databases 17:45 Migration 18:36 Developer Tools 21:39 Analytics 29:25 IoT 33:24 End User Computing 34:08 Machine Learning 40:21 AR and VR 41:11 Application Integration 43:57 Management and Governance 48:04 Customer Engagement 49:13 Media 50:17 Mobile 50:36 Security 51:26 Gaming 51:39 Robotics 52:13 Training Shownotes: Special Discount for Intersect Tickets: https://int.aws/podcast use discount code 'podcast' - note that tickets are limited! Topic || Infrastructure Announcing the new AWS Middle East (Bahrain) Region | https://aws.amazon.com/about-aws/whats-new/2019/07/announcing-the-new-aws-middle-east--bahrain--region-/ Topic || Storage EBS default volume type updated to GP2 | https://aws.amazon.com/about-aws/whats-new/2019/07/ebs-default-volume-type-updated-to-gp2/ AWS Backup will Automatically Copy Tags from Resource to Recovery Point | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-backup-will-automatically-copy-tags-from-resource-to-recovery-point/ Configuration update for Amazon EFS encryption of data in transit | https://aws.amazon.com/about-aws/whats-new/2019/07/configuration-update-for-amazon-efs-encryption-data-in-transit/ AWS Snowball and Snowball Edge available in Seoul – Amazon Web Services | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-snowball-and-aws-snowball-edge-available-in-asia-pacific-seoul-region/ Amazon S3 adds support for percentiles on Amazon CloudWatch Metrics | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-s3-adds-support-for-percentiles-on-amazon-cloudwatch-metrics/ Amazon FSx Now Supports Windows Shadow Copies for Restoring Files to Previous Versions | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-fsx-now-supports-windows-shadow-copies-for-restoring-files-to-previous-versions/ Amazon CloudFront Announces Support for Resource-Level and Tag-Based Permissions | https://aws.amazon.com/about-aws/whats-new/2019/08/cloudfront-resource-level-tag-based-permission/ Topic || Compute Amazon EC2 AMD Instances are Now Available in additional regions | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ec2-amd-instances-available-in-additional-regions/ Amazon EC2 P3 Instances Featuring NVIDIA Volta V100 GPUs now Support NVIDIA Quadro Virtual Workstation | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ec2-p3-nstances-featuring-nvidia-volta-v100-gpus-now-support-nvidia-quadro-virtual-workstation/ Introducing Amazon EC2 I3en and C5n Bare Metal Instances | https://aws.amazon.com/about-aws/whats-new/2019/08/introducing-amazon-ec2-i3en-and-c5n-bare-metal-instances/ Amazon EC2 C5 New Instance Sizes are Now Available in Additional Regions | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-ec2-c5-new-instance-sizes-are-now-available-in-additional-regions/ Amazon EC2 Spot Now Available for Red Hat Enterprise Linux (RHEL) | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ec2-spot-now-available-red-hat-enterprise-linux-rhel/ Amazon EC2 Now Supports Tagging Launch Templates on Creation | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ec2-now-supports-tagging-launch-templates-on-creation/ Amazon EC2 On-Demand Capacity Reservations Can Now Be Shared Across Multiple AWS Accounts | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ec2-on-demand-capacity-reservations-shared-across-multiple-aws-accounts/ Amazon EC2 Fleet Now Lets You Modify On-Demand Target Capacity | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-ec2-fleet-modify-on-demand-target-capacity/ Amazon EC2 Fleet Now Lets You Set A Maximum Price For A Fleet Of Instances | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-ec2-fleet-now-lets-you-submit-maximum-price-for-fleet-of-instances/ Amazon EC2 Hibernation Now Available on Ubuntu 18.04 LTS | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ec2-hibernation-now-available-ubuntu-1804-lts/ Amazon ECS services now support multiple load balancer target groups | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ecs-services-now-support-multiple-load-balancer-target-groups/ Amazon ECS Console now enables simplified AWS App Mesh integration | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ecs-console-enables-simplified-aws-app-mesh-integration/ Amazon ECR now supports increased repository and image limits | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ecr-now-supports-increased-repository-and-image-limits/ Amazon ECR Now Supports Immutable Image Tags | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ecr-now-supports-immutable-image-tags/ Amazon Linux 2 Extras now provides AWS-optimized versions of new Linux Kernels | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-linux-2-extras-provides-aws-optimized-versions-of-new-linux-kernels/ Lambda@Edge Adds Support for Python 3.7 | https://aws.amazon.com/about-aws/whats-new/2019/08/lambdaedge-adds-support-for-python-37/ AWS Batch Now Supports the Elastic Fabric Adapter | https://aws.amazon.com/about-aws/whats-new/2019/08/aws-batch-now-supports-elastic-fabric-adapter/ Topic || Network Elastic Fabric Adapter is officially integrated into Libfabric Library | https://aws.amazon.com/about-aws/whats-new/2019/07/elastic-fabric-adapter-officially-integrated-into-libfabric-library/ Now Launch AWS Glue, Amazon EMR, and AWS Aurora Serverless Clusters in Shared VPCs | https://aws.amazon.com/about-aws/whats-new/2019/08/now-launch-aws-glue-amazon-emr-and-aws-aurora-serverless-clusters-in-shared-vpcs/ AWS DataSync now supports Amazon VPC endpoints | https://aws.amazon.com/about-aws/whats-new/2019/08/aws-datasync-now-supports-amazon-vpc-endpoints/ AWS Direct Connect Now Supports Resource Based Authorization, Tag Based Authorization, and Tag on Resource Creation | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-direct-connect-now-supports-resource-based-authorization-tag-based-authorization-tag-on-resource-creation/ Topic || Databases Amazon Aurora Multi-Master is Now Generally Available | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-aurora-multimaster-now-generally-available/ Amazon DocumentDB (with MongoDB compatibility) Adds Aggregation Pipeline and Diagnostics Capabilities | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-documentdb-with-mongodb-compatibility-adds-aggregation-pipeline-and-diagnostics-capabilities/ Amazon DynamoDB now helps you monitor as you approach your account limits | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-dynamodb-now-helps-you-monitor-as-you-approach-your-account-limits/ Amazon RDS for Oracle now supports new instance sizes | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-rds-for-oracle-now-supports-new-instance-sizes/ Amazon RDS for Oracle Supports Oracle Management Agent (OMA) version 13.3 for Oracle Enterprise Manager Cloud Control 13c | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-rds-for-oracle-supports-oracle-management-agent-oma-version133-for-oracle-enterprise-manager-cloud-control13c/ Amazon RDS for Oracle now supports July 2019 Oracle Patch Set Updates (PSU) and Release Updates (RU) | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-rds-for-oracle-supports-july-2019-oracle-patch-set-and-release-updates/ Amazon RDS SQL Server now supports changing the server-level collation | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-rds-sql-server-supports-changing-server-level-collation/ PostgreSQL 12 Beta 2 Now Available in Amazon RDS Database Preview Environment | https://aws.amazon.com/about-aws/whats-new/2019/08/postgresql-beta-2-now-available-in-amazon-rds-database-preview-environment/ Amazon Aurora with PostgreSQL Compatibility Supports Publishing PostgreSQL Log Files to Amazon CloudWatch Logs | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-aurora-with-postgresql-compatibility-support-logs-to-cloudwatch/ Amazon Redshift Launches Concurrency Scaling in Five additional AWS Regions, and Enhances Console Performance Graphs in all supported AWS Regions | https://aws.amazon.com/about-aws/ whats-new/2019/08/amazon-redshift-launches-concurrency-scaling-five-additional-regions-enhances-console-performance-graphs/ Amazon Redshift now supports column level access control with AWS Lake Formation | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-redshift-spectrum-now-supports-column-level-access-control-with-aws-lake-formation/ Topic || Migration AWS Migration Hub Now Supports Import of On-Premises Server and Application Data From RISC Networks to Plan and Track Migration Progress | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-migration-hub-supports-import-of-on-premises-server-application-data-from-risc-networks-to-track-migration-progress/ Topic || Developer Tools AWS CodePipeline Achieves HIPAA Eligibility | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-codepipeline-achieves-hipaa-eligibility/ AWS CodePipeline Adds Pipeline Status to Pipeline Listing | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-codepipeline-adds-pipeline-status-to-pipeline-listing/ AWS Amplify Console adds support for automatically deploying branches that match a specific pattern | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-amplify-console-support-git-based-branch-pattern-detection/ Amplify Framework Adds Predictions Category | https://aws.amazon.com/about-aws/whats-new/2019/07/amplify-framework-adds-predictions-category/ Amplify Framework adds local mocking and testing for GraphQL APIs, Storage, Functions, and Hosting | https://aws.amazon.com/about-aws/whats-new/2019/08/amplify-framework-adds-local-mocking-and-testing-for-graphql-apis-storage-functions-hostings/ Topic || Analytics AWS Lake Formation is now generally available | https://aws.amazon.com/about-aws/whats-new/2019/08/aws-lake-formation-is-now-generally-available/ Announcing PartiQL: One query language for all your data | https://aws.amazon.com/blogs/opensource/announcing-partiql-one-query-language-for-all-your-data/ AWS Glue now supports the ability to run ETL jobs on Apache Spark 2.4.3 (with Python 3) | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-glue-now-supports-ability-to-run-etl-jobs-apache-spark-243-with-python-3/ AWS Glue now supports additional configuration options for memory-intensive jobs submitted through development endpoints | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-glue-now-supports-additional-configuration-options-for-memory-intensive-jobs-submitted-through-deployment-endpoints/ AWS Glue now provides the ability to bookmark Parquet and ORC files using Glue ETL jobs | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-glue-now-provides-ability-to-bookmark-parquet-and-orc-files-using-glue-etl-jobs/ AWS Glue now provides FindMatches ML transform to deduplicate and find matching records in your dataset | https://aws.amazon.com/about-aws/whats-new/2019/08/aws-glue-provides-findmatches-ml-transform-to-deduplicate/ Amazon QuickSight adds support for custom colors, embedding for all user types and new regions! | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-quicksight-adds-support-for-custom-colors-embedding-for-all-user-types-and-new-regions/ Achieve 3x better Spark performance with EMR 5.25.0 | https://aws.amazon.com/about-aws/whats-new/2019/08/achieve-3x-better-spark-performance-with-emr-5250/ Amazon EMR now supports native EBS encryption | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon_emr_now_supports_native_ebs_encryption/ Amazon Athena adds Support for AWS Lake Formation Enabling Fine-Grained Access Control on Databases, Tables, and Columns | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-athena-adds-support-for-aws-lake-formation-enabling-fine-grained-access-control-on-databases-tables-columns/ Amazon EMR Integration With AWS Lake Formation Is Now In Beta, Supporting Database, Table, and Column-level access controls for Apache Spark | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-emr-integration-with-aws-lake-formation-now-in-beta-supporting-database-table-column-level-access-controls/ Topic || IoT AWS IoT Device Defender Expands Globally | https://aws.amazon.com/about-aws/whats-new/2019/08/aws-iot-device-defender-expands-globally/ AWS IoT Device Defender Supports Mitigation Actions for Audit Results | https://aws.amazon.com/about-aws/whats-new/2019/08/aws-iot-device-defender-supports-mitigation-actions-for-audit-results/ AWS IoT Device Tester v1.3.0 is Now Available for Amazon FreeRTOS 201906.00 Major | https://aws.amazon.com/about-aws/whats-new/2019/07/aws_iot_device_tester_v130_for_amazon_freertos_201906_00_major/ AWS IoT Events actions now support AWS Lambda, SQS, Kinesis Firehose, and IoT Events as targets | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-iot-events-supports-invoking-actions-to-lambda-sqs-kinesis-firehose-iot-events/ AWS IoT Events now supports AWS CloudFormation | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-iot-events-now-supports-aws-cloudformation/ Topic || End User Computing AWS Client VPN now adds support for Split-tunnel | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-client-vpn-now-adds-support-for-split-tunnel/ Introducing AWS Chatbot (beta): ChatOps for AWS in Amazon Chime and Slack Chat Rooms | https://aws.amazon.com/about-aws/whats-new/2019/07/introducing-aws-chatbot-chatops-for-aws/ Amazon AppStream 2.0 Adds CLI Operations for Programmatic Image Creation | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-appstream-2-adds-cli-operations-for-programmatic-image-creation/ NICE DCV Releases Version 2019.0 with Multi-Monitor Support on Web Client | https://aws.amazon.com/about-aws/whats-new/2019/08/nice-dcv-releases-version-2019-0-with-multi-monitor-support-on-web-client/ New End User Computing Competency Solutions | https://aws.amazon.com/about-aws/whats-new/2019/08/end-user-computing-competency-solutions/ Amazon WorkDocs Migration Service | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon_workdocs_migration_service/ Topic || Machine Learning SageMaker Batch Transform now enables associating prediction results with input attributes | https://aws.amazon.com/about-aws/whats-new/2019/07/sagemaker-batch-transform-enable-associating-prediction-results-with-input-attributes/ Amazon SageMaker Ground Truth Adds Data Labeling Workflow for Named Entity Recognition | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-sagemaker-ground-truth-adds-data-labeling-workflow-for-named-entity-recognition/ Amazon SageMaker notebooks now available with pre-installed R kernel | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-sagemaker-notebooks-available-with-pre-installed-r-kernel/ New Model Tracking Capabilities for Amazon SageMaker Are Now Generally Available | https://aws.amazon.com/about-aws/whats-new/2019/08/new-model-tracking-capabilities-for-amazon-sagemaker-now-generally-available/ Amazon Comprehend Custom Entities now supports multiple entity types | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-comprehend-custom-entities-supports-multiple-entity-types/ Introducing Predictive Maintenance Using Machine Learning | https://aws.amazon.com/about-aws/whats-new/2019/07/introducing-predictive-maintenance-using-machine-learning/ Amazon Transcribe Streaming Now Supports WebSocket | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-transcribe-streaming-now-supports-websocket/ Amazon Polly Launches Neural Text-to-Speech and Newscaster Voices | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-polly-launches-neural-text-to-speech-and-newscaster-voices/ Manage a Lex session using APIs on the client | https://aws.amazon.com/about-aws/whats-new/2019/08/manage-a-lex-session-using-apis-on-the-client/ Amazon Rekognition now detects violence, weapons, and self-injury in images and videos; improves accuracy for nudity detection | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-rekognition-now-detects-violence-weapons-and-self-injury-in-images-and-videos-improves-accuracy-for-nudity-detection/ Topic || AR and VR Amazon Sumerian Now Supports Physically-Based Rendering (PBR) | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-sumerian-now-supports-physically-based-rendering-pbr/ Topic || Application Integration Amazon SNS Message Filtering Adds Support for Attribute Key Matching | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-sns-message-filtering-adds-support-for-attribute-key-matching/ Amazon SNS Adds Support for AWS X-Ray | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-sns-adds-support-for-aws-x-ray/ Temporary Queue Client Now Available for Amazon SQS | https://aws.amazon.com/about-aws/whats-new/2019/07/temporary-queue-client-now-available-for-amazon-sqs/ Amazon MQ Adds Support for AWS Key Management Service (AWS KMS), Improving Encryption Capabilities | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-mq-adds-support-for-aws-key-management-service-improving-encryption-capabilities/ Amazon MSK adds support for Apache Kafka version 2.2.1 and expands availability to EU (Stockholm), Asia Pacific (Mumbai), and Asia Pacific (Seoul) | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-msk-adds-support-apache-kafka-version-221-expands-availability-stockholm-mumbai-seoul/ Amazon API Gateway supports secured connectivity between REST APIs & Amazon Virtual Private Clouds in additional regions | https://aws.amazon.com/about-aws/whats-new/2019/08/amazon-api-gateway-supports-secured-connectivity-between-reset-apis-and-amazon-virtual-private-clouds-in-additional-regions/ Topic || Management and Governance AWS Cost Explorer now Supports Usage-Based Forecasts | https://aws.amazon.com/about-aws/whats-new/2019/07/usage-based-forecasting-in-aws-cost-explorer/ Introducing Amazon EC2 Resource Optimization Recommendations | https://aws.amazon.com/about-aws/whats-new/2019/07/introducing-amazon-ec2-resource-optimization-recommendations/ AWS Budgets Announces AWS Chatbot Integration | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-budgets-announces-aws-chatbot-integration/ Discovering Documents Made Easy in AWS Systems Manager Automation | https://aws.amazon.com/about-aws/whats-new/2019/07/discovering-documents-made-easy-in-aws-systems-manager-automation/ AWS Systems Manager Distributor makes it easier to create distributable software packages | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-systems-manager-distributor-makes-it-easier-to-create-distributable-software-packages/ Now use AWS Systems Manager Maintenance Windows to select resource groups as targets | https://aws.amazon.com/about-aws/whats-new/2019/07/now-use-aws-systems-manager-maintenance-windows-to-select-resource-groups-as-targets/ Use AWS Systems Manager to resolve operational issues with your .NET and Microsoft SQL Server Applications | https://aws.amazon.com/about-aws/whats-new/2019/08/use-aws-systems-manager-to-resolve-operational-issues-with-your-net-and-microsoft-sql-server-applications/ CloudWatch Logs Insights adds cross log group querying | https://aws.amazon.com/about-aws/whats-new/2019/07/cloudwatch-logs-insights-adds-cross-log-group-querying/ AWS CloudFormation now supports higher StackSets limits | https://aws.amazon.com/about-aws/whats-new/2019/08/aws-cloudformation-now-supports-higher-stacksets-limits/ Topic || Customer Engagement Introducing AI-Driven Social Media Dashboard | https://aws.amazon.com/about-aws/whats-new/2019/07/introducing-ai-driven-social-media-dashboard/ New Amazon Connect integration for ChoiceView from Radish Systems on AWS | https://aws.amazon.com/about-aws/whats-new/2019/07/new-amazon-connect-integration-for-choiceview-from-radish-systems-on-aws/ Amazon Pinpoint Adds Campaign and Application Metrics APIs | https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-pinpoint-adds-campaign-and-application-metrics-apis/ Topic || Media AWS Elemental Appliances and Software Now Available in the AWS Management Console | https://aws.amazon.com/about-aws/whats-new/2019/08/aws-elemental-appliances-and-software-now-available-in-aws-management-console/ AWS Elemental MediaConvert Expands Audio Support and Improves Performance | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-elemental-mediaconvert-expands-audio-support-and-improves-performance/ AWS Elemental MediaConvert Adds Ability to Prioritize Transcoding Jobs | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-elemental-mediaconvert-adds-ability-to-prioritize-transcoding-jobs/ AWS Elemental MediaConvert Simplifies Editing and Sharing of Settings | https://aws.amazon.com/about-aws/whats-new/2019/08/aws-elemental-mediaconvert-simplifies-editing-and-sharing-of-settings/ AWS Elemental MediaStore Now Supports Resource Tagging | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-elemental-mediastore-now-supports-resource-tagging/ AWS Elemental MediaLive Enhances Support for File-Based Inputs for Live Channels | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-elemental-medialive-enhances-support-for-file-based-inputs-for-live-channels/ Topic || Mobile AWS Device Farm improves device start up time to enable instant access to devices | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-device-farm-improves-device-start-up-time-to-enable-instant-access-to-devices/ Topic || Security Introducing the Amazon Corretto Crypto Provider (ACCP) for Improved Cryptography Performance | https://aws.amazon.com/about-aws/whats-new/2019/07/introducing-the-amazon-corretto-crypto-provider/ AWS Secrets Manager now supports VPC endpoint policies | https://aws.amazon.com/about-aws/whats-new/2019/07/AWS-Secrets-Manager-now-supports-VPC-endpoint-policies/ Topic || Gaming Lumberyard Beta 1.20 Now Available | https://aws.amazon.com/about-aws/whats-new/2019/07/lumberyard-beta-120-now-available/ Topic || Robotics AWS RoboMaker now supports offline logs and metrics for the AWS RoboMaker CloudWatch cloud extension | https://aws.amazon.com/about-aws/whats-new/2019/07/aws-robomaker-now-supports-offline-logs-metrics-aws-robomaker-cloudwatch-cloud-extension/ Topic || Training New AWS Certification Exam Vouchers Make Certifying Groups Easier | https://aws.amazon.com/about-aws/whats-new/2019/07/new-aws-certification-exam-vouchers-make-certifying-groups-easier/ Announcing New Resources and Website to Accelerate Your Cloud Adoption | https://aws.amazon.com/about-aws/whats-new/2019/07/announcing-new-resources-and-website-to-accelerate-your-cloud-adoption/ AWS Developer Series Relaunched on edX | https://aws.amazon.com/about-aws/whats-new/2019/08/aws-developer-series-relaunched-on-edx/

Simon takes you through the December updates to finish up 2018! Shownotes: Topic || Customer Engagement 0:23 Amazon Pinpoint Announces Event-Based Campaigns, Driving Personalization and Engagement | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-pinpoint-announces-event-based-campaigns-driving-personalization-and-engagement/ Amazon Pinpoint Announces a New Email Deliverability Dashboard to Help Customers Reach their Users' Inboxes | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-pinpoint-announces-a-new-email-deliverability-dashboard-to-help-customers-reach-their-users-inboxes/ Amazon Connect Adds New Contact API to Get Contact Attributes | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-connect-adds-new-contact-api-to-get-contact-attributes/ Topic || Storage 2:05 Amazon S3 Inventory adds Apache Parquet output format | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-s3-announces-parquet-output-format-for-inventory/ AWS Storage Gateway Increases File Gateway Performance - Amazon Web Services | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-storage-gateway-announces-increased-throughput-and-adds-new-/ Topic || Networking & Content Delivery 3:32 Amazon Virtual Private Clouds can now be shared with other AWS Accounts | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-virtual-private-clouds-can-now-be-shared-with-other-aws-accounts/ Introducing AWS Client VPN to Securely Access AWS and On-Premises Resources | https://aws.amazon.com/about-aws/whats-new/2018/12/introducing-aws-client-vpn-to-securely-access-aws-and-on-premises-resources/ New AWS Direct Connect locations in Silicon Valley and Stockholm | https://aws.amazon.com/about-aws/whats-new/2018/12/new-aws-direct-connect-locations-silicon-valley-stockholm/ Amazon CloudFront announces ten new Edge locations in North America, Europe, and Asia | https://aws.amazon.com/about-aws/whats-new/2018/12/cloudfront-dec2018-10-edge-locations/ Amazon API Gateway Simplifies Building Real-Time Two-Way Communication Applications with WebSocket APIs | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-api-gateway-launches-support-for-websocket-apis/ Amazon Route 53 Adds Alias Record Support For API Gateway and VPC Endpoints | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-route-53-adds-alias-record-support-for-api-gateway-and-vpc-endpoints/ Topic || Database 7:41 Introducing Workload Qualification Framework to Project Plan Your Database Migrations to AWS | https://aws.amazon.com/about-aws/whats-new/2018/12/introducing-workload-qualification-framework-to-plan-your-database-migration-projects/ AWS Database Migration Service Adds Support for Parallel Full Load and Enhanced LOB Migration | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-database-migration-service-adds-support-for-parallel-full-load/ Amazon RDS Enhances Automatic Minor Version Upgrades | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-rds-enhances-auto-minor-version-upgrades/ Amazon RDS for PostgreSQL Now Supports R5 Instance Types | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-rds-postgresql-now-supports-r5-instance-types/ Amazon RDS Supports Publishing PostgreSQL Log Files to Amazon CloudWatch Logs | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-rds-supports-postgresql-logfiles-publish-to-amazon-cloudwatch-logs/ Amazon RDS Performance Insights Supports Counter Metrics for Aurora PostgreSQL | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-rds-performance-insights-supports-counter-metrics-for-aurora-postgresql/ Amazon RDS for PostgreSQL Supports New Minor Versions 10.6, 9.6.11, 9.5.15, and 9.4.20 | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-rds-postgresql-supports-minor-version-106/ Amazon Aurora with PostgreSQL Compatibility Supports PostgreSQL 10.5 | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-aurora-postgresql-supports-postgresql-105/ Amazon Aurora with PostgreSQL Compatibility Adds Query Plan Management | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-aurora-postgresql-compatibility-adds-query-plan-management/ Announcing the New Amazon DynamoDB Key Diagnostics Library | https://aws.amazon.com/about-aws/whats-new/2018/12/announcing-the-new-amazon-dynamodb-key-diagnostics-library/ Amazon DynamoDB Increases the Number of Global Secondary Indexes and Projected Index Attributes You Can Create Per Table | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-dynamodb-increases-the-number-of-global-secondary-indexes-and-projected-index-attributes-you-can-create-per-table/ Amazon DynamoDB Accelerator (DAX) Adds Support for DynamoDB Transactions | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-dynamodb-accelerator-adds-support-for-dynamodb-transactions/ Amazon MQ Now Supports ActiveMQ Minor Version 5.15.8 | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-mq-now-supports-activemq-minor-version5-15-8/ Topic || Compute 14:13 Amazon ECR Console Version 2 | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-ecr-console-version-2/ Amazon ECR now allows Repository Tagging | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-ecr-now-allows-repository-tagging/ Amazon EC2 Introduces Partition Placement Groups | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-ec2-ntroduces-partition-placement-groups/ AWS Auto Scaling is Now Available in 8 more Regions Worldwide and Offers Predictive Scaling for Amazon EC2 | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-auto-scaling-is-now-available-in-8-more-regions-worldwide/ Amazon EC2 C5d, M5d, and R5d Instances are Now Available in Additional AWS Regions | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-ec2-c5d-m5d-and-r5d-instances-are-now-available-in-additional-aws-regions/ AWS Fargate Platform Version 1.3 Adds Secrets Support | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-fargate-platform-version-1-3-adds-secrets-support/ Amazon EKS Adds Managed Cluster Updates and Support for Kubernetes Version 1.11 | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-eks-adds-managed-cluster-updates-and-support-for-kubernetes/ AWS Server Migration Service Adds Support for Multi-Server Migration | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-server-migration-service-adds-support-for-multi-server-migration/ AWS Batch now supports Amazon EC2 C5n Instances Featuring 100 Gbps of Network Bandwidth | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-batch-now-supports-amazon-ec2-c5n-instances-featuring-100-gbps-of-network-bandwidth/ AWS Batch Now Supports Amazon EC2 P3dn Instances | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-batch-now-supports-amazon-ec2-p3dn-instances/ New AWS ParallelCluster Features | https://aws.amazon.com/about-aws/whats-new/2018/12/new-aws-parallelcluster-features/ New SAM PUBLISH Command Simplifies Publishing Applications to the AWS Serverless Application Repository | https://aws.amazon.com/about-aws/whats-new/2018/12/sam-publish-command-simplifies-publishing-apps-to-serverless-application-repository/ AWS Elastic Beanstalk Adds Tag-Based Permissions | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-elastic-beanstalk-adds-tag-based-permissions/ Topic || Developer Tools 20:39 AWS X-Ray Adds the Ability to Group Traces by Root Cause | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-xray-adds-the-ability-to-group-traces-by-root-cause/ AWS CodePipeline Supports VPC Endpoints | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-codepipeline-supports-vpc-endpoints/ AWS CloudFormation macros can now be used in templates with nested stacks | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-cloudformation-macros-can-now-be-used-in-templates-with-nest/ Quickly Create, Build, and Deploy Amazon Alexa Skills from AWS | https://aws.amazon.com/about-aws/whats-new/2018/12/quickly-create-build-and-deploy-amazon-alexa-skills-from-aws/ Topic || Machine Learning 22:07 Amazon Transcribe now supports speech-to-text in French, Italian, and Brazilian Portuguese | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-transcribe-now-supports-speech-to-text-in-french-italian-and-brazilian-portuguese/ Topic || Security, Identity and Compliance 22:27 AWS IAM Console Now Available In German, Portuguese, Spanish, Italian, and Traditional Chinese | https://aws.amazon.com/about-aws/whats-new/2018/12/iam-console-available-in-new-languages/ Automate AWS IAM Permissions Analysis Using the New IAM Access Advisor APIs | https://aws.amazon.com/about-aws/whats-new/2018/12/iam_access_advisor_apis/ Introducing Notifications for New Amazon GuardDuty Finding Types and Feature Releases | https://aws.amazon.com/about-aws/whats-new/2018/12/Introducing-Notifications-for-New-Amazon-GuardDuty-Finding-Types-and-Feature-Releases/ AWS Organizations Supports AWS License Manager Cross Account Sharing Capabilities | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-organizations-supports-aws-license-manager/ AWS Shield Adds Advanced DDoS Protection for AWS Global Accelerator | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-shield-adds-advanced-ddos-protection-for-aws-global-accelerator/ AWS Systems Manager Automation Now Supports at Scale Action | https://aws.amazon.com/about-aws/whats-new/2018/12/AWS-Systems-Manager-Automation-Now-Supports-at-Scale-Actions/ AWS Service Catalog – Integration with AWS Organizations | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-service-catalog-announces-integration-with-aws-organizations/ The AWS WAF Security Automations solution now includes a monitoring dashboard | https://aws.amazon.com/about-aws/whats-new/2018/12/the-aws-waf-security-automations-solution-now-includes-a-monitoring-dashboard/ Announcing rule group exception for Managed Rules for AWS WAF | https://aws.amazon.com/about-aws/whats-new/2018/12/announcing-rule-group-exception-for-managed-rules-for-aws-waf/ AWS Firewall Manager Available in Four Additional Regions | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-firewall-manager-now-available-in-four-more-regions/ Topic || Application Integration 26:59 Amazon SQS now Supports Amazon VPC Endpoints using AWS PrivateLink - Amazon Web Services | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-sqs-vpc-endpoints-aws-privatelink/ Amazon MQ Introduces Network of Brokers Feature | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-mq-introduces-network-of-brokers-feature/ Topic || Desktop & App Streaming 27:27 AppStream 2.0 introduces APIs to simplify app entitlements and enable delivery of virtualized apps | https://aws.amazon.com/about-aws/whats-new/2018/12/appstream-2-0-introduces-apis-to-simplify-app-entitlements-and-e/ Topic || Analytics 28:10 Support for Spark 2.4.0, and Hue 4.3.0 on Amazon EMR release 5.20.0 | https://aws.amazon.com/about-aws/whats-new/2018/12/support-for-spark-240-hue-430-on-amazon-emr-release-5200/ Amazon Redshift now runs VACUUM DELETE automatically | https://aws.amazon.com/about-aws/whats-new/2018/12/amazon-redshift-automatic-vacuum/ Topic || Internet of Things 29:41 Introducing AWS CloudFormation Template Support for AWS IoT Analytics | https://aws.amazon.com/about-aws/whats-new/2018/12/introducing-aws-cloudformation-template-support-for-aws-iot-analytics/ AWS IoT Device Defender Adds Support for Two New Security Metrics | https://aws.amazon.com/about-aws/whats-new/2018/12/aws-iot-device-defender-adds-support-for-two-new-security-metrics/ MediaTek MT7697H System on Chip is Qualified for Amazon FreeRTOS | https://aws.amazon.com/about-aws/whats-new/2018/12/mediatek-mt7697h-system-on-chip-qualified-amazon-freertos/ Topic || Other 30:35 Announcing Programmatic Access to AWS Pricing Information in China via the AWS Price List API | https://aws.amazon.com/about-aws/whats-new/2018/12/announcing-aws-price-list-api-availability-in-china/ Introducing the Media Services Application Mapper | https://aws.amazon.com/about-aws/whats-new/2018/12/introducing-the-media-services-application-mapper/ New Quick Start Deploys Varnish Cache Plus (VCP) on the AWS Cloud | https://aws.amazon.com/about-aws/whats-new/2018/12/new-quick-start-deploys-varnish-on-aws/ Announcing 15 Free Digital Training Courses on New AWS Services Launched at re:Invent 2018 | https://aws.amazon.com/about-aws/whats-new/2018/12/announcing-15-free-digital-training-courses-on-new-aws-services-launched-at-re-invent-2018/

This week we look at a shorter list of services in the Application integration group.  The list may be short, but these are highly useful and valuable services for application developers.  These are excellent ways to integrate your applications (as you would expect from the name). AWS Step Functions Coordinate Distributed Applications The Step Functions free tier includes 4,000 free state transitions per month. All charges are metered daily and billed monthly.  This service provides a way for you to visually connect pieces of your application when the pieces are serverless or otherwise stand-alone functions. Amazon Simple Queue Service (SQS) Managed Message Queues Simple Queue service is a bundled solution.  It runs on a server and uses storage for the queue.   Thus, the two pieces combine for the pricing and utilization.  You can get started with Amazon SQS for free.  Accordingly, all customers can make 1 million Amazon SQS requests for free each month. Some applications might be able to operate within this Free Tier limit. Amazon Simple Notification Service (SNS) Managed Message Topics for Pub/Sub There is a free tier that covers your first million push notifications (publishes and deliveries) every month.  The notification service is similar to a queue.  However, instead of a push and pop approach, it is a subscribe/publish model.  In general, your application subscribes to a topic.  Then, when a message appears on the queue, the application receives it and can process accordingly. Amazon MQ Managed Message Broker for ActiveMQ The Amazon Message broker Free Tier includes up to 750 hours of a single-instance mq.t2.micro broker per month.  The storage starts at up to 1GB of storage per month for one year on the free tier.  All new AWS accounts are eligible to receive the 12 months of AWS Free Tier access.  As always, you can check out the AWS Free Tier page for details.

In this session, we discuss best practices for building serverless applications that handle high throughput and bursty data using Amazon SQS, Amazon SNS, and AWS Lambda, including new features such as message filtering and SNS/SQS as event sources for Lambda will be covered in depth. Hear from our customers Enel and Letgo as they share their experiences and deployment strategies. Enel is a multinational energy company that is present in 34 countries across 5 continents, and serves nearly 71 million end-users. The company uses AWS as its platform for IoT and energy management. Letgo is one of the largest and fastest-growing apps to buy and sell locally, with over 100 million downloads and hundreds of millions of listings. Letgo uses AWS to process their online marketplace transactions.

The 2018 FIFA World Cup was the biggest production in Fox Sports 20-year history in terms of personnel, hours, and scale. With 64 matches broadcast, it totalling more than four previous World Cups combined. To deliver this, they deployed a revolutionary joint solution from AWS, Aspera, and a portfolio of APN partners that enabled the live delivery of a broadcast video from Russia to Los Angeles for remote production. In this session, learn the architectural patterns of large-scale data and content movement to the cloud over long distances with low-latency requirements, the high-availability requirements around broadcast workloads, and tradeoffs to consider. We cover how Amazon S3, Amazon EC2, AWS Lambda, Amazon SQS, and other services processed over 1.9 PB of original content over the 30-day tournament.

Data and events are the lifeblood of any modern application. By using stateless, loosely coupled microservices communicating through events, developers can build massively scalable systems that can process trillions of requests in seconds. In this talk, we cover design patterns for using Amazon SQS, Amazon SNS, AWS Step Functions, AWS Lambda, and Amazon S3 to build data processing and real-time notification systems with unbounded scale and serverless cost characteristics. We also explore how these approaches apply to practical use cases, such as training machine learning models, media processing, and data cleansing.

In this session, learn from market-leader Vonage how and why they re-architected their QoS-sensitive, highly available and highly performant legacy real-time communications systems to take advantage of Amazon EC2, Enhanced Networking, Amazon S3, ASG, Amazon RDS, Amazon ElastiCache, AWS Lambda, StepFunctions, Amazon SNS, Amazon SQS, Amazon Kinesis, Amazon EFS, and more. We also learn how Aspect, a multinational leader in call center solutions, used AWS Lambda, Amazon API Gateway, Amazon Kinesis, Amazon ElastiCache, Amazon Cognito, and Application Load Balancer with open-source API development tooling from Swagger, to build a comprehensive, microservices-based solution. Vonage and Aspect share their journey to TCO optimization, global outreach, and agility with best practices and insights.

In the cloud, modern apps are decoupled into independent building blocks, called microservices, which are easier to develop, deploy, and maintain. Messaging is a central tool used to connect and coordinate these microservices. AWS offers multiple messaging services, which address a variety of use cases. In this session, learn how to choose the service that's best for your use case as we present the key technical features of each. We pay special attention to integrating messaging services with serverless technology. We cover Amazon Kinesis, Amazon SQS, and Amazon SNS in detail with discussion of other services as appropriate.

Welcome Gabe Hollombe on-board AWS TechChat in this latest episode. Hosts Dean and Gabe start the episode with the latest AWS stats, general availability of Amazon Neptune, Amazon EKS and Amazon Sumerian. They then go into the latest from Amazon Cognito, AWS AppSync, AWS MobileHub, AWS CodeBuild, AWS CodePipeline, Amazon SQS, Application Load Balancer and Amazon SageMaker.

AWS Elastic Beanstalk provides an easy way for you to quickly deploy, manage, and scale applications in the AWS Cloud. Through interactive demos and code samples, this session will teach you how to deploy your code using Elastic Beanstalk, provision and use other AWS services (Amazon SNS, Amazon SQS, and Amazon DynamoDB, AWS CodeCommit) use your application's health metrics to tune performance, scale your application to handle millions of requests, perform zero-downtime deployments with traffic routing, and keep the underlying application platform up-to-date with managed updates.

In this session, we first look at common approaches to refactoring common legacy .NET applications to microservices and AWS serverless architectures. We also look at modern approaches to .NET-based architectures on AWS. We then elaborate on running .NET Core microservices in Docker containers natively on Linux in AWS while examining the use of AWS SDK and .NET Core platform. We also look at the use of the various AWS services such as Amazon SNS, Amazon SQS, Amazon Kinesis, and Amazon DynamoDB, which provide the backbone of the platform. For example, Experian Consumer Services runs a large ecommerce platform that is now cloud based in the AWS. We look at how they went from monolithic platform to microservices, primarily in .NET Core. With a heavy push to move to Java and open source, we look at the development process, which started in the beta days of .NET Core, and how the direction Microsoft was going allowed them to use existing C# skills while pushing themselves to innovate in AWS. The large, single team of Windows based developers was broken down into several small teams to allow for rapid development into an all Linux environment.

The BBC iPlayer is the biggest audio and video-on-demand service in the UK. Over one-third of the country submits 10 million video playback requests every day, and the service publishes over 10,000 hours of media every week. Moving iPlayer to the cloud has enabled the BBC to shorten the time-to-market of content from 10 hours to 15 minutes. In this session, the BBC's lead architect describes the approach behind creating iPlayer architecture, which uses Amazon SQS and Amazon SNS in several ways to improve elasticity, reliability, and maintainability. You see how BBC uses AWS messaging to choreograph the 200 microservices in the iPlayer pipeline, maintain data consistency as media traverses the pipeline, and refresh caches to ensure timely delivery of media to users. This is a rare opportunity to see the internal workings and best practices of one of the largest on-demand content delivery systems operating today.

Given the complexity and scale of contemporary games, it's been a dream of game creators to algorithmically generate game content. Nexon wanted to create a large-scale, open-world MMORPG called Durango, where algorithmic generation is desperately needed to minimize development costs and maintain game longevity. In-game objects such as trees and plants are placed based on complex rules, with the intention of mimicking the realistic ecosystem that evolves continuously. However, a large amount of computation and a careful orchestration of various computing resources are required due to the immense size of the in-game lands. Nexon achieved this goal by leveraging AWS services to take advantage of massive parallelism supported by the infrastructure. In this talk, Nexon discusses the architecture they settled on for algorithmic generation of game content in a large scale, and the AWS services involved such as Amazon SQS and Amazon ECS with automatic scaling and spot instances.

This session covers how the team at Ubisoft evolved For Honor's infrastructure using Amazon ECS and supporting systems (Amazon CloudFront, Amazon ElastiCache, Amazon Elasticsearch Service, Amazon SQS, and AWS Lambda, with monitoring through DataDog) from a proof of concept to an infrastructure as code solution. The team shares war stories about supporting both internal and live environments, and the challenges of bridging cloud and on-premises systems.

Serverless and AWS Lambda specifically enable developers to build super-scalable application components with minimal effort. You can use Amazon Kinesis and Amazon SQS to create a universal event stream to orchestrate Lambdas into much more complex applications. Now, using AWS Step Functions, we can build large distributed applications with Lambdas using visual workflows. See how Step Functions are different from Amazon SWF, how to get started with Step Functions, and how to use them to take your Lambda-based applications to the next level. We start with a few granular functions and stitch them up using Step Functions. As we build out the application, we add monitoring to ensure that changes we make actually improve things, not make them worse. Leave the session with actionable learnings for using Step Functions in your environment right away.   Session sponsored by Datadog

Vivint Solar is a leading full-service residential solar provider in the United States with more than 100,000 solar systems to its credit. Vivint Solar uses MicroStrategy on AWS to keep track of internal processes, including operations, finance, customer support, and human resources. With MicroStrategy on AWS, Vivint Solar could deploy a fully configured MicroStrategy environment in just an hour, giving their IT team more time to focus on other high-value projects. Today, Vivint Solar uses over 25 different AWS tools and services including, AWS Lambda, AWS Step Functions, Amazon SQS, AWS CloudFormation, and AWS CodeDeploy to track and analyze their data. Join us in this session to learn how Vivint Solar is using analytics to transform the way they do business and revolutionize the way people think about renewable energy. Session sponsored by MicroStrategy

RR 323: Queuing and Amazon SQS with Kinsey Ann Durham This episode of Ruby Rogues features panelists Charles Max Wood, Dave Kimura, and Eric Berry. Special guest Kinsey Ann Durham joins to talk about queuing and Amazon SQS. Tune in to learn more! [00:01:19] Kinsey Ann Durham Kinsey writes code for a company called Go Spot Check. She is always a lead mentor in a San Francisco based company called Bloc. [00:02:50] Background on Amazon SQS Go Spot Check is using Amazon SQS on a smaller scale. Kinsey thinks it is sasy to use. She recommends using something like Amazon SQS or even RabbitMQ. It has provided the company with the ability to explore different architecture patterns and tools. [00:04:50] Can you talk a little about your company and what led to using Amazon SQS? Go Spot Check is a start up in Denver. They focus on recording and data collection for big companies that need to know what is happening in retail, grocery stores, and bars. The focus is on alcohol and retail brands. The company analyzes the data collected that previously held no insight. Go Spot Check is currently moving into a computer vision aspect. Kinsey works off a separate service off of main aspect of Go Spot Check. [00:06:46] What does your stack look like? Is it built off Ruby? Yes, it is a Rails API only. The computer vision is done in Python. [00:08:45] Are you feeding the images through the queue? How does the queuing fit in? Started using Amazon SQS because they wanted to have a more decoupled way of developing. This allowed them to decide the contract between the two services and decide what they wanted it to look like up front. Kinsey describes that it is easy to create fake messages for testing with Amazon SQS. Image data is sent back and forth through the queue. The company does a lot of planograms. Information is taken from that data and posted onto a queue from the machine learning side of things. On the Rail side of things, the data can be picked up in API and sent back to the main app. [00:10:50] Does it accept binary data in the queue? It does not send actual images. All comparison data that has been processed is sent from the machine learning aspect side of things. An article has been published that shows that people do send images in the queue. [00:11:35] Do you use SQS in parallel with SNS (Simple Notification Service)? Kinsey says that they haven’t used SNS. This is because there hasn’t been a need. They are using it to post messages to communicate between different services. [00:12:40] What point would you need to consider a SQS over a Sidekick? Kinsey didn’t look into using Sidekick; she was excited to use SQS. She wanted to try it out and see if it was easy to use. Thought it would be more complex than it has been. She enjoys the free features of Amazon such as message visibility and timeout, which is handled by them. It can be customized and two different queues can be used. [00:16:15] How do you write the workers for an SQS queue? Kinsey has a plain Ruby object in the API that she can reuse with any queue. There are three queues in the company. [00:19:45] Are there any other uses for queues and SQS? Kinsey hasn’t come across any personally but she is sure there are some. [00:23:40] What if you’re someone who is new? Where would you recommend they get started? Suggest getting started with SQS Amazon, SQS documentation. Can get up to speed quickly. Amazon SQS is easy to get up and running. Kinsey is tailoring her Ruby Dev Summit talk to people who are new. [00:30:35] How do you go about mentoring? Kinsey loves mentoring. Developers have side projects or freelance work, but Kinsey likes to mentor because she feels like she makes a difference while continuing to learn. An important part of mentorship is giving support. This support level to students’ means not only offering students help with technical skills. Her goal is to build a well-rounded developer: someone who will be a great team member and people will want to work with in the future. This involves helping students build soft skills such as networking, interviewing skills, and helping them build confidence. [00:33:52] How would people get involved with mentorship? Kinsey is involved with an organization called Bloc - they are always hiring mentors. She shares that people can always get involved in their local community. Schools are looking for mentors. People at local meet ups and Rails Bridge are also both good ways to volunteer. Kinsey learned through mentors - she didn’t go to school to learn code. Mentors changed her life and are important to her, which is why she now mentors. [00:36:30] Advice For Women Kinsey’s advice for women who want to work in the technology world is to go for it. She urges women to get as many people and resources on their side as possible, including great developers who are willing to mentor. She emphasizes the importance of confidence and says to be ready for comments on gender. She believes that - while there are definitely still diversity issues with socioeconomic background, sexual orientation, race, gender, etc. it is getting better – women are more welcome in the technology field than they have previously been. There are technology organizations that are doing well and have no problems with welcoming women into the workplace. People in the field need to be open to having discussions about gender inequality. Open dialogue with team members is the key to solving problems. Some people have grown up not realizing the way they think is wrong. They don’t connect that what they say or think is offensive because it is all they know; it is unconscious to them. This is the type of person that is hard to change. Picks Eric: Open Collective  Open Collective – Women Who Work  Dave: Health insurance Charles: Profit First  Secrets of the Millionaire Mind Kinsey: Guide program applications for mentors at RubyConf Release It  Links for Kinsey Twitter Instagram GitHub

RR 323: Queuing and Amazon SQS with Kinsey Ann Durham This episode of Ruby Rogues features panelists Charles Max Wood, Dave Kimura, and Eric Berry. Special guest Kinsey Ann Durham joins to talk about queuing and Amazon SQS. Tune in to learn more! [00:01:19] Kinsey Ann Durham Kinsey writes code for a company called Go Spot Check. She is always a lead mentor in a San Francisco based company called Bloc. [00:02:50] Background on Amazon SQS Go Spot Check is using Amazon SQS on a smaller scale. Kinsey thinks it is sasy to use. She recommends using something like Amazon SQS or even RabbitMQ. It has provided the company with the ability to explore different architecture patterns and tools. [00:04:50] Can you talk a little about your company and what led to using Amazon SQS? Go Spot Check is a start up in Denver. They focus on recording and data collection for big companies that need to know what is happening in retail, grocery stores, and bars. The focus is on alcohol and retail brands. The company analyzes the data collected that previously held no insight. Go Spot Check is currently moving into a computer vision aspect. Kinsey works off a separate service off of main aspect of Go Spot Check. [00:06:46] What does your stack look like? Is it built off Ruby? Yes, it is a Rails API only. The computer vision is done in Python. [00:08:45] Are you feeding the images through the queue? How does the queuing fit in? Started using Amazon SQS because they wanted to have a more decoupled way of developing. This allowed them to decide the contract between the two services and decide what they wanted it to look like up front. Kinsey describes that it is easy to create fake messages for testing with Amazon SQS. Image data is sent back and forth through the queue. The company does a lot of planograms. Information is taken from that data and posted onto a queue from the machine learning side of things. On the Rail side of things, the data can be picked up in API and sent back to the main app. [00:10:50] Does it accept binary data in the queue? It does not send actual images. All comparison data that has been processed is sent from the machine learning aspect side of things. An article has been published that shows that people do send images in the queue. [00:11:35] Do you use SQS in parallel with SNS (Simple Notification Service)? Kinsey says that they haven’t used SNS. This is because there hasn’t been a need. They are using it to post messages to communicate between different services. [00:12:40] What point would you need to consider a SQS over a Sidekick? Kinsey didn’t look into using Sidekick; she was excited to use SQS. She wanted to try it out and see if it was easy to use. Thought it would be more complex than it has been. She enjoys the free features of Amazon such as message visibility and timeout, which is handled by them. It can be customized and two different queues can be used. [00:16:15] How do you write the workers for an SQS queue? Kinsey has a plain Ruby object in the API that she can reuse with any queue. There are three queues in the company. [00:19:45] Are there any other uses for queues and SQS? Kinsey hasn’t come across any personally but she is sure there are some. [00:23:40] What if you’re someone who is new? Where would you recommend they get started? Suggest getting started with SQS Amazon, SQS documentation. Can get up to speed quickly. Amazon SQS is easy to get up and running. Kinsey is tailoring her Ruby Dev Summit talk to people who are new. [00:30:35] How do you go about mentoring? Kinsey loves mentoring. Developers have side projects or freelance work, but Kinsey likes to mentor because she feels like she makes a difference while continuing to learn. An important part of mentorship is giving support. This support level to students’ means not only offering students help with technical skills. Her goal is to build a well-rounded developer: someone who will be a great team member and people will want to work with in the future. This involves helping students build soft skills such as networking, interviewing skills, and helping them build confidence. [00:33:52] How would people get involved with mentorship? Kinsey is involved with an organization called Bloc - they are always hiring mentors. She shares that people can always get involved in their local community. Schools are looking for mentors. People at local meet ups and Rails Bridge are also both good ways to volunteer. Kinsey learned through mentors - she didn’t go to school to learn code. Mentors changed her life and are important to her, which is why she now mentors. [00:36:30] Advice For Women Kinsey’s advice for women who want to work in the technology world is to go for it. She urges women to get as many people and resources on their side as possible, including great developers who are willing to mentor. She emphasizes the importance of confidence and says to be ready for comments on gender. She believes that - while there are definitely still diversity issues with socioeconomic background, sexual orientation, race, gender, etc. it is getting better – women are more welcome in the technology field than they have previously been. There are technology organizations that are doing well and have no problems with welcoming women into the workplace. People in the field need to be open to having discussions about gender inequality. Open dialogue with team members is the key to solving problems. Some people have grown up not realizing the way they think is wrong. They don’t connect that what they say or think is offensive because it is all they know; it is unconscious to them. This is the type of person that is hard to change. Picks Eric: Open Collective  Open Collective – Women Who Work  Dave: Health insurance Charles: Profit First  Secrets of the Millionaire Mind Kinsey: Guide program applications for mentors at RubyConf Release It  Links for Kinsey Twitter Instagram GitHub

RR 323: Queuing and Amazon SQS with Kinsey Ann Durham This episode of Ruby Rogues features panelists Charles Max Wood, Dave Kimura, and Eric Berry. Special guest Kinsey Ann Durham joins to talk about queuing and Amazon SQS. Tune in to learn more! [00:01:19] Kinsey Ann Durham Kinsey writes code for a company called Go Spot Check. She is always a lead mentor in a San Francisco based company called Bloc. [00:02:50] Background on Amazon SQS Go Spot Check is using Amazon SQS on a smaller scale. Kinsey thinks it is sasy to use. She recommends using something like Amazon SQS or even RabbitMQ. It has provided the company with the ability to explore different architecture patterns and tools. [00:04:50] Can you talk a little about your company and what led to using Amazon SQS? Go Spot Check is a start up in Denver. They focus on recording and data collection for big companies that need to know what is happening in retail, grocery stores, and bars. The focus is on alcohol and retail brands. The company analyzes the data collected that previously held no insight. Go Spot Check is currently moving into a computer vision aspect. Kinsey works off a separate service off of main aspect of Go Spot Check. [00:06:46] What does your stack look like? Is it built off Ruby? Yes, it is a Rails API only. The computer vision is done in Python. [00:08:45] Are you feeding the images through the queue? How does the queuing fit in? Started using Amazon SQS because they wanted to have a more decoupled way of developing. This allowed them to decide the contract between the two services and decide what they wanted it to look like up front. Kinsey describes that it is easy to create fake messages for testing with Amazon SQS. Image data is sent back and forth through the queue. The company does a lot of planograms. Information is taken from that data and posted onto a queue from the machine learning side of things. On the Rail side of things, the data can be picked up in API and sent back to the main app. [00:10:50] Does it accept binary data in the queue? It does not send actual images. All comparison data that has been processed is sent from the machine learning aspect side of things. An article has been published that shows that people do send images in the queue. [00:11:35] Do you use SQS in parallel with SNS (Simple Notification Service)? Kinsey says that they haven’t used SNS. This is because there hasn’t been a need. They are using it to post messages to communicate between different services. [00:12:40] What point would you need to consider a SQS over a Sidekick? Kinsey didn’t look into using Sidekick; she was excited to use SQS. She wanted to try it out and see if it was easy to use. Thought it would be more complex than it has been. She enjoys the free features of Amazon such as message visibility and timeout, which is handled by them. It can be customized and two different queues can be used. [00:16:15] How do you write the workers for an SQS queue? Kinsey has a plain Ruby object in the API that she can reuse with any queue. There are three queues in the company. [00:19:45] Are there any other uses for queues and SQS? Kinsey hasn’t come across any personally but she is sure there are some. [00:23:40] What if you’re someone who is new? Where would you recommend they get started? Suggest getting started with SQS Amazon, SQS documentation. Can get up to speed quickly. Amazon SQS is easy to get up and running. Kinsey is tailoring her Ruby Dev Summit talk to people who are new. [00:30:35] How do you go about mentoring? Kinsey loves mentoring. Developers have side projects or freelance work, but Kinsey likes to mentor because she feels like she makes a difference while continuing to learn. An important part of mentorship is giving support. This support level to students’ means not only offering students help with technical skills. Her goal is to build a well-rounded developer: someone who will be a great team member and people will want to work with in the future. This involves helping students build soft skills such as networking, interviewing skills, and helping them build confidence. [00:33:52] How would people get involved with mentorship? Kinsey is involved with an organization called Bloc - they are always hiring mentors. She shares that people can always get involved in their local community. Schools are looking for mentors. People at local meet ups and Rails Bridge are also both good ways to volunteer. Kinsey learned through mentors - she didn’t go to school to learn code. Mentors changed her life and are important to her, which is why she now mentors. [00:36:30] Advice For Women Kinsey’s advice for women who want to work in the technology world is to go for it. She urges women to get as many people and resources on their side as possible, including great developers who are willing to mentor. She emphasizes the importance of confidence and says to be ready for comments on gender. She believes that - while there are definitely still diversity issues with socioeconomic background, sexual orientation, race, gender, etc. it is getting better – women are more welcome in the technology field than they have previously been. There are technology organizations that are doing well and have no problems with welcoming women into the workplace. People in the field need to be open to having discussions about gender inequality. Open dialogue with team members is the key to solving problems. Some people have grown up not realizing the way they think is wrong. They don’t connect that what they say or think is offensive because it is all they know; it is unconscious to them. This is the type of person that is hard to change. Picks Eric: Open Collective  Open Collective – Women Who Work  Dave: Health insurance Charles: Profit First  Secrets of the Millionaire Mind Kinsey: Guide program applications for mentors at RubyConf Release It  Links for Kinsey Twitter Instagram GitHub

Join Dr Pete and Russ in the latest episode of AWS TechChat as they share security best practices and news around Amazon Rekognition, Amazon DynamoDB, Amazon ECS, AWS Greengrass, Amazon SQS, AWS CodeStar, AWS Lambda, AWS X-Ray, Amazon Aurora, Amazon RDS, Amazon EMR and AWS Deep Learning AMIs.

AWS Elastic Beanstalk provides an easy way for you to quickly deploy, manage, and scale applications in the AWS Cloud. Through interactive demos and code samples, this session will teach you how to deploy your code using Elastic Beanstalk, provision and use other AWS services (Amazon SNS, Amazon SQS, and Amazon DynamoDB), use your application’s health metrics to tune performance, scale your application to handle millions of requests, perform zero-downtime deployments with traffic routing, and keep the underlying application platform up-to-date with managed updates. Code samples for demos will be available to all session attendees.

Enterprises rely on messaging to integrate services and applications and to exchange information critical to running their business. However, managing and operating dedicated message-oriented middleware and underlying infrastructure creates costly overhead and can compromise reliability. In this session, enterprise architects and developers learn how to improve scalability, availability, and operational efficiency by migrating on-premises messaging middleware to a managed cloud service using Amazon SQS. Hear how Capital One is using SQS to migrate several core banking applications to the cloud to ensure high availability and cost efficiency. We also share some exciting new SQS features that allow even more workloads to take advantage of the cloud.

Did you know that AWS enables builders to architect solutions for price? Beyond the typical challenges of function, performance, and scale, you can make your application cost effective. Using different architectural patterns and AWS services in concert can dramatically reduce the cost of systems operation and per-transaction costs. This session uses practical examples aimed at architects and developers. Using code and AWS CloudFormation in concert with services such as Amazon EC2, Amazon ECS, Lambda, Amazon RDS, Amazon SQS, Amazon SNS, Amazon S3, CloudFront, and more, we demonstrate the financial advantages of different architectural decisions. Attendees will walk away with concrete examples, as well as a new perspective on how they can build systems economically and effectively.

Nike+ is at the core of the Nike digital product ecosystem, providing services to enhance your athletic experience through quantified activity tracking and gamification. As one of the first movers at Nike to migrate out of the datacenter to AWS, they share the evolution in building a reactive platform on AWS to handle large, complex data sets. They provide a deep technical view of how they process billions of metrics a day in their quantified-self platform, supporting millions of customers worldwide. You’ll leave with ideas and tools to help your organization scale in the cloud. Come learn from experts who have built an elastic platform using Java, Scala, and Akka, leveraging the power of many AWS technologies like Amazon EC2, ElastiCache, Amazon SQS, Amazon SNS, DynamoDB, Amazon ES, Lambda, Amazon S3, and a few others that helped them (and can help you) get there quickly.

AWS serverless architecture components such as Amazon S3, Amazon SQS, Amazon SNS, CloudWatch Logs, DynamoDB, Amazon Kinesis, and Lambda can be tightly constrained in their operation. However, it may still be possible to use some of them to propagate payloads that could be used to exploit vulnerabilities in some consuming endpoints or user-generated code. This session explores techniques for enhancing the security of these services, from assessing and tightening permissions in IAM to integrating tools and mechanisms for inline and out-of-band payload analysis that are more typically applied to traditional server-based architectures.

Batch computing is a common way for developers, scientists and engineers to run a series of jobs on a large pool of shared compute resources, such as servers, virtual machines, and containers. Amazon ECS makes it easy to run and manage Docker-enabled applications across a cluster of Amazon EC2 instances. In this session will show you how to run batch jobs using Amazon ECS and together with other AWS services, such as AWS Lambda and Amazon SQS. We will see how you can leverage Amazon EC2 Spot Instances to power your ECS cluster and easily scale your batch workloads. You'll hear from Mapbox on how they use ECS to power their entire batch processing architecture to collect and process over 100 million miles of sensor data per day that they use for powering their maps. Mapbox will also discuss how they optimize their batch processing framework on ECS using Spot Instances and demo their open source framework that will help you get up and running with ECS in minutes.

In this episode Simon covers a raft of updates that have happened in the last few weeks - hopefully something for everyone! AWS CodeDeploy with Amazon CloudWatch Events: http://docs.aws.amazon.com/codedeploy/latest/userguide/monitoring-cloudwatch-events.html Amazon CloudFront HTTP/2 support: https://aws.amazon.com/about-aws/whats-new/2016/09/amazon-cloudfront-now-supports-http2/ and Query String Whitelisting: https://aws.amazon.com/about-aws/whats-new/2016/08/announcing-query-string-whitelisting-for-amazon-cloudfront/ Reader endpoint for Amazon Aurora: https://aws.amazon.com/blogs/aws/new-reader-endpoint-for-amazon-aurora-load-balancing-higher-availability/ and Release 1.7 updates: https://aws.amazon.com/blogs/aws/amazon-aurora-update-parallel-read-ahead-faster-indexing-numa-awareness/ AWS Config ALB Integration: https://aws.amazon.com/about-aws/whats-new/2016/08/aws-config-adds-support-for-application-load-balancers/ Mobile Hub Integrate Tab: https://aws.amazon.com/about-aws/whats-new/2016/09/aws-mobile-hub-simplifies-backend-feature-integration-into-existing-mobile-apps/ AWS Storage Gateway Console Update: https://aws.amazon.com/about-aws/whats-new/2016/08/weve-redesigned-the-aws-storage-gateway-console/ Auto Scaling for EC2 Spot Fleets: https://aws.amazon.com/blogs/aws/new-auto-scaling-for-ec2-spot-fleets/ AWS Elastic Beanstalk Nginx with Tomcat:https://aws.amazon.com/about-aws/whats-new/2016/08/aws-elastic-beanstalk-supports-nginx-proxy-server-with-tomcat/, ALB support: https://aws.amazon.com/about-aws/whats-new/2016/08/aws-elastic-beanstalk-supports-application-load-balancer/, .NET enhancements: https://aws.amazon.com/about-aws/whats-new/2016/08/aws-elastic-beanstalk-supports-asp-net-core-and-multi-app-net-support/ Amazon CloudWatch Logs Console enhancements: https://aws.amazon.com/blogs/aws/improvements-to-cloudwatch-logs-dashboards/ 50 Tags per AWS Resource: http://blogs.aws.amazon.com/security/post/Tx3O5RCX34VOGY6/Now-Organize-Your-AWS-Resources-by-Using-up-to-50-Tags-per-Resource EC2 Auto Scaling Group Metric Improvements: https://aws.amazon.com/about-aws/whats-new/2016/08/free-auto-scaling-group-metrics-with-graphs/ Amazon WorkSpaces Updates: https://aws.amazon.com/blogs/aws/amazon-workspaces-update-hourly-usage-and-expanded-root-volume/ New Oldest Message Metric Amazon CloudWatch metric for Amazon SQS: https://aws.amazon.com/about-aws/whats-new/2016/08/new-amazon-cloudwatch-metric-for-amazon-sqs-monitors-the-age-of-the-oldest-message/

Brian Chapados talks about using Ruby and Amazon's Simple Que Service web service to build backdoors into systems.