Podcasts about Memcached

Hosts Lois Houston and Nikita Abraham are joined by Senior Principal Database & MySQL Instructor Bill Millar who explains Oracle's newest caching solution called True Cache. Available in Oracle Database 23ai, True Cache is an automatically managed, in-memory, read-only cache that improves application performance dramatically. Bill provides an overview of its features and highlights the benefits of using True Cache.   Oracle MyLearn: https://mylearn.oracle.com/ou/course/oracle-database-23ai-new-features-for-administrators/140830/   Oracle University Learning Community: https://education.oracle.com/ou-community   LinkedIn: https://www.linkedin.com/showcase/oracle-university/   X: https://twitter.com/Oracle_Edu   Special thanks to Arijit Ghosh, David Wright, and the OU Studio Team for helping us create this episode.   --------------------------------------------------------   Episode Transcript:   00:00 Welcome to the Oracle University Podcast, the first stop on your cloud journey. During this series of informative podcasts, we'll bring you foundational training on the most popular Oracle technologies. Let's get started! 00:26 Lois: Hello and welcome to the Oracle University Podcast. I'm Lois Houston, Director of Innovation Programs with Oracle University, and with me is Nikita Abraham, Principal Technical Editor. Nikita: Hi everyone! Last week, we had quite a power-packed episode. We discussed the 23ai new feature for Automatic SQL Plan Management. We also looked at the 23ai automatic feature that enhances SecureFiles LOB Write Performance as well as the update to Wide Columns. 00:59 Lois: Yeah, and in today's episode, we will look at True Cache, another 23ai new feature. To tell us all about it, we have Bill Millar back with us. Bill is a Senior Principal Database & MySQL Instructor with Oracle University. We'll ask Bill to give us an overview of True Cache, talk about its configuration and deployment, and discuss how to apply True Cache to our applications.  Nikita: To kick things off, Bill, can you give us a high-level overview of what True Cache is? How does it differ from other caching solutions like Redis or Memcached? 01:35 Bill: True Cache is an in-memory cache. It is read-only. True Cache is deployed in front of a primary database, and it is automatically managed. It keeps the most frequently accessed data in the cache, and it keeps the cache consistent with the primary database. They call it diskless, but it's not. It does require some space for SP file, redo logs, control files, and such. But it's very similar to Active Data Guard.  The queries can be offloaded to the True Cache for faster query response. And the data in the query cache is consistent. Unlike other mid-tier caches like Redis or Memcached, a query to the True Cache returns only committed data, and the data is always consistent. It's secure. Why? Because we implement our Oracle database security policies and you can control access to the cache.  02:33 Lois: So, why should we use True Cache?  Bill: Improve application performance without having to rewrite any applications. That can save considerable amount of time, effort, and expense. Reduces the application response time. So the closer the True Cache is to the application, the faster the response. Now, you do need a large amount of memory. We're talking memory here. It's an in-memory storage area, and depending on how you configure it, you can have it shared, you can have it divided. We mentioned it's automatically maintained. So there's no application changes required, and it is transparent to the application. Again, simplifies that development and maintenance.  03:15 Nikita: How does it impact application performance, and what kind of scenarios would benefit the most from implementing True Cache? Bill: So at a high-level view, True Cache or primary database, the application configuration serves as other things that are going to decide where is it going to query the data from, from the True Cache or from the primary database.  The True Cache satisfies that query. And that's where the data will be fetched from. If not, then from the primary database. On start up, True Cache is empty. So it starts reading large chunks of data to populate the True Cache. So after a block is cached, then again, it can be automatically updated, apply the redo to it-- very similar to the Oracle Active Data Guard. In the data returned, it is always going to be consistent.  04:04 Lois: Is it going to be current data? Bill: Maybe, maybe not. If it's been updated in the primary, if they redo apply hasn't occurred yet, then it's not the most consistent. But as far as the query cache is concerned, it is the most current because we only display consistent. You can have multiple True Caches. You can save the same database application service to the True Cache as you can partition it.  04:28 Nikita: I'm curious about the memory requirements, Bill. How crucial is memory for True Cache's performance? Bill: You need to have significant amount of memory. Memory, memory, memory. So True Cache is completely memory, memory. So I want to have all my data possible in there. The more memory you have, the less likely something is going to age out. And of course, just like with the standard caching, you can also pin objects to stay in the True Cache.  Yeah, like I said, there are some requirements for storage, even though it's called diskless because of, again, redo log files, configuration files like the control files, SP file. And again it is read only.  05:11 Lois: Can you explain the differences between using physical and logical connections with True Cache? How does this impact the way applications interact with the database? Bill: So with using the True Cache, we have two physical connections, and we can have one to the primary database and one to the True Cache. Each connection has a database application service associated with it, and it's going to choose which connection to use based whether it's going to go to the True Cache or to the primary database.  The second way is the application maintains one logical connection that uses the application service for the primary database. It's the JDBC thin driver, starting with Oracle Database 23 is available. It's going to maintain the physical connections to the primary database and the True Cache itself. Now, the logical connection, the one logical and one physical, is for Java applications only.  Applications that work with JSON, we extend the HTTP entity tag support for that. So a database GET request to the True Cache is going to compute the ETag, insert it into the return document.  06:27 Nikita: But what happens if there's a mismatch when the modified document is put back into the primary database?  Bill: Well, then the database is going to verify. OK, what happens with that?  It's going to verify the document row still matches that ETag for that. If with that put command, let's say, I have new data here, the row is going to match that ETag that was automatically updated. If there's no match, another user has changed the data and the PUT request is rejected. So the PUT request can be retired using the new data.  07:05 Are you planning to become an Oracle Certified Professional this year? Whether you're a seasoned IT pro or just starting your career, getting certified can give you a significant boost. And don't worry, we've got your back! Join us at one of our cert prep live events in the Oracle University Learning Community. You'll get insider tips from seasoned experts and learn from other professionals' experiences. Plus, once you've earned your certification, you'll become part of our exclusive forum for Oracle-certified users. So, what are you waiting for? Head over to mylearn.oracle.com and create an account to jump-start your journey towards certification today! 07:48 Nikita: Welcome back! Now, how do you configure True Cache, Bill? Bill: You can configure True Cache one of two ways. You can either use the Database Configuration Assistant, which actually makes it a little simpler to configure it, and you can also manually create it.  You have some environment options. One is a uniform configuration where you can deploy identical True Cache that use the same database application service. Another way is partition configuration. The data is going to be divided across multiple True Caches, which, each cache is a different subset of the data. You can also deploy True Cache in a RAC environment. As one might expect, there are some additional configuration steps for a RAC environment.  You want to make sure you verify your configuration, that the database application services are working as expected after you configure it. And then, optionally, you can enable DML redirection. What that will do, it writes data to the primary database, and that data is automatically updated in the cache. It's very similar how to the Oracle Active Data Guard works. Because the DML redirection uses more resources, it's not recommended for update-intensive applications.  There is a parameter, a ADG_REDIRECT_DML initialization parameter, that you will set to True in order to do that.  09:18 Lois: Bill, what are the specific challenges or considerations that administrators should be aware of during the configuration process?  Bill: You do need to make sure your network is configured for True Cache in the primary database. So optionally, you can create a remote listener for high availability. But you create your True Cache. You go ahead, and make sure that you have your primary database. You want the network configuration for both of those. And then you create the True Cache. Once the True Cache is created, you're going to create the application services associated with the database. And then, you're going to start the database application services on the True Cache.  When it comes to naming the application service names, each primary database application is going to be associated with a corresponding True Cache application service. To help simplify things a little bit, in the naming convention, you'll notice in our examples-- for example, if we have SALES as the primary database service, then we have the True Cache, we have SALES_TC, standing for True Cache, so it's easily identified.  You don't have to do that, but it's kind of recommended to do that, some way that you're going to identify it. So we're going to start our True Cache services. And you only start the True Cache services on the True Cache instances. Because it's the database services on the database that you need to make sure are started. And they are read-only.  10:46 Lois: Are there some best practices for maximum availability architecture? Bill: Uniform configuration seems to be a popular one. Why? Because I am going to have the both True Caches can be shared. That way, hopefully, I'm getting full usage out of both. And maybe if I have one service going to one, it might be minimally used. Whereas, the other one might be over. Hey, I could use more memory over here.  We'll also recommend use the JDBC 23ai UCP, Universal Connection Pool, for the application. So that can lessen the impact. If one True Cache becomes unavailable, as far as, OK, I need to reroute over here-- benefit of uniform configuration also. Prepopulate the cache. You want to go ahead and run the critical workload for that. If you have a planned outage, and you need to shut down the True Cache, you want to make sure you stop the database application service on that True Cache.  And then, how are you going to design your True Cache? Are you going to partition it? Are you going to have uniform? Which partition option are you going to use? So you can try to design that to help minimize the number of fetches it has to do from the primary database. And the more you can keep in the True Cache, the better the performance is going to be.  12:09 Nikita: What do I need to keep in mind when it comes to managing True Cache? Bill: One thing you might need to do for managing the True Cache is to monitor the True Cache. There's a couple different ways that we can do it. One, you can use the V$ view, the V$TRUE_CACHE view. And, of course, you can always use the Automatic Workload Repository.  12:30 Lois: Bill, we already spoke about this a bit, but can you tell us more about using True Cache in an application? Bill: There's two ways of using True Cache, as we've seen, physical and logical. Physical, it's going to maintain two connections, front one to the primary database and one to the True Cache. The application can decide which connection to use, based off of what it is trying to do.  If it's just reading, long as it's for a service that's configured with True Cache, it can read the True Cache. If it's going to write something, it's going to update, insert, whatever the case might be, it's for the primary database. And you can use any existing client driver as long as you're using the physical connection method. Any programming language will also work.  With the one logical connection method, it uses the application service for the primary database. You're going to use the JDBC Thin driver, starting with 23ai. You can use it and it maintains the connection to the primary database and True Cache. This model only works with Java applications, though. It maintains the physical connections.  We're going to enable the driver connection. And then, we're going to set the read only. We're going to set it to read only, true. Read only, false, whatever the case might be. And the read only mode is false for a connection by default. False is the default. Java applications only.  14:14 Nikita: What are some best practices for load balancing in a uniform configuration? Bill: You have multiple--multiple True Caches. They're going to service the same database application. They're going to cache the same data. It's the listener that's going to distribute the load balances. So the listener will automatically distribute and load each session to each cache. It will do it randomly and it will do it based off a load. Where can it configure? Where can it send for the best performance.  To route the request to the best performing True Cache, you want to make sure that you are using the same listener. So that remote listener parameter should point to the same listener, which is also the primary database listener. Single instance primary database local listener or scan listener, whichever one you're using, points to the primary. For the application for the JDBC URL, should point to the primary database.  You'll remember that Thin driver is going to create that logical connection, and it's going to create the physical connection to the primary database into each True Cache. To simplify things and possibly avoid connection issues, you might consider using the LISTENER_NETWORK, so the initialization parameter instead of specifying the remote and local listener separately. Because with the local--with the listener networks, all listeners within the same network name will cross register.  15:44 Lois: Before we wrap up, are there any complementary features that you would recommend using alongside True Cache to further enhance performance or simplify management? Bill: There are features that can complement True Cache-- the server-side result set cache.  So you can create--you can go ahead and create the result set that's part of the library cache set aside, a portion of that. You're going to go in, you're going to configure what objects will use that. You can still use that even with True Cache.  There's also the KEEP Buffer Pool that can be used. It's a separate pool that you set aside as part of the buffer cache. And you want to make sure you size it so the object that you want to keep in memory in the buffer cache that you size it appropriately. But again, some configuration, you configure the key pool, plus also you go in and alter the objects to use it.  And then lastly, there's the database smart flash cache. So again, if your data doesn't fit into memory, you can expand the capacity of by adding flash devices. When you configure the flash cache, if you are using transparent data encryption data, the local flash devices is not supported. So if it's TD encrypted on the primary database, it's going to stay in the buffer cache of the primary database.  17:11 Nikita: Ok! I think we can close the episode with that. Thank you, once again, for joining us, Bill.  Lois: Yes thanks! We're learning so much from you. To learn more about what we discussed today, including the various configuration options that are available, visit mylearn.oracle.com and search for the Oracle Database 23ai New Features for Administrators course. Join us next week for a discussion on some more Oracle Database 23ai new features. Until then, this is Lois Houston… Nikita: And Nikita Abraham signing off! 17:46 That's all for this episode of the Oracle University Podcast. If you enjoyed listening, please click Subscribe to get all the latest episodes. We'd also love it if you would take a moment to rate and review us on your podcast app. See you again on the next episode of the Oracle University Podcast.

To kick off season two of Grafana's Big Tent podcast, our host Mat Ryer is back and he's bringing along some heavy hitters! Get ready for a deep dive into the world of caching with Memcached maintainer Alan Kasindorf (aka dormando), along with caching aficionados Danny Kopping and Ed Welch. They'll discuss CPU-level to application-level caching and share strategies that supercharge performance, especially in high-traffic, distributed systems like Grafana Loki. 

Luciano and Eoin chat about Luciano's experience attending AWS re:Invent 2023 in Las Vegas for the first time. They talk about the massive scale of the event, logistical challenges getting around between venues, highlights from the keynotes and announcements, and tips for networking and getting the most out of re:Invent. Luciano shares his perspective on the AI focus, meeting people in real life after connecting online, rookie mistakes to avoid, and why re:Invent is worth the investment for anyone working in the AWS space. AWS Bites is brought to you by fourTheorem, an Advanced AWS Partner. If you are moving to AWS or need a partner to help you go faster, check us out at fourtheorem.com ! In this episode, we mentioned the following resources. - Amazon Q: https://aws.amazon.com/blogs/aws/introducing-amazon-q-a-new-generative-ai-powered-assistant-preview/ - Efi Merdler-Kravitz's talk on "Rustifying serverless" with AWS Lambda (YouTube): https://www.youtube.com/watch?v=Mdh_2PXe9i8 - ElastiCache Serverless for Redis and Memcached: https://aws.amazon.com/blogs/aws/amazon-elasticache-serverless-for-redis-and-memcached-now-generally-available/ - Throughput increase and dead letter queue redrive for SQS FIFO: https://aws.amazon.com/blogs/aws/announcing-throughput-increase-and-dead-letter-queue-redrive-support-for-amazon-sqs-fifo-queues/ - Step Functions Workflow Studio in AWS Application Composer: https://aws.amazon.com/blogs/aws/aws-step-functions-workflow-studio-is-now-available-in-aws-application-composer/ - Lambda scales 12x faster: https://aws.amazon.com/blogs/aws/aws-lambda-functions-now-scale-12-times-faster-when-handling-high-volume-requests/ - Step Function redrive from a failed state: https://aws.amazon.com/blogs/compute/introducing-aws-step-functions-redrive-a-new-way-to-restart-workflows/ Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X, formerly Twitter: - https://twitter.com/eoins - https://twitter.com/loige #aws #reinvent2023 #reinvent #networkingevents

AWS Morning Brief for the week of August 28, 2023, with Corey Quinn. Links: Amazon Aurora Global Database introduces Global Database Failover Amazon ElastiCache for Memcached simplifies creating new clusters in the AWS Management Console Improvements to multi-account management for Amazon GuardDuty AWS Certificate Manager introduces Enterprise Controls to help govern certificate issuance AWS Cost Explorer announces support for AWS Billing Conductor AWS Microservice Extractor now supports visualizing very large enterprise applications AWS re:Post launches an enhanced search experience Announcing AWS ROSA console support for the ROSA with hosted control planes preview EC2 Hibernate now supports Amazon EC2 M7i and M7i-flex instances Manage Cost Allocation Tags with Last-Updated and Last-Used timestamps  Protecting an AWS Lambda function URL with Amazon CloudFront and Lambda@Edge  Choose AWS Graviton and cloud storage for your Ethereum nodes infrastructure on AWS  How Amazon Finance Technologies built an event-driven and scalable remittance service using Amazon DynamoDB Upgrade from Amazon Aurora Serverless v1 to v2 with minimal downtime Next Big Things for Retail – Generative AI leads the pack but isn't alone Explain medical decisions in clinical settings using Amazon SageMaker Clarify Build a serverless store finder site using Amazon Location Service Configuring client IP address preservation with a Network Load Balancer in AWS Global Accelerator  How to use pulse-level control on OQC's superconducting quantum computer AWS Digital Sovereignty Pledge: Announcing new dedicated infrastructure options

Neste conteúdo falaremos sobre algumas estratégias muito interessantes para fazer uso de cache, e com elas ARQ a sua solução de arquitetura de software será muito mais acertada, afinal trabalhar com cache não é simplemente adotar o uso de redis ou memcached.

Twitter is a social media platform that does some incredibly complex stuff when it comes to distributed systems engineering to keep the website up and running. Twitter has open sourced a lot of projects for others to use. Twitter created a fork of Memcached called Twemcache and also a fork of Redis to handle the The post Caching at Twitter with Yao Yue appeared first on Software Engineering Daily.

Twitter is a social media platform that does some incredibly complex stuff when it comes to distributed systems engineering to keep the website up and running. Twitter has open sourced a lot of projects for others to use. Twitter created a fork of Memcached called Twemcache and also a fork of Redis to handle the The post Caching at Twitter with Yao Yue appeared first on Software Engineering Daily.

Redis, best known as a data cache or real-time data platform, is evolving into much more, Tim Hall, chief of product at the company told The New Stack in a recent TNS Makers podcast. Redis is an in-memory database or memory-first database, which means the data lands there and people are using us for both caching and persistence. However, these days, the company has a number of flexible data models, but one of the brand promises of Redis is developers can store the data as they're working with it. So as opposed to a SQL database where you might have to turn your data structures into columns and tables, you can actually store the data structures that you're working with directly into Redis, Hall said.  Primary Database? “About 40% of our customers today are using us as a primary database technology,” he said. “That may surprise some people if you're sort of a classic Redis user and you knew us from in-memory caching, you probably didn't realize we added a variety of mechanisms for persistence over the years.” Meanwhile, to store the data, Redis does store it on disk, sort of behind the scenes while keeping a copy in memory. So if there's any sort of failure, Redis can recover the data off of disk and replay it into memory and get you back up and running. That's a mechanism that has been around about half a decade now. Yet, Redis is playing what Hall called the ‘long game', particularly in terms of continuing to reach out to developers and showing them what the latest capabilities are. “If you look at the top 10 databases on the planet, they've all moved into the multimodal category. And Redis is no different from that perspective” Hall said. “So if you look at Oracle it was traditionally a relational database, Mongo is traditionally JSON documents store only, and obviously Redis is a key-value store. We've all moved down the field now. Now, why would we do that? We're all looking to simplify the developer's world, right?” Yet, each vendor is really trying to leverage their core differentiation and expand out from there. And the good news for Redis is speed is its core differentiation. “Why would you want a slow data platform? You don't, Hall said. “So the more that we can offer those extended capabilities for working with things like JSON, or we just launched a data structure called t-digest, that people can use along and we've had support for Bloom filter, which is a probabilistic data structure like all of these things, we kind of expand our footprint, we're saying if you need speed, and reducing latency, and having high interactivity is your goal Redis should be your starting point. If you want some esoteric edge case functionality where you need to manipulate JSON in some very strange way, you probably should go with Mongo. I probably won't support that for a long time. But if you're just working with the basic data structures, you need to be able to query, you need to be able to update your JSON document. Those straightforward use cases we support very, very well, and we support them at speed and scale.” Customer View As a Redis customer, Alain Russell, CEO at Blackpepper, a digital e-commerce agency in Auckland, New Zealand, said his firm has undergone the same transition. “We started off as a Redis as a cache, that helped us speed up traditional data that was slower than we wanted it,” he said. “And then we went down a cloud path a couple of years ago. Part of that migration included us becoming, you know, what's deemed as ‘cloud native.' And we started using all of these different data stores and data structures and dealing with all of them is actually complicated. You know, and from a developer perspective, it can be a bit painful.” So, Blackpepper started looking for how to make things simpler, but also keep their platform very fast and they looked at the Redis Stack. “And honestly, it filled all of our needs in one platform. And we're kind of in this path at the moment, we were using the basics of it. And we're very early on in our journey, right? We're still learning how things work and how to use it properly. But we also have a big list of things that we're using other data stores for traditional data, and working out, okay, this will be something that we will migrate to, you know, because we use persistent heavily now, in Redis.” Twenty-year-old Blackpepper works with predominantly traditional retailers and helps them in their omni-channel journey. Commercial vs. Open Source Hall said there are three modes of access to the Redis technology: the Redis open source project, the Redis Stack – which the company recommends that developers start with today -- and then there's Redis Enterprise Edition, which is available as software or in the cloud. “It's the most popular NoSQL database on the planet six years running,” Hall said. “And people love it because of its simplicity.” Meanwhile, it takes effort to maintain both the commercial product and the open source effort. Allen, who has worked at Hortonworks, InfluxData, said “Not every open source company is the same in terms of how you make decisions about what lands in your commercial offering and what lands in open source and where the contributions come from and who's involved.” For instance, “if there was something that somebody wanted to contribute that was going to go against our commercial interest, we probably not would not merge that,” Hall said. Redis was run by project founder Salvatore Sanfilippo, for many, many years, and he was the sole arbiter of what landed and what did not land in Redis itself. Then, over the last couple of years, Redis created a core steering committee. It's made up of one individual from AWS, one individual from Alibaba, and three Redis employees who look after the contributions that are coming in from the Redis open source community members who want to contribute those things. “And then we reconcile what we want from a commercial interest perspective, either upstream, or things that, frankly, may have been commoditized and that we want to push downstream into the open source offering, Hall said. “And so the thing that you're asking about is sort of my core existential challenge all the time, that is figuring out where we're going from a commercial perspective. What do we want to land there first? And how can we create a conveyor belt of commercial opportunity that keeps us in business as a software company, creating differentiation against potential competitors show up? And then over time, making sure that those things that do become commoditized, or maybe are not as differentiating anymore, I want to release those to the open source community. But this upstream/downstream kind of challenge is something that we're constantly working through.” Blackpepper was an open source Redis user initially, but they started a journey where they used Memcached to speed up data. Then they migrated to Redis when they moved to the AWS cloud, Russell said. Listen to the Podcast The Redis TNS Makers podcast goes on to look at the use of AI/ML in the platform, the acquisition of RESP.app, the importance of JSON and RediSearch, and where Redis is headed in the future.

Memcached is an in memory cache with one major feature be a transient cache. Memcached has a very simple design. It was originally designed to help with database load by storing the query result in memory to avoid further querying the database. By default it has no authentication, a simple text protocols, servers don't talk to each other. This video discuss the architecture of the cache, design choices and have some critics of the design choices. I go through a demo at the end using docker, telnet and nodes. Enjoy 0:00 Intro 4:40 What is Memcached? 7:45 Memory management 16:00 LRU 25:17 Threading and Connections 30:40 Read Example 34:30 Write Example 36:17 Write and Read collisions 39:40 Locking 40:30 Distributed Cache 43:30 Memcached with Docker/Telnet/NodeJS 45:00 Spin up a Memcached Docker container and telnet 52:17 Memcached and NodeJS 56:15 Four Memached Servers with NodeJS 01:01:00 Summary Resources https://www.cloudflare.com/learning/ddos/memcached-ddos-attack/ https://holmeshe.me/understanding-memcached-source-code-IV/ https://github.com/memcached/memcached/blob/master/doc/protocol.txt https://docs.oracle.com/cd/E17952_01/mysql-5.6-en/ha-memcached-using-threads.html https://holmeshe.me/understanding-memcached-source-code-I/ https://docs.oracle.com/cd/E17952_01/mysql-5.6-en/ha-memcached-using-memory.html https://support-acquia.force.com/s/article/360005256114-Memcached-in-detail https://www.alibabacloud.com/blog/redis-vs-memcached-in-memory-data-storage-systems_592091 https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final197.pdf https://memcached.org/blog/persistent-memory-2/ https://memcached.org/blog/modern-lru/ --- Support this podcast: https://anchor.fm/hnasr/support

On The Cloud Pod this week, the team discusses the new Madrid region's midday siesta shutdown. Plus: Broadcom acquires VMWare for $61 billion, Azure gets paradigmatic with 5G, and you can now take the 2022 Google-DORA DevOps survey. A big thanks to this week's sponsor, Foghorn Consulting, which provides full-stack cloud solutions with a focus on strategy, planning and execution for enterprises seeking to take advantage of the transformative capabilities of AWS, Google Cloud and Azure. This week's highlights

Neben relationalen Datenbanken gibt es noch eine ganz andere Welt: NoSQL.Doch wofür steht eigentlich NoSQL? Kein SQL? Not Only SQL? Was ist eigentlich die Geschichte hinter dem Hype? Warum wurde diese Art von Datenbanken erfunden? Wofür sind diese gut? Folgen NoSQL Datenbank auch dem ACID-Concept? Was ist Eventual Consistency? Und was sind Neo4J, M3, Cassandra, und Memcached für Datenbanken? Eine Episode voller Buzzwords … Hoffen wir auf ein Bingo.Bonus: Warum Wolfgang keinen Manta fährt und ob Andy bald mit einem Ferrari zum einkaufen fährt.Feedback an stehtisch@engineeringkiosk.dev oder via Twitter an https://twitter.com/EngKioskLinksACID: https://de.wikipedia.org/wiki/ACIDBASE: https://db-engines.com/de/article/BASECAP-Theorem: https://de.wikipedia.org/wiki/CAP-TheoremEventual Consistency: https://de.wikipedia.org/wiki/Konsistenz_(Datenspeicherung)#Verteilte_SystemeMichael Stonebraker / The End of an Architectural Era (It's Time for a Complete Rewrite): http://nms.csail.mit.edu/~stavros/pubs/hstore.pdfMongoDB: https://www.mongodb.com/Presto: https://prestodb.io/SAP HANA: https://www.sap.com/germany/products/hana.htmlRedis: https://redis.io/Neo4J: https://neo4j.com/M3: https://m3db.io/InfluxDB: https://www.influxdata.com/VictoriaMetrics: https://victoriametrics.com/Cassandra: https://cassandra.apache.org/Memcached: https://memcached.org/MySQL: https://www.mysql.com/de/MySQL Memcached Plugin: https://dev.mysql.com/doc/refman/5.6/en/innodb-memcached.htmlSprungmarken(00:00:00) Intro(00:00:53) Wolfgangs Auto, Entlastungspaket in Deutschland(00:03:23) Heutiges Thema: NoSQL Datenbanken und CO2-Einsparung durch Datenbank-Optimierungen(00:07:20) Was ist anders zur Episode 19 (Datenbanken) und ist NoSQL überhaupt noch ein Thema?(00:08:39) Was verstehen wir unter dem Begriff NoSQL und woher kommt es eigentlich?(00:15:58) Tip: Für Side Projects besser vertikal anstatt horizontal skalieren(00:16:50) NoSQL: Speziellere Lösungen mit Fokus auf Einfachheit und Benutzerfreundlichkeit(00:18:38) Braucht man heute noch Datenbank-Administratoren (DBA)?(00:21:13) Der Job des klassischen System-Administrator ist weiterhin relevant(00:23:15) Gibt es wirklich keine Datenbank-Schemas in der NoSQL-Welt?(00:27:23) Schema-Lose Möglichkeit in relationalen Datenbanken und Arbeit in die Datenbank oder Software auslagern(00:30:53) NoSQL hat die ACID-Properties aufgeweicht und warum ACID nachteilig für die Skalierung ist(00:33:28) Das NoSQL BASE Akronym(00:36:15) Der Client muss die Datenbank ordentlich nuzten um ACID-Garantien zu bekommen(00:41:35) Was bedeutet eigentlich NoSQL? Kein SQL? Not Only SQL?(00:43:38) Haupt-Speicher Datenbanken und was SAP damit zu tun hat(00:48:02) Was ist Neo4J für eine Datenbank und welcher Use-Case kann damit abgedeckt werden?(00:50:49) Was ist M3 für eine Datenbank und welcher Use-Case kann damit abgedeckt werden?(00:53:06) Was ist Cassandra für eine Datenbank und welcher Use-Case kann damit abgedeckt werden?(00:54:20) Was ist Memcached für eine Datenbank und welcher Use-Case kann damit abgedeckt werden?(00:58:44) OutroHostsWolfgang Gassler (https://twitter.com/schafele)Andy Grunwald (https://twitter.com/andygrunwald)Engineering Kiosk Podcast: Anfragen an stehtisch@engineeringkiosk.dev oder via Twitter an https://twitter.com/EngKiosk

About VenkatVenkat Venkataramani is CEO and co-founder of Rockset. In his role, Venkat helps organizations build, grow and compete with data by making real-time analytics accessible to developers and data teams everywhere. Prior to founding Rockset in 2016, he was an Engineering Director for the Facebook infrastructure team that managed online data services for 1.5 billion users. These systems scaled 1000x during Venkat's eight years at Facebook, serving five billion queries per second at single-digit millisecond latency and five 9's of reliability. Venkat and his team also created and contributed to many noted data technologies and open-source projects, including Facebook's TAO distributed data store, RocksDB, Memcached, MySQL, MongoRocks, and others. Prior to Facebook, Venkat worked on tools to make the Oracle database easier to manage. He has a master's in computer science from the University of Wisconsin-Madison, and bachelor's in computer science from the National Institute of Technology, Tiruchirappalli.Links Referenced: Company website: https://rockset.com Company blog: https://rockset.com/blog TranscriptAnnouncer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.Corey: This episode is sponsored by our friends at Revelo. Revelo is the Spanish word of the day, and its spelled R-E-V-E-L-O. It means “I reveal.” Now, have you tried to hire an engineer lately? I assure you it is significantly harder than it sounds. One of the things that Revelo has recognized is something I've been talking about for a while, specifically that while talent is evenly distributed, opportunity is absolutely not. They're exposing a new talent pool to, basically, those of us without a presence in Latin America via their platform. It's the largest tech talent marketplace in Latin America with over a million engineers in their network, which includes—but isn't limited to—talent in Mexico, Costa Rica, Brazil, and Argentina. Now, not only do they wind up spreading all of their talent on English ability, as well as you know, their engineering skills, but they go significantly beyond that. Some of the folks on their platform are hands down the most talented engineers that I've ever spoken to. Let's also not forget that Latin America has high time zone overlap with what we have here in the United States, so you can hire full-time remote engineers who share most of the workday as your team. It's an end-to-end talent service, so you can find and hire engineers in Central and South America without having to worry about, frankly, the colossal pain of cross-border payroll and benefits and compliance because Revelo handles all of it. If you're hiring engineers, check out revelo.io/screaming to get 20% off your first three months. That's R-E-V-E-L-O dot I-O slash screaming.Corey: This episode is sponsored in part by LaunchDarkly. Take a look at what it takes to get your code into production. I'm going to just guess that it's awful because it's always awful. No one loves their deployment process. What if launching new features didn't require you to do a full-on code and possibly infrastructure deploy? What if you could test on a small subset of users and then roll it back immediately if results aren't what you expect? LaunchDarkly does exactly this. To learn more, visit launchdarkly.com and tell them Corey sent you, and watch for the wince.Corey: Welcome to Screaming in the Cloud. I'm Corey Quinn. Today's promoted guest episode is one of those questions I really like to ask because it can often come across as incredibly, well, direct, which is one of the things I love doing. In this case, the question that I am asking is, when you look around at the list of colossal blunders that people make in the course of careers in technology and the rest, it's one of the most common is, “Oh, yeah. I don't like the way that this thing works, so I'm going to build my own database.” That is the siren call to engineers, and it is often the prelude to horrifying disasters. Today, my guest is Venkat Venkataramani, co-founder and CEO at Rockset. Venkat, thank you for joining me.Venkat: Thanks for having me, Corey. It's a pleasure to be here.Corey: So, it is easy for me to sit here in my beautiful ivory tower that is crumbling down around me and use my favorite slash the best database imaginable, which is TXT records shoved into Route 53. Now, there are certainly better databases than that for most use cases. Almost anything really, to be honest with you, because that is a terrifying pattern; good joke, terrible practice. What is Rockset as we look at the broad landscape of things that store data?Venkat: Rockset is a real-time analytics platform built for the cloud. Let me break that down a little bit, right? I think it's a very good question when you say does the world really need another database? Don't we have enough already? SQL databases, NoSQL databases, warehouses, and lake houses now.So, if you really break it down, the first digital transformation that happened in the '80s was when people actually retired pen and paper records and started using a relational database to actually manage their business records and what have you instead of ledgers and books and what have you. And that was the first digital transformation. That was—and Oracle called the rows in a table ‘records' for a reason. They're called records to this date. And then, you know, 20 years later, when all businesses were doing system of record and transactions and transactional databases, then analytics was born, right?This was, like, the whole reason why I wanted to make better data-driven business decisions, and BI was born, warehouses and data lakes started becoming more and more mainstream. And there was really a second category of database management systems because the first category it was very good at to be a system of record, but not really good at complex analytics that businesses are asking to be able to guide their decisions. Fast-forward 20 years from then, the nature of applications are changing. The world is going from batch to real-time, your data never stops coming, advent of Apache Kafka and technologies like that, 5G, IoTs, data is coming from all sorts of nooks and corners within an enterprise, and now customers in enterprises are acquiring the data in real-time at a scale that the world has never seen before.Now, how do you get analytics out of that? And then if you look at the database market—entire market—there are still only two large categories of databases: OLTP databases for transaction processing, and warehouses and data lakes for batch analytics. Now suddenly, you need the speed of OLTP at the scale of batch, right, in terms of, like, complexity of compute, complexity of storage. So, that is really why we thought the data management space needs that third leg, and we call it real-time analytics platform or real-time analytics processing. And this is where the data never stops coming; the queries never stopped coming.You need the speed and the scale, and it's about time we innovate and solve the problem well because in 2015, 2016, when I was researching for this, every company that was looking to solve build applications that were real-time applications was building a custom Rube Goldberg machine of sorts. And it was insanely complex, it was insanely expensive. Fast-forward now, you can build a real-time application in a matter of hours with the simplicity of the cloud using Rockset.Corey: There's a lot to be said that the way we used to do things after the first transformation and we got into the world of batch processing, where—in the days of punch cards, which was a bit before my time and I believe yours as well—where they would drop them off and then the next day, or two days, they would come back later after the run, they would get the results only to figure out syntax error because you put the wrong card first or something like that. And it was maddening. In time, that got better, but still, nightly runs have become a thing to the point where even now, by default, if you wind up looking at the typical timing of a default Linux install, for example, you see that in the middle of the night is when a bunch of things will rotate when various cleanup jobs get done, et cetera, et cetera. And that seemed like a weird direction to go in. One of the most famous Google April Fools Day jokes was when they put out their white paper on MapReduce.And then Yahoo fell for it hook, line, and sinker, built out Hadoop, and we've been stuck with this idea of performing these big query jobs on top of existing giant piles of data, where ideally, you can measure it with a wall clock; in practice, you often measure the calendar in some cases. And as the world continues to evolve, being able to do streaming processing and understand in real-time what is going on, is unlocking different approaches, at least by all accounts. Do you have an example you can give me of a problem that real-time analytics solves for a customer? Because I can sit here and talk all day about how things might theoretically work, but I have to get out of my Route 53-based ivory tower over here, what are customers seeing?Venkat: That's a great question. And I want one hundred percent agree. I think Google did build MapReduce, and I think it's a very nice continuation of what happened there and what is happening in the world now. And built MapReduce and they quickly realized re-indexing the whole world [laugh] every night, as the size of the internet is exploding is a bad idea. And you know how Google index is now? They do real-time indexing.That is how they index the wor—you know, web. And they look for the changes that are happening in the internet, and they only index the changes. And that is exactly the same principle behind—one of the core principles behind Rockset's real-time analytics platform. So, what is the customer story? So, let me give you one of my favorite ones.So, the world's number one or number two buy now, pay later company, they have hundreds of millions of users, they have 300,000-plus merchants, they operate in, like, maybe 100-plus countries, so many different payment methods, you can imagine the complexity. At any given point in time, some part of the product is broken, well, Apple Pay stopped working in Switzerland for this e-commerce merchant. Oh God, like, we got to first detect that. Forget even debugging and figuring out what happened and having an incident response team. So, what did they do as they scale the number of payments processed in the system across the world—it's, like, in millions; first, it was millions in the day, and there was millions in an hour—so like everybody else, they built a batch-based system.So, they would accumulate all these payment records, and every six hours—so initially, it was a day, and then afterwards, you know, you try to see how far I can push it, and they couldn't push it beyond every six hours. Every six hours, some batch job would come and process through all the payments that happened, have some statistical models to detect, hey, here are some of the things that you might want to double-click and follow up on. And as they were scaling, the batch job that they will kick off every six hours was starting to take more than six hours. So, you can see how the story goes. Now, fast-forward, they came to us and say—it's almost like Rockset has, like, a big red button that says, “Real-time this.”And then they kind of like, “Can you make this real-time? Because not only that we are losing millions of potential revenue dollars in a year because something stops working and we're not processing payments, and we don't find out about that up to, like, three hours later, five hours later, six hours later, but our merchants are also very unhappy. We are also not able to protect our customers' business because that is all we are about.” And so fast-forward, they use Rockset, and simply using SQL now they have all the metrics and statistical computation that they want to do, happens in real-time, that are accurate up to the second. All of their anomaly detectors run every minute and the anomaly detectors take, like, hundreds of milliseconds to run.And so, now they've cut down the business observability, I would say. It's not metrics and machine observability is actually the—you know, they have now business observability in real-time. And that not only actually saves them a lot of potential revenue loss from downtimes, that's also allowing them to build a better product and give their customers a better experience because they are now telling their merchants and their customers that something is not working in some part of your e-commerce footprint before even the customers notice that something is wrong. And that allows them to build a better product and a better customer experience than their competitors. So, this is a very real-world example of why companies and enterprises are moving from batch to real-time.Corey: With the stories that you, and frankly, a lot of other data analytics companies tend to fall back on all the time has been stories of the ones you're telling, where you're talking about the largest buy now, pay later lender, for example. These are companies operating at massive scale who have tremendous existing transaction volume, and they're built out already. That's great, but then I wanted to try to cut to the truth of some of these things. And when I visit your pricing page at Rockset, it doesn't have what I would expect if that were the only use case. And what that would be is, “Great. Call here to conta—open up a sales quote, and we'll talk to you et cetera, et cetera, et cetera.”And the answer then is, “Okay, I know it's going to have at least two commas in it, ideally, not three, but okay, great.” Instead, you have a free tier where it's, “Hey, we'll give you a pile of credits, here's some limits on our free account, et cetera, et cetera.” Great. That is awesome. So, it tells me that there is a use case here for folks who have not already, on some level, made a good show of starting the process of conquering the world.Rather, someone with an idea some evening at two in the morning can wind up diving in and getting started. What is the Twitter for Pets, in my garage, spare-time side project story for using something like Rockset? What problem will I have as I wind up building those things out, when I don't have any user traffic or data yet, but I want to, you know for once in my life, do the smart thing in advance rather than building an impressive tower of technical debt?Venkat: That is the first thing we built, by the way. When we finish our product, the first thing we built was self-service. The first thing we built was a free forever tier, which has certain limits because somebody has to pay the bill, right? And then we also have compute instances that are very, very affordable that cost you, like, approximately $1 a day. And so, we built all of that because real-time analytics is not a need that only, like, the large-scale companies have. And I'll give you a very, very simple example.Let's say you're building a game, it's a mobile game. You can use Amazon DynamoDB and use AWS Lambdas and have a serverless stack and, like, you're really only paying… you're kind of keeping your footprint very, very small, and you're able to build a very lively game and see if it gets [wider 00:12:16], and it's growing. And once it grows, you can have all the big company scaling problems. But in the early days, you're just getting started. Now, if you think about DynamoDB and Lambdas and whatnot, you can build almost every part of the game except probably the leaderboard.So, how do I build a leaderboard when thousands of people are playing and all of their individual gameplays and scores and everything is just another simple record in DynamoDB. It's all serverless. But DynamoDB doesn't give me a SQL SELECT *, order by score, limit 100, distinct by the same player. No, this is a analytical question, and it has to be updated in real-time, otherwise, you really don't have this thing where I just finished playing. I go to the leaderboard, and within a second or two, if it doesn't update, you kind of lose people along the way. So, this is one of actually a very popular use case, when the scale is much smaller, which is, like, Rockset augments NoSQL database like a Dynamo or a Mongo where you can continue to use that for—or even a Postgres or MySQL for that case where you can use that as your system of record and keep it small, but cover all of your compute-heavy and analytical parts of your application with Rockset.So, it's almost like kind of a CQRS pattern where you use your OLTP database as your system of record, you connect Rockset to it, and so—Rockset comes in with built-in connectors, by the way, so you don't have to write a single line of code for your inserts and updates and deletes in your transactional database to get reflected in Rockset within one to two seconds. And so now, all of a sudden you have a fully indexed, fast SQL replica of your transactional database that on which you can do all sorts of analytical queries and that's fully isolated with your transactional database. So, this is the pattern that I'm talking about. The mobile leaderboard is an example of that pattern where it comes in very handy. But you can imagine almost everybody building some kind of an application has certain parts of it that is very analytical in nature. And by augmenting your transactional database with Rockset, you can have your cake and eat it too.Corey: One of the challenges I think that at least I've run into when it comes to working with data—and let's be clear, I tend to deal with data in relatively small volumes, mostly. The stuff that's significantly large, like, oh, I don't know, AWS bills from large organizations, the format of those is mostly predefined. When I'm building something out, we're using, I don't know, DynamoDB or being dangerous with SQLite or whatnot, invariably I find that even at small-scale, I paint myself into a corner by data model design or how I wind up structuring access or the rest, and the thing that I'm doing that makes perfect sense today winds up being incredibly challenging to change later. And I still, in production and have a DynamoDB table that has the word ‘test' in its name because of course I do.It's not a great place to find yourself in some cases. And I'm curious as to what you've seen, as you've been building this out and watching customers, especially ones who already had significant datasets as they move to you. Do you have any guidance around how to avoid falling down that particular well?Venkat: I will say a lot of the complexity in this world is by solving the right problems using the wrong tool, or by solving the right problem on the wrong part of the stack. I'll unpack this a little bit, right? So, when your patterns change, your application is getting more complex, it is demanding more things, that doesn't necessarily mean the first part of the application you build—and let's say DynamoDB was your solution for that—was the wrong choice. That is the right choice, but now you're expanded the scope of your application and the demand that you have on your backend transactional database. And now you have to ask the question, now in the expanded scope, which ones are still more of the same category of things on why I chose Dynamo and which ones are actually not at all?And so, instead of going and abusing the GSIs and other really complex and expensive indexing options and whatnot, that Dynamo, you know, has built, and has all sorts of limitations, instead of that, what do I really need and what is the best tool for the job, right? What is the best system for that? And how do I augment? And how do I manage these things? And this goes to the first thing I said, which is, like, this tremendous complexity when you start to build a Rube Goldberg machine of sorts.Okay, now, I'm going to start making changes to Dynamo. Oh, God, like, how do I pick up all of those things and not miss a single record? Now, replicate that to another second system that is going to be search-centric or reporting-centric, and do I have to rethink this once in a while? Do I have to build and manage these pipelines? And suddenly, instead of going from one system to two system, you actually end up going from one system to, like, four different things that with all the pipes and tubes going into the middle.And so, this is what we really observed. And so, when you come in to Rockset and you point us at your DynamoDB table, you don't write a single line of code, and Rockset will automatically scan your Dynamo tables, move that into Rockset, and in real-time, your changes, insert, updates, deletes to Dynamo will be reflected in Rockset. And this is all using Dynamo Streams API, Dynamo Scan API, and whatnot, behind the scenes. And this just gives you an example of if you use the right tool for the job here, when suddenly your application is demanding analytical queries on Dynamo, and you do the right research and find the right tool, your complexity doesn't explode at all, and you can still, again, continue to use Dynamo for what it is very, very good at while augmenting that with a system built for analytics with full-featured SQL and other capabilities that I can talk about, for the parts of your application for which Dynamo is not a good fit. And so, if you use the right tool for the job, you should be in very good place.The other thing is part about this wrong part of the stack. I'll give a very kind of naive example, and then maybe you can extrapolate that to, like, other patterns on how people could—you know, accidental complexities the worst. So, let's just say you need to implement access control on your data. Let's say the best place to implement access control is at the database level, just happens to be that is the right thing. But this database that I picked, doesn't really have role-based access control or what have you, it doesn't really give me all the security features to be able to protect the data the way I want it.So, then what I'm going to do is, I'm going to go look at all the places that is actually having business logic and querying the database and I'm going to put a whole bunch of permission management and roles and privileges, and you can just see how that will be so error-prone, so hard to maintain, and it will be impossible to scale. And this is what is the worst form of accidental complexity because if you had just looked at it that one week or two weeks, how do I get something out, or the database I picked doesn't have it, and then the two weeks, you feel like you made some progress by, kind of like, putting some duct tape if conditions on all the access paths. But now, [laugh] you've just painted yourself into a really, really bad corner.And so, this is another variation of the same problem where you end up solving the right problems in the wrong part of the stack, and that just introduces tremendous amount of accidental complexity. And so, I think yeah, both of these are the common pitfalls that I think people make. I think it's easy to avoid them. I would say there's so much research, there's so much content, and if you know how to search for these things, they're available in the internet. It's a beautiful place. [laugh]. But I guess you have to know how to search for these things. But in my experience, these are the two common pitfalls a lot of people fall into and paint themselves in a corner.Corey: Couchbase Capella Database-as-a-Service is flexible, full-featured and fully managed with built in access via key-value, SQL, and full-text search. Flexible JSON documents aligned to your applications and workloads. Build faster with blazing fast in-memory performance and automated replication and scaling while reducing cost. Capella has the best price performance of any fully managed document database. Visit couchbase.com/screaminginthecloud to try Capella today for free and be up and running in three minutes with no credit card required. Couchbase Capella: make your data sing.Corey: A question I have, though, that is an extension is this—and I want to give some flavor to it—but why is there a market for real-time analytics? And what I mean by that is, early on in my tenure of fixing horrifying AWS bills, I saw a giant pile of money being hurled over at effectively a MapReduce cluster for Elastic MapReduce. Great. Okay, well, stream-processing is kind of a thing; what about migrating to that? Well, that was a complete non-starter because it wasn't just the job running on those things; there were downstream jobs, and with their own downstream jobs. There were thousands of business processes tied to that thing.And similarly, the idea of real-time analytics, we don't have any use for that because of, oh I don't know, I only wind up pulling these reports on a once-a-week basis, and that's fine, so what do I need that updated for in real-time if I'm looking at them once a week? In practice, the answer is often something aligned with the, “Well, yeah, but you had a real-time updating dashboard, you would find that more useful than those reports.” But people's expectations and business processes have shaped themselves around constraints that now can be removed, but how do you get them to see that? How do you get them to buy in on that? And then how do you untangle that enormous pile of previous constraint into something that leverages the technology that's now available for a brighter future?Venkat: I think [unintelligible 00:21:40] a really good question, who are the people moving to real-time analytics? What do they see? And why can they do it with other tech? Like, you know, as you say… EMR, you know, it's just MapReduce; can't I just run it in sort of every twenty-four hours, every six hours, every hour? How about every five minutes? It doesn't work that way.Corey: How about I spin up a whole bunch of parallel clusters on different timescales so I constantly—Venkat: [laugh].Corey: Have a new report coming in. It's real-time, except—Venkat: Exactly.Corey: You're constantly putting out new ones, but they're just six hours delayed every time.Venkat: Exactly. So, you don't really want to do this. And so, let me unpack it one at a time, right? I mean, we talked about a very good example of a business team which is building business observability at the buy now, pay later company. That's a very clear value-prop on why they want to go from batch to real-time because it saves their company tremendous losses—potential losses—and also allows them to build a better product.So, it could be a marketing operations team looking to get more real-time observability to see what campaigns are working well today and how do I double down and make sure my ad budget for the day is put to good use? I don't have to mention security operations, you know, needing real-time. Don't tell me I got owned three days ago. Tell me—[laugh] somebody is, you know, breaking glass and might be, you know, entering into your house right now. And tell me then and not three days later, you know—Corey: “Yeah, what alert system do you have for security intrusion?” “So, I read the front page of_The New York Times_ every morning and waiting to see my company's name.” Yeah, there probably are better ways to reduce that cycle time.Venkat: Exactly, right. And so, that is really the need, right? Like, I think more and more business teams are saying, “I need operational intelligence and not business intelligence.” Don't make me play Monday morning quarterback.My favorite analogy is it's the middle of the third quarter. I'm six points down. A couple of people, star players in my team and my opponent's team are injured, but there's some in offense, some in defense. What plays do I do and how do I play the game slightly differently to change the outcome of the game and win this game as opposed to losing by six points. So, that I think is kind of really what is driving businesses.You know, I want to be more agile, I want to be more nimble, and take, kind of, being data-driven decision-making to another level. So that, I think, is the real force in play. So, now the real question is, why can they do it already? Because if you go ask a hundred people, “Do you want fast analytics on real-time data or slow analytics on stale data?” How many people are going to say give me slow and stale? Zero, right? Exactly zero people.So, but then why hasn't it happened yet? I think it goes back to the world only has seen two kinds of databases: Transaction processing systems, built for system of record, don't lose my data kind of systems; and then batch analytics, you know, all these warehouses and data lakes. And so, in real-time analytics use cases, the data never stops coming, so you have to actually need a system that is running 24/7. And then what happens is, as soon as you build a real-time dashboard, like this example that you gave, which is, like, I just want all of these dashboards to automatically update all the time, immediately people respond, says, “But I'm not going to be like Clockwork Orange, you know, toothpicks in my eyelids and be staring at this 24/7. Can you do something to alert or detect some anomalies and tap on my shoulder when something off is going on?”And so, now what happens is somebody's actually—a program more than a person—is actually actively monitoring all of these metrics and graphs and doing some analysis, and only bringing this to your attention when you really need to because something is off, right? So, then suddenly what happens is you went from, accumulate all the data and run a batch report to [unintelligible 00:25:16], like, the data never stops coming, the queries never stopped coming, I never stop asking questions; it's just a programmatic way of asking those things. And at that point, you have a data app. This is not a analytics dashboard report anymore. You have a full-fledged application.In fact, that application is harder to build and scale than any application you've ever built before [laugh] because in those situations, again, you don't have this torrent of data coming in all the time and complex analytical questions you're asking on the data 24/7, you know? And so, that I think is really why real-time analytics platform has to be built as almost a third leg. So, this is what we call data apps, which is when your data never stops coming and your queries never stop coming. So, this is really, I think, what is pushing all the expensive EMR clusters or misusing your warehouse, misusing your data lakes. At the end of the day, is what is I think blowing up your Snowflake bills, is what blowing up your warehouse builds because you somehow accidentally use the wrong tool for the job [laugh] going back to the one that we just talked about.You accidentally say, “Oh, God, like, I just need some real-time.” With enough thrust, pigs can fly. Is that a good idea? Probably not, right? And so, I don't want to be building a data app on my warehouse just because I can. You should probably use the best tool for the job, and really use something that was built ground up for it.And I'll give you one technical insight about how real-time analytics platforms are different than warehouses.Corey: Please. I'm here for this.Venkat: Yes. So really, if you think about warehouses and data lakes, I call them storage-optimized systems. I've been building databases all my life, so if I have to really build a database that is for batch analytics, you just break down all of your expenses in terms of let's say, compute and storage. What I'm burning 24/7 is storage. Compute comes and goes when I'm doing a batch data load, or I'm running—an analyst who logs in and tries to run some queries.But what I'm actually burning 24/7 is storage, so I want to compress the heck out of the data, and I want to store it in very cheap media. I want to store it—and I want to make the storage as cheap as possible, so I want to optimize the heck out of the storage use. And I want to make computation on that possible but not efficient. I can shuffle things around and make the analysis possible, but I'm not trying to be compute-efficient. And we just talked about how, as soon as you get into real-time analytics, you very quickly get into the data app business. You're not building a real-time dashboard anymore, you're actually building your application.So, as soon as you get into that, what happens is you start burning both storage and compute 24/7. And we all know, relatively, [laugh] compute and RAM is about a hundred to a thousand times more expensive than storage in the grand scheme of things. And so, if you actually go and look at your Snowflake bill, if you go look at your warehouse bill—BigQuery, no matter what—I bet the computational part of it is about 90 to 95% of the bill and not the storage. And then, if you again, break down, okay, who's spending all the compute, and you'll very quickly narrow down all these real-time-y and data app-y use cases where you can never turn off the compute on your warehouse or your BigQuery, and those are the ones that are blowing up your costs and complexity. And on the Rockset side, we are actually not storage-optimized; we're compute-optimized.So, we index all the data as it comes in. And so, the storage actually goes slightly higher because the, you know, we stored the data and also the indexes of those data automatically, but we usually fold the computational cost to a quarter of what a typical warehouse needs. So, the TCO for our customers goes down by two to four folds, you know? It goes down by half or even to a quarter of what they used to spend. Even though their storage cost goes up in net, that is a very, very small fraction of their spend.And so really, I think, good real-time analytics platforms are all compute-optimized and not storage-optimized, and that is what allows them to be a lot more efficient at being the backend for these data applications.Corey: As someone who spends a lot of time staring into the depths of AWS bills, I think that people also lose sight of the reality that it doesn't matter what you're spending on AWS; it invariably pales in comparison to what you're spending on people to work with these things. The reason to go to cloud is not because it is the cheapest possible way to get computers to do things; it's because it's a capability story. It's about unlocking capacity and capabilities you do not have otherwise. And that dramatically increases your feature velocity and it lets you to achieve things faster, sooner, with better results. And unlocking a capability is always going to be more interesting to a company than saving money on it. When a company cares first, last, and always about just save money, make the bill lower, the end, it's usually a company in decline. Or alternately, something very strange is going on over there.Venkat: I agree with that. One of our favorite customers told us that Rockset took their six-month roadmap and shrunk it to a single afternoon. And their supply chain SaaS backend for heavy construction, 80% of concrete that are being delivered and tracked in North America follows through their platform, and Rockset powers all of their real-time analytics and reporting. And before Rockset, what did they have? They had built a beautiful serverless stack using DynamoDB, even have AWS Lambdas and what-have-you.And why did they have to do all serverless? Because the entire team was two people. [laugh]. And maybe a third person once in a while, they'll get, so 2.5. Brilliant people, like, you know, really pioneers of building an entire data stack on AWS in a serverless fashion; no pipes, no ETL.And then they were like, oh God, finally, I have to do something because my business demands and my customers are demanding real-time reporting on all of these concrete trucks and aggregate trucks delivering stuff. And real-time reporting is the name of the game for them, and so how do I power this? So, I have to build a whole bunch of pipes, deliver it to, like, some Elasticsearch or some kind of like a cluster that I had to keep up in real-time. And this will take me a couple of months, that will take me a couple of months. They came into Rockset on a Thursday, built their MVP over the weekend, and they had the first working version of their product the following Tuesday.So—and then, you know, there was no turning back at that point, not a single line of code was written. You know, you just go and create an account with Rockset, point us at your Dynamo, and then off you go. You know, you can use start using SQL and go start building your real-time application. So again, I think the tremendous value, I think a lot of customers like us, and a lot of customers love us. And if you really ask them what is one thing about Rockset that you really like, I think it'll come back to the same thing, which is, you gave me a lot of time back.What I thought would take six months is now a week. What I thought would be three weeks, we got that in a day. And that allows me to focus on my business. I want to spend more time with my stakeholders, you know, my CPO, my sales teams, and see what they need to grow our business and succeed, and not build yet another data pipeline and have data pipelines and other things coming out of my nose, you know? So, at the end of the day, the simplicity aspects of it is very, very important for real-time analytics because, you know, we can't really realize our vision for real-time being the new default in every enterprise for whenever analytics concern without making it very, very simple and accessible to everybody.And so, that continues to be one of our core thing. And I think you're absolutely right when you say the biggest expense is actually the people and the time and the energy they have to spend. And not having to stand up a huge data ops team that is building and managing all of these things, is probably the number one reason why customers really, really like working with our product.Corey: I want to thank you for taking so much time to talk me through what you're working on these days. If people want to learn more, where's the best place to find you?Venkat: We are Rockset, I'll spell it out for your listeners ROCKSET—rock set—rockset.com. You can go there, you can start a free trial. There is a blog, rockset.com/blog has a prolific blog that is very active. We have all sorts of stories there, and you know engineers talking about how they implemented certain things, to customer case studies.So, if you're really interested in this space, that's one on space to follow and watch. If you're interested in giving this a spin, you know, you can go to rockset.com and start a free trial. If you want to talk to someone, there is, like, a ‘Request Demo' button there; you click it and one of our solutions people or somebody that is more familiar with Rockset would get in touch with you and you can have a conversation with them.Corey: Excellent. And links to that will of course go in the [show notes 00:34:20]. Thank you so much for your time today. I appreciate it.Venkat: Thanks, Corey. It was great.Corey: Venkat Venkataramani, co-founder and CEO at Rockset. I'm Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you've enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you've hated this podcast, please leave a five-star review on your podcast platform of choice along with an insulting crappy comment that I will immediately see show up on my real-time dashboard.Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.Announcer: This has been a HumblePod production. Stay humble.

Carter Morgan and Stephanie Wong host Shopify guests Camilo Lopez and Tai Dickerson this week. Shopify streamlines the online purchasing process so merchants and customers can transact with confidence. Camilo and Tai talk in-depth about Shopify's tech stack and why the choices made are so important to performance. Shopify engineers use a combination of Ruby on Rails, MySQL and Google products like Kubernetes. Resiliency systems like active-active configurations, chat ops for quick solutions, and bot and overload protection are worked in. By leveraging these tools and staying flexible in their resiliency efforts, Shopify is able to adjust to new merchant requirements and teams are able to work efficiently. While tech continues to progress and change, the Shopify culture remains a driving force for advancement, Camilo tells us. The company ideals and axioms help steer the brand and dictate which technologies they'll use to solve new and changing client demands. The 2014 outage shaped the future of these cultural ideals, emphasizing the need for quick action and resiliency components like constraints to ensure system safety. Shopify engineers also built enhanced testing tools like Toxiproxy to simulate poor network conditions and account for potential issues. The 2021 Black Friday Cyber Monday shopping season was Shopify's biggest yet. Camilo and Tai describe how Shopify's resiliency culture and intense prep work made the biggest shopping weekend of the year so successful. By offering educational tools and a support network that values good communication, their company culture continues to grow, and Tai tells us how it's not just the software that should be resilient. Building a resilient, flexible company culture is just as important. Camilo talks about Shopify's recent shift to a completely remote work place and the new challenges and opportunities it presents. Camilo Lopez Camilo has worked at Shopify for more than 10 years, he has been an IC and a manager leading teams that take care of Shopify's scalability and resiliency. Tai Dickerson Tai is a production engineer at Shopify, where she shares her passion for resilience engineering with others via paper discussions and as a leader in Shopify's Resiliency SIG. Cool things of the week Machine images is GA docs New Cloud Logging and Monitoring capabilities Monitoring third-party applications: MariaDB docs Monitoring third-party applications: MySQL docs Monitoring third-party applications: Memcached docs Starting with version 2.8.0, the Ops Agent supports Ubuntu 21.10. For more information, see Linux operating systems docs Interview Shopify site Kubernetes site GKE site Kafka site Redis site Elastic Search site Memcached docs Toxiproxy site Shopify Engineering site Shopify Careers site BFCM Twitter Thread site Shopify engineers deliver on peak performance during Black Friday Cyber Monday 2021 blog Cloud, Load, and Modular Code: What 2022 Looks Like for Shopify blog Terri Haber on Resiliency at Scale site Terri Haber on Enforced Pacing site Bart Jedrocha on Load Testing site Bart Jedrocha on Tooling for Load Testing site Bart Jedrocha on The Future of Load Testing site Ryan McIlmoyl on Code Red site Ryan McIlmoyl on Working with IMOC site Camilo Lopez on The 2014 Outage site Camilo Lopez on Holiday Season Learnings site Tai Dickerson on Doing Things Differently site Tai Dickerson on Learning & Community site What's something cool you're working on? Stephanie is working on season 2 of the Where the Internet Lives podcast. Carter is working on season 2 of VM End to End. Hosts Carter Morgan and Stephanie Wong

Steph tells a cute story about escape artist huskies, and on a technical note, shares a journey in regards to class variables and modules inheritance. Chris talks about how he's starting to pursue analytics and one of the things that he's struggling with that he's always historically struggled with is the idea of historical data. He's also noticed a lack of formalization of certain things and is working with his team to remedy that. This episode is brought to you by ScoutAPM (https://scoutapm.com/bikeshed). Give Scout a try for free today and Scout will donate $5 to the open source project of your choice when you deploy. Mike Burns: How to Skim a Pull Request (https://thoughtbot.com/blog/a-smelly-list) RSpec Documentation (https://rspec.info/documentation/) Don't Let the Internet Dupe You, Event Sourcing is Hard (https://chriskiehl.com/article/event-sourcing-is-hard) Datomic (https://www.datomic.com/) timefora_boolean (https://github.com/calebhearth/time_for_a_boolean) Sentry (https://sentry.io/) Become a Sponsor (https://thoughtbot.com/sponsorship) of The Bike Shed! Transcript: CHRIS: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Chris Toomey. STEPH: And I'm Steph Viccari. CHRIS: And together, we're here to share a bit of what we've learned along the way. So, Steph, it's an entirely new year. What is new in your new year? STEPH: Well, the year is off to an interesting start because we helped rescue a husky. CHRIS: Rescue as in now this is your dog or rescue as in the dog was trapped in a well, and another dog told you about the dog being trapped in a well, and then you helped the trapped? [laughs] Which of those situations are we working with? STEPH: [laughs] I'm really wishing it was the second version [laughs] where there's a dog that tells me about another dog trapped in a well. No, this is a third version where there was a husky that was wandering around the gym that we go to. And so Tim, my husband, called and said that "There's this husky, and he's super sweet, but he seems very lost." And our gym is located near a major road, and so we were worried that he was going to wander about and get hit. So I hopped into our car and took a crate and a leash, and he hopped right in. Clearly, he belonged to somebody; he'd just escaped. So he hops right in, and then we bring him home. But I put him in the backyard because I want to keep him separate from our dog, Utah, just because I don't know this dog, and I want to keep him safe. And I go back inside to grab a few things. I come back out, and the husky is gone. And I'm like, well, shit. [laughs] Now I'm starting to understand why this husky is missing or why this husky seemed lost. So then I started looking for the husky, and Tim comes home. He's helping me look for the husky. And it was one of those awful moments where we live near...it's not a major road, but people tend to speed on it. And the husky and I happen to see each other across the road. And so the husky was like, oh, human friend and starts coming across the road towards me. And there's this large SUV that's also coming from the other direction. I'm like, oh, this is it. This is my nightmare. This is becoming real. This dog is about to get hit. Thankfully, the driver saw the husky and stopped in time, so everything was fine. And the husky just finished trotting across the road to me, brought him in, kept him in the kennel in the garage. We didn't have any backyard adventures after that. The husky then thanked us by howling most of the night. [laughs] So this poor husky has had an adventure. We've had an adventure. And then, around 4:30 in the morning, I go out because I'm checking on the husky and going to let him out. And I'm scrolling on the app called Nextdoor. And I see that someone posted a picture of this exact husky that's like, "Please help me find my dog." And I was like, yes. Because we were going to have to take him to a county shelter or at least go see if he had a chip so then we could return him. But thankfully, we found the owner. I found out the husky's name is Sebastian. And then we had him for a few more hours, and then we had a wonderful husky and human reunion. CHRIS: That story had everything. It had ups; it had downs; it had huskies. It had escape artist huskies, in fact. I have...this is only through Reddit because that's how people learn about things in the world, but huskies are a rather vocal dog breed. So when you say the dog was howling, huskies have a particular way of almost singing, and it kind of sounds like yelling rather than more traditional dog sounds. Was that the experience you had? STEPH: Luckily, it wasn't too bad. His howling was more just; he didn't want to be in the crate. He seems like an indoor dog. So he's like, what am I doing outside in the garage? I should be indoors. And so he wasn't too loud. It was more he was just bemoaning his situation. But our dog Utah could hear him upset in the garage. And so that was also getting Utah upset because he didn't understand why there was a dog so close. And that was what led to the sleepless night because we couldn't get both of them to calm down. Because then, as soon as one of them calm down, the other one would get the other one riled. CHRIS: As it so often happens. STEPH: I'm so grateful that it turned out to be a happy story, though. That part was wonderful. And if we see the husky again, now we know his name is Sebastian and that he'll just come home with us. [chuckles] And we'll know how to return him since he seems to be an escape artist. CHRIS: And we were best friends forever. STEPH: On a more technical note, I have quite the journey to share in regards to class variables and modules inheritance. But before I dive in, I'm curious, what's new in your world? CHRIS: Oh. Well, I'm excited to dig into that story. But I've got two smaller things in my world this week that are top of mind. I don't really have answers on them. I have more questions. One is we're starting to pursue analytics. We want to try and understand our system a little bit better. What is the experience of our users? How are they coming into the system? What are they doing? How long does it take them to do the things that we want them to do? All those sorts of questions you want to be able to answer about your application. And one of the things that I'm struggling with that I've always historically struggled with is the idea of historical data. So data changes over time, and often we actually want to know about those transition points. We want to know about the different states that a user or any record in the system has been in. And I'm finding myself feeling the same pain that I felt many times and starting to think again about the relevant options out there in the world. To give a slightly more pointed example of what we're dealing with, users come in, and then there are a few steps for them to actually sign up for the application. And so their user record or their application, if you will, will go through a couple of different states. So they can be basically approved directly, and now they're an active user of the system, that's one option. But they can also end up in a state where they're pending review. And then eventually, depending on the outcome of that review, whether it's manual or someone intervenes or what have you, then eventually they can transition to either being denied or being accepted. And then they'll again be an active user. And so there's a question now of how many of the users that end up in that pending state end up transitioning into active. And as I looked at the database, I was like, I do not have this information right now. I know their current state. And the logs could tell me all of this. We don't have proper log archiving right now. And I also don't have a system for, like, let me pull down gigabytes of logs and try and sift through that to understand the answer, especially for something domain level like this. But this is one specific example that represents a category of things in my mind. The stuff that I've looked at in this space otherwise is Event Sourcing. So the idea that rather than having a discrete representation of the state of your application, you store every event as an individual log, essentially of like user did X, thing happened, Y occurred. And then, at any given point, you need to know about the state of your system; you just reduce all of those events through some magical reducer that produces the current state. I also very recently read an article called Event sourcing is Hard. So I have that in my head as a counterpoint. This seems like a thing that is non-trivial to do, makes sense for a certain scale. But of course, like anything else, it has its trade-offs. Another thing that I've looked at and never really pursued mostly because it's in a different ecosystem, is Datomic, D-A-T-O-M-I-C, which I think I've mentioned before. But it's a database that actually stores data in this historical format. And so you can ask for the current value, but then you can also ask for what are all the states that this user has been in? And what are the timestamps of those changes? One small thing that we do have that I really like...so this is one example of us; I think leaning into wanting to have more information, higher fidelity information, is often we want to know something like was this ticket paid? Did someone pay for this ticket? And so paid is a BooleanProperty on this ticket record within our system. So the ticket can be held for a little while and eventually gets paid. And now, yes, it has been paid for. It is good. You can use it. But often, we want to know not just that it's paid but when it was paid. And so there's a gem that we are using on the project called timeforaboolean by former thoughtboter, Caleb Hearth. And it does a wonderful job of basically instead of storing a Boolean value in the database, you store a timestamp. But then the Boolean can be inferred. If there is a value, if there's a timestamp for that record in the database, then there are a bunch of helper methods that get introduced that say, like, paid? That's now a method that I can ask, and it will tell us that. But we can also find the paidat, paid_at value. And so we have this higher fidelity data when we need it, but we can also collapse it down to the simpler representation. Because most often, all we need to know is, have they paid for it? Cool, then they're good. They can come into the concert, that sort of thing. But yeah, this is a broader question that I don't have a great answer to. I think Postgres and Rails and just the nature of how we approach these applications pushes us in a certain direction. Another thing I'm exploring is downstream analytic systems. What if I send a bunch of events to them, and they act as a half-event sourcing type thing? But yeah, this is going to be, I think, an open question for me for a while. STEPH: Yeah, you said a lot of really good options. When you're talking about in our ecosystem, we get pushed in one direction or the other that makes me think of the projects that I've been on. Typically, what they'll reach for first is something like a Papertrail. So then, that way, they can check for the historical versions of an object and how it was changed and see who changed it. That's one way to track the logs. I like the idea that if you can outsource it and send all of those events to a logging system and then essentially ask for that data back as you need it. You made me think of a recent project as well where we needed to track the state. So it was a patient matching system. And we really needed to know when a patient match was created or disconnected and then who did that and perhaps for what reason. And to ensure that we had as much information as possible, we took that opportunity to just create a record for it. So we had a patient match record or...I forgot the name of the other one where we created where a patient did not have a match. But we were creating a record every time someone did that. Granted, probably that's not going to happen nearly as often as someone paying for an event or the situations that you're describing. This was ideally infrequently that someone was going to unmatch a patient because it meant that our system had matched people that shouldn't be matched, and then a human had intervened. But yeah, it's interesting the space that you're in. And you listed all the good things that I would have thought of. CHRIS: I think you listed Papertrail, which is one that I hadn't actually thought of yet for this particular instance. This only came up earlier today also. So this is new in my head that I'm really being pushed in this direction. But I think Papertrail could be a good solution for where we're at. But it is one of those where you often don't know the thing you want to know. And I'm terrified of losing data of like; I had the data. I knew it at one point in time, but now I can't reknow it in the future because I didn't write it down. That's one of the things that I just don't want to happen in the world. And so finding those ways of like, how can we architect a system so that we can do the normal, straightforward, boring things most of the time but then when we need to expand out the analytics dimension of the system that we're working on...and trying to thread that needle and find the ideal optimization on both sides is a tricky one. But yeah, I'll definitely take another look at Papertrail and see if that...at a minimum, I think that's a good solution for where we're at now. And then this is going to be a thought that's going to roll around in the back of my head for a while. So if I come up with anything else, perhaps a grander solution, I'll certainly bring that back to The Bike Shed. But yeah, what else is up in your world? I want to hear the story of the class variables. STEPH: Well, it is quite a journey. So I hope you're ready. Specifically, I was pairing with Joël, who was working on fixing a test that had been marked as being skipped for a while. We weren't really sure why. We figured maybe because it's flaky. But then, as Joël had restored that test, he realized it was actually failing consistently. So it was a test that was failing for a reason folks maybe didn't understand, but they decided to cancel or to skip that test. But they didn't actually want to get rid of it because it seemed like a pretty important test based on the description. So Joël saw it and got excited because it seemed very relevant to some of the work he was already doing. So then, he is now investigating why this test is failing consistently. So in this story, we have four main characters: we have a class, two modules, and a class variable. So enter the class stage left. All right, so this class defines a class variable which I have to say is not something I work with very much in Ruby. So class variables kind of felt a bit novel and diving back into like, oh yeah, these are a thing. So the class defines a class variable that's called cache and assigns this variable to an instance of a cache. So then this class includes two modules who we'll call Module A and Module B. And we'll enter them stage right. And both of these models look to see if cache is already set. And if it's not, they also set the cache class variable. So with that information, in our test, we don't want to exercise the real cache just because then if other tests are reading from that cache, which is proving to be a source of flakiness for these tests, then they are overriding each other's expectations, and it's causing some of the tests to flake. So instead, we want to use a fake cache, just like an in-memory cache. So the test and its setup is already overriding. It's setting that class variable to say, hey, I want you to be a fake cache, just be in-memory. However, while executing that test, one of the modules is checking to see if that cache is set, which is being set in our test setup. So test setup sets the value. We're running the test but then in the module, the model checks to see if it's set, and it's suddenly nil instead of using the cache that we had set. So now it's defaulting back to say, "Oh, it's unset. So let me go back and set it to the real cache," which is exactly what we're trying to avoid. So then the question became, if we're setting the class variable in our class, why is it being populated in one of the modules but it's not being populated in the other module? So one of them has it set to the in-memory cache, but the other one does not. So I'm going to gloss over some of the details because this stuff is pretty tangling. But essentially, when the test is running, and it's loading the class, and we are overriding that class variable, it's getting shared with one of the modules because as soon as one of the models does set that class variable, there's a bidirectional link that gets set between the parent class which is the module in this case, and the class itself. And as soon as that module sets the class variable, then they're going to talk to each other, and they're going to reference the same value. However, this only seems to happen for one of the parents. You can't do this for both. So if you have two parents that are trying to share a class variable with the same class, that doesn't work. So that's a particular bug that we were running into. I do have some good news because if anybody is very nervous about the situation that I'm describing, I feel you. The good news is that in Ruby 3, they actually warn when this is happening and have introduced an error. So you don't have this inheritance confusion that can come out of the fact that these parent classes are also trying to share a class variable with this child class. So in Ruby 3, if you are writing a class variable in that class but then you try to overwrite that class variable in the parent of that class or by the module that's being included, then an error is going to be raised. So it's going to warn you if you're creating this bidirectional link between those two class variables and that you shouldn't be overriding the child's ownership of that class variable. Instead, if you're going to use class variables, which, one, is not my cup of tea, but if you're going to use class variables, it should be defined in the parent class, and then it can be shared downstream in the inheritance versus trying to go upstream and then having your ancestors essentially override some of those class variables. So all of that is to say we were on a very interesting journey of understanding how class variables work, how the inheritance works, how that bidirectional link is getting established, and then how Ruby 3 comes in to warn us if something funky is happening. CHRIS: Oh, that is interesting. And I'm now going to catalog that as a piece of information that my brain will retain for roughly the amount of time that we are recording this podcast and then immediately forget. STEPH: As you should. [laughs] CHRIS: It's one of the reasons that I try to avoid inheritance. And I try to avoid class variables as much as possible because of this category of problem, a very subtle bug that you have to try and really hone in. And you have to be very smart to debug this sort of thing. I don't want to be that smart. I want to code in a way that I can be less smart on any given Thursday. That's my goal in life. I will ask one other question, though. So there's just a cache that this class and pair of modules are hanging around with, and then you want to swap it out for in-memory. This sounds remarkably like the Rails cache. Is this cache distinct special? Could it not just be backed by rails.cache, THE cache within the rails context, which can be backed by Memcached, or Redis, or in-memory when you're in tests, or the NullStore, which I think is the default in development is probably how that goes? Is there a particular reason? Is this a special cache? Is there additional behavior that this cache has beyond the normal thing? Or is it just like, at some point, someone's like, oh, I need a cache. I'm just going to use a class variable, that'll be easy, which it definitely is, but then you run into complexities. And caches are one of those hard things to get right. So it's one where I would immediately be like, whoa, whoa, I would love to not make up our own cache here. So I'm wondering, is there a distinct reason, or is it just this happened, and here we are? STEPH: So I think we are using a custom cache that we are pointing to. So it is another service. It's not a Rails cache or an abstraction that we can point to and use. It is a different cache that we are using. And I'm trying to think back to the exact code. But there is a method that essentially checks to say, hey, should I use the real cache? Should I use the in-memory cache? And that is something that we've explored to find a way to make this more global for the test suite because we really want to control this for all the tests. Because it's very easy to not realize in the test that you should avoid using that shared global cache. And so that way, the tests don't interact with each other but instead always use an individualized cache for each test to make sure that it is self-sufficient and independent. But we haven't gotten that far yet in figuring out how we can take a more global approach with this. CHRIS: Gotcha. So I don't know the details. I assume there are reasons here. But just to play this out, if we find ourselves saying we have a reason to have a distinct cache, to have a special cache over here, but it's a cache...and caches fundamentally, that word always will raise my attention. It will be like, okay, this is a place that bugs will come and aggregate. And we need a distinct one that has special behavior as an external service, or that is just something like in... There's a wonderful blog post that Mike Burns wrote at one point that was about...I think it was something like things that will make me look at your pull request in more detail. And I really loved it because it did capsulate all of these like, yeah, there are good reasons to do everything on this list. But if you do any of them, I will look at your pull requests and be like, oh, that's interesting. Why are we doing that, though? Do we have to do that? Are you sure? Are you triple sure we have to do that? And this is definitely one of those things where caches automatically catch my attention. Even if we're using the built-in cache, I'm like, do we need to? Is that a definite thing? And then all the more so when we're using a custom bespoke one. Again, I assume that there are reasons that there's something special that's going on here. Perhaps the caching behavior is distinct from just it's Redis, and we throw data. And if it falls out the backside, that's fine. Maybe you need entirely different behavior here. But it is something that I would poke at a bunch. STEPH: Yeah, you're asking a lot of good questions. I will have to go back and look at some of the code because we spent enough time in Ruby specifics that I didn't pay as much attention to the cache. Because right now, as we are working on these tests, we're trying to fix just the test without changing the application code, one, because that feels like a safer space. And if the test is flaky, we're just trying to change the test first. But some of these tests we're starting to realize I'm not sure we can fix the test without also changing some of the application code, or the way that we do have to fix the test is really an incentive to back up and say maybe now's the time that we look at some of the application code. Because another question that comes to mind is why use a class variable, and does this need to be shared by the class and the modules? And there's a part of me that suspects that maybe some of this logic was extracted to a module, but then it wasn't cleaned up in the other places. And so that's why we still have a reference. And it's essentially then being shared and set and unset and reset in those different places. So I think you ask some good questions, and I have some more questions of my own when we have time to revisit that portion of the test and application. As another example of some of the tests that I've been working on, one of the tests that I...because we have a list, we can usually tell some of the tests that are flaky. So one of the ones that I was investigating was a similar issue where there was a shared resource, and someone had tried to mock it out. So they had taken the time to say, hey, I don't actually want to use that real resource that's over there; instead, I want to just return the scanned value. But instead, they'd accidentally stubbed out a class-level method instead of the instance-level method. And so it was running, but it wasn't actually stubbing anything else since that's the method that's not getting called. So that was just an oversight for that test. So I fixed that test. But I noticed that we were using allow any instance of, so then I did take the time to go through that file and change and move away from the use of allow any instance of. And for folks that are less familiar with allow any instance of, RSpec has some really great docs that talk about how it's very helpful for dealing with legacy code. But essentially, it is a code smell that you're using; allow any instance of because you are saying that my test is or my code is so complex that I can't really mock out the specific instances that I want to and then return specific behavior. So instead, I'm having to use this more global approach to say, hey, any instance of this method, I want you to mock it out versus this very specific instance that I know that I'm working with. But we can include a link in the show notes because there's a nice write-up that talks about some of the reasons that allow any instance of is not recommended. So that's been kind of fun. There's been a little bit of joy to get to refactor away from that and actually stub out a specific instance. Part of the work, too, that I'm noticing as Joël and I are going through these tests is leaving breadcrumbs for other developers as well because they have a very large team. And they're very junior friendly, which is just incredible. I love that so much about this company. And because they do hire a lot of juniors, then it is a tough codebase. It's a fairly old codebase. So as these juniors are coming in, they're seeing a lot of these patterns. And they're propagating these old patterns that aren't necessarily the best patterns to propagate. But they're doing their best, and then they are reusing what they're seeing. So part of the work as we are revising these tests, my hope is that people will see some of these newer patterns and use those instead of following some of the older patterns. CHRIS: I can only imagine that you're writing borderline novels in your pull request descriptions and commit messages there. I do wonder, is there an index of those that you're collecting? So there's like, here's the test remediation examples list, and you're slowly adding to them. This was a weird one with a class variable. And this was a weird one that had flakiness due to waiting or asynchronous behavior. And gathering examples of those, but specifically from the codebase. I could see that being a really useful artifact because I happily traverse through git blame all the time. But I don't know that that's always a thing. And frankly, I have to work for it sometimes. So if there is that list of here are pull requests that specifically did X, Y, and Z, I think that could be super useful. STEPH: Yeah, that's a great idea. And yes, they have some shared team documentation that speaks to specifically flaky tests because they're aware that this is a problem. They are working together to address this. And they have documentation that states ways to avoid flaky tests. If you encounter a flaky test, here are some of the ways that you can triage to find out what's wrong. So as Joël and I have been finding good examples, then we've been contributing to that document. And they also have team meetings. So our plan is to attend some of those meetings and be like, "Hey, this is just some of the stuff that we've seen this week, some of the things that we improved and changed," and share the progress that we're making. Since everyone is aware that there are these developers that are working hard to improve the test suite, but then share that information with the rest of the team so they too can feel...one, they can just see the changes that are taking place. But they too can also benefit and apply those strategies themselves when they see a flaky test. Oh, but you did just remind me of a thing. So one of the tests that I was going through...I'm very intentionally going through and making the smallest change possible. So I will do the gross, ugly fix whatever it is to get something to pass, and then I will commit it. And then I'll think about okay, well, how can I make this better? So essentially, I have the fix, whether it's pretty or not. And then, after that, I start to have other commits that make it prettier. And so, I had a pull request that had four commits that told the story that I was very happy about and progressed along in a more positive direction. And I issued that, and I discovered that Gerrit, when it sees four commits, it split all of them into their own change request. And so, instead of having what I thought would be this nice story, now got split across these four change requests. And I thought, well, that's less helpful. So I ended up squashing two of them, but I still kept three of them because they stood alone, and each told a story. But that's something that I've learned about Gerrit. CHRIS: Always so interesting how our tools shape our work. STEPH: And it made me think back to the listener who asked the question about ensuring that CI runs for each commit. Well, here you go, Gerrit. [chuckles] Gerrit does it for you. It ensures that every commit gets split into its own change request. CHRIS: I mean, as you said earlier, not my cup of tea but... [laughs] STEPH: Yeah, I'm still lukewarm. I'm still discovering Gerrit and how we get along. Mid-roll Ad And now a quick break to hear from today's sponsor, Scout APM. Scout APM is leading-edge application performance monitoring that's designed to help Rails developers quickly find and fix performance issues without having to deal with the headache or overhead of enterprise platform feature bloat. With a developer-centric UI and tracing logic that ties bottlenecks to source code, you can quickly pinpoint and resolve those performance abnormalities like N+1 queries, slow database queries, memory bloat, and much more. Scout's real-time alerting and weekly digest emails let you rest easy knowing Scout's on watch and resolving performance issues before your customers ever see them. Scout has also launched its new error monitoring feature add-on for Python applications. Now you can connect your error reporting and application monitoring data on one platform. See for yourself why developers call Scout their best friend and try our error monitoring and APM free for 14 days; no credit card needed. And as an added-on bonus for Bike Shed listeners, Scout will donate $5 to the open-source project of your choice when you deploy. Learn more at scoutapm.com/bikeshed. That's scoutapm.com/bikeshed. What else is going on in your world? CHRIS: In my world, we keep adding new users to the system. We keep doing more stuff. These are all wonderful things, the direction you certainly want to be heading. But as we're doing that, I've recognized that we had a lack of process and a lack of formalization of certain things. And a lot of the noise of the work was just coming to me because I was the person that everybody knew. I can ask a question; Chris will know the answer, et cetera. And then there were things that we needed to keep an eye on. But because it was everyone's job, it was no one's job. So we've introduced the idea of a point person on the engineering team. So this is a role that will rotate each week. I think you and I have worked on a handful of projects that had something similar to this. There was a team that we worked with that had an ad hoc list, which were just little tasks that needed to be done by developers. So there was one person who would run with that. I've heard it called captain before, the sprint captain. We're not really doing sprints. So for various reasons, that title didn't work for me. But point person is what I went with here. And so the idea is rather than having product management or anyone else in the organization just individually reaching out to developers, we want to try and choke that off, have a single point of communication. And so just today, I introduced into Slack, a group, but it's a group of one person. So @pointdev is technically the handle for this person. It's a group in Slack. And each week, we'll rotate who the members of that team are. And technically, you could add multiple, but the idea is this is just one person. So we'll rotate the person. And what ends up happening is if anyone...say the product manager says, "@pointdev, what's the status on..." blah, blah, blah, that will notify the person who is the point person that week. So that's a nice feature in Slack so that we can condense it down and say rather than asking individuals, ask this alias. We're introducing one layer of abstraction in our communication tools, much like we do in our software. So I'm drafting now the list of like, here's all the stuff that I think this person...because we're trying to push all of the quote, unquote, "other work" the non-product feature development work into this person's purview for a given week. So it's monitor Sentry for any new errors as they come up, triage them, and figure out what we want to do. Ideally, and this is perhaps aspirational, I would like to keep inbox zero in Sentry. I know how you feel about that more generally and perhaps even more specifically within the world of errors, but that's my dream. We're going to see how it goes. STEPH: I don't know if people know I am the opposite of inbox zero. This is the life that I'm living. CHRIS: What about with errors, though? What about something like Sentry? STEPH: I want to say that I would be a better human with my email. But I'm going, to be honest [laughs] and say that I would probably have the same approach where I am not an inbox zero person. I've come to terms with it. I used to really strive and think I needed to change. But I have reached a point of comfort with this is who I am. There are many like us, so shout out to all y'all. CHRIS: Oh yeah, by far the more common approach, I think. So specifically with the errors, I struggle a bit with it because what ends up happening is we are implicitly ignoring the errors. And if we're doing that, I would rather just sit around and have a conversation and be like, let's just explicitly ignore them. There's a button in the UI. We can ignore them. If this is not a real error, we can add it to the list of things that we do not report on. We can ignore that error. We can ignore it for a week and add a card to Trello that has a due date that says, "Hey, we got to work on this." But let's take that implicit indifference to that particular error mode of our application and make it explicit. Let's draw that line in the sand such that when I see a new error pop up, I'm like, oh, that seems like something I should do something about. I really want high signal-to-noise when I'm seeing errors coming. And so I'm willing to work for that. But it is a trade-off, and it does take effort. And it's noisy, especially browser extensions, and whatnot, just fighting the page. Facebook showed up one day. I don't know how Facebook got in there. Someone was browsing our website from within Facebook's browser, which I didn't know was a thing, but they had their own thing. And it fires a bunch of events, and Sentry was just like, let me slurp all of those up. Those seem fun. That was noisy. So we had to turn those off, but we explicitly turned them off. STEPH: I do like the approach that you're taking where it's one person, and then it's a rotating shift because I think that makes it more reasonable for someone's who's like, hey, this is going to be noisy for a week. And then you're going to look through these emails and check all these errors, and then either silence them because you don't think that they're interesting or mute them for now. Or if you're going to convert it into a ticket, set a due date, whatever the triage approach is going to be. But that feels more achievable versus inbox zero for life is just exhausting. But I feel like if you're doing it rotating week by week, that seems like a nice approach and also easier to keep it at inbox zero because that way, you are keeping up with all the errors. Because I agree; otherwise, what's the point of tracking all the errors if you're just going to ignore them? CHRIS: Yeah, definitely the rotating, I think, is critical. I think the other thing that's been critical specifically on the error front is we've had now a handful of meetings where we triage the backlog together, the backlog of errors. So like, what all is coming into Sentry? What's going on? And we go through the process of determining is this a real thing? Should we fix this? Should we ignore it? And we do that together so that it becomes not just one person's intuition about whether or not this is important or not or what the source of it might be but a shared intuition such that now any one of us, when it's our week, can ideally represent the team in that way and be like, never mind, never tell us about this again because it's very easy to silence things in Sentry that you would actually like to know about when they become real. But right now, we have this edge case that is an ignorable version. So trying to get there that's been fun. But yeah, once again, Sentry, that's one of the things on this person's list. There are ad hoc support tickets for our operations team. So anything that needs to happen on a user's behalf that currently needs a developer to console, let's funnel all of those to this one individual, respond to any new questions. So this is where that Slack handle will be useful. Check for any stuck jobs in Sidekiq. So is there anything that's been retrying for a while? Because it probably shouldn't. Maybe one or two retries is cool, but past that, something has gone wrong. And we should either get in there and fix it or just kill that job because it's never going to succeed, which is quite often the case but go in there and keep an eye on those and then look for anything. We're starting to use due dates within Trello, which is currently our project management system. We'll see. Someday we're definitely going to grow out of that. But for now, it's good enough and checking for anything that's overdue or coming up in the next week in terms of due dates and just making sure that we're being responsive to that. And so, I really like the idea of having this be a named set of things and a singular focus for one individual. Because again, that idea of like, if it's everybody's job, it's nobody's job. Or if it's nobody's job, then it's my job, and I don't want it to exclusively be my job. [chuckles] So I'm trying to make it not exclusively my job and to share the knowledge about it and make sure that these are skills that we all have and ideas and et cetera. But also, I would be fine to answer fewer questions in Slack each day. STEPH: I have to admit, as soon as you were telling me that you had established this role, I was quietly congratulating you on helping delegate some of these responsibilities to the team. Because like you said, you are then the person that takes on all these tasks. CHRIS: There's a laziness to that. Like, it's easy for me to just answer the questions. It's harder for me to put up a wall and say, "No, no, we have a process for this." And quite possibly, what's going to happen behind the scenes is that questions are going to come in to whoever is this point person. They're not going to know the answer. They're going to reach out to me, and then that conversation is still going to happen. But even by doing that now, now that person will see that answer, will understand the thinking or the background, the context that I have. And so it's that weird thing of like, it would be so much easier for me to just answer one question. But to answer all the questions, well, I can't do that. And so I'm working to try and do more of the delegation to try and hand things off when they're in a known state and to identify this sort of stuff so that the team broadly can be stronger and better able to support everyone else in the organization. So that's the dream. We'll see how it goes. STEPH: Yeah, I love that approach. I'm also thinking how interesting this role is because I'm imagining a mix between someone who is like the front point person at like an ER. So like, things are coming in, and they're in a tragic state and need help and need to be diagnosed. But at the same time, you mentioned they're going around. They're checking Sidekiq. They're looking at some email errors. So they're also that night shift guard that's walking around with a flashlight just poking in each room. So it seems like a very stressful and low-key role all at the same time, all mixed up into one week. That person probably needs a beer at the end of the week. CHRIS: There is a version of the story in my head that is...I wouldn't say this feels like a failure mode, but I would rather this not have to exist at all. I would rather things to be calmly humming along and not require a dedicated person each week to deal with the noise. I don't think that's realistic, certainly not as early on as we are in our organization. But I do wonder, is this a crutch? Is this something that we should be paying more attention to? And I know in teams that you and I have worked with in the past that has been a recognition of like, this is a crutch. But it's a costly crutch. Like, we're taking an entire...in our case, it's not requiring the entirety of a developer's week. They're able to do this pretty easily and then still get a bunch...like, 75% of their time is still feature work. But we're just choking down who's the person that will be responding to questions when they pop up so that fewer individuals are interrupted? But I have seen organizations where this definitely filled an entire week and spilled out more than. And then there was the recognition of that and the addition of another person that comes along and tries to fix stuff along the way as opposed to just responding. And so I want to make sure this isn't a band-aid but is, in fact, a necessary layer that we then try and shore up, you know, we should have fewer errors. That feels true. Okay, cool. Let's fix the bugs in the app. And these ad hoc things that an admin needs to have done can that be a button in the UI? Can they actually self-serve in those cases? And we're slowly moving towards those. Ideally, fewer jobs get stuck in Sidekiq. And so, my hope is that this isn't a job that gets harder and harder over time. It's a job that potentially, if we're being honest, probably stays about this hard. I don't think it's ever going to be just like, nope, nobody needs to do anything. The app just runs, and it's great. And it never has bugs. But that is a question in my mind as I start to embrace this thing of like one person is dedicated for a week to this. And if right now it's only 25% of their time, okay, that's probably fine. But if suddenly it's 50% of their time or 75% or 100% of their time for that whole week, that becomes too high of a bar in my mind. And I want to keep a close eye on it and make sure it's not trending in that direction. And I will be one of the people on the rotation. So I'll get to be in the trenches. STEPH: I appreciate all the thoughtfulness that you're putting into it. And I'm thinking back on a project where we had a similar rotation because we had an issue Slack channel. And so anytime there was an issue, then it would get posted in there. And before, it was going out to everyone, or there was one particular person that was always picking it up and then trying to delegate it to others as they needed to. But then we started a similar rotation. And one of the key benefits that I found from that is it signaled to the team, hey, this person might get pulled away. They can pick another ticket or two, but we need to give them lower priority tickets because there's a chance that they're going to get pulled away to work on something else. And that's okay, and we're going to plan for it. Versus without this role in mind, then you had people all taking on high priority tickets, but then someone had to be the one that's like, well, I'm going to punt on my high priority and feel stressed about the fact that I've got this other thing to deal with. But then, I didn't actually do the work that I planned for. So I feel like you're helping introduce calmness into the week, even if it is a stressful role. But then there's the goal that this becomes less of a stressful role, and if you see it trending in the opposite direction, then that's something to investigate. But I also feel like triage and communication is such an important part of being a developer that it also feels very relevant upskilling for the whole team to go through. So there's also that benefit of where this approach also empowers the rest of the team to also experiences, build empathy, look for additional fixes, and then also build these important skills. Overall, I really applaud your thoughtfulness. And I think it's a really good idea. And it will be interesting to see which direction that this role trends if it gets easier or if it's getting harder over time. CHRIS: Well, thanks. I appreciate that. And I'll certainly report back as we develop this but hopefully, it stays about where it is. That feels right. And I think I'll probably...that's one of those things that I will monitor. And if I feel it moving in the wrong direction, then step in and try and get it back to this space because this feels like a maintainable reasonable amount. And we shouldn't be fixing every bug and adding every button to the UI. That's just actually not how it works, unfortunately, would love to. That's not true. You shouldn't have every button in the UI. That's so many buttons. But broadly, I hope we can maintain roughly this, and I think identifying it and laying it out now I'm feeling good about having that structure. So yeah, we'll see how it goes. Will report back. But again, thank you for the kind words. With that tour of a bunch of different things, should we wrap up? STEPH: Let's wrap up. CHRIS: The show notes for this episode can be found at bikeshed.fm. STEPH: This show is produced and edited by Mandy Moore. CHRIS: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review on iTunes as it really helps other people find the show. STEPH: If you have any feedback for this or any of our other episodes, you can reach us at @_bikeshed or reach me on Twitter @SViccari. CHRIS: And I'm @christoomey. STEPH: Or you can reach us at hosts@bikeshed.fm via email. CHRIS: Thanks so much for listening to The Bike Shed, and we'll see you next week. All: Byeeeeeeeee!!!!! Announcer: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success.

What do you get when you mix a worm and a hammerhead shark? Also ants. Steph made some cool new discoveries in bug-land. She also talks about deploys versus releases and how her and her team has changed their deploy structure. Two words: feature flags. Chris talks about cookies: cookie sessions, cookie payloads, cookie footprints, cookie storing. Mmm cookies! The convo wraps up with lamenting over truthiness in code. Truthy or falsy? What's your call? Flipper (https://www.flippercloud.io/) Bike Shed - Ask a Question Form (https://docs.google.com/forms/d/e/1FAIpQLSdaFfPYoWmtV3IR3eQRjNz731GJ_a2X6CpZFxKjdPZeztGXKA/viewform) Transcript: STEPH: At the top of my notes for today, I have marauder ants and hammerhead worms. [laughs] CHRIS: I'm sorry, what? I lost you there for...not lost you, but I stopped following. I...what? Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Chris Toomey. STEPH: And I'm Steph Viccari. CHRIS: And together, we're here to share a bit of what we've learned along the way. So, Steph, how's your week going? STEPH: Hey, Chris, it's been a good week. It's been busy, lots has been happening. I learned about a new creature that's in our backyard. They're called hammerhead worms. Have you ever heard of those? CHRIS: I've heard of hammerhead and worms, but not together. The combination is new and novel for me. STEPH: Cool. Cool. So take a hammerhead shark and a worm and combine the two and then you have a hammerhead worm. And it rained really heavily here recently because there's a tropical storm that's making its way up the East Coast. And when I was outside on the porch, I noticed that there were these new worms or worms that I'd never seen before on the back porch. And so I had to Google them to understand because they had the interesting hammer-shaped head. And I found out that they're called hammerhead worms. They're toxic worms that prey on earthworms. And they're basically immortal because if you cut them into multiple pieces, each section can regenerate into a fully developed organism within a few weeks, which is bananas. And a lot of people online highly recommend that you should kill them because they are a toxic predator and they prey on earthworms, which you want in your garden and in your yard. But I didn't, but I learned about them. CHRIS: Wow. That's got some layers there, toxic, intense worms that you can cut in half. And so does their central nervous system just spread throughout their whole body? Where's their brain? How does it...I don't have any real thoughts here. That's just a bunch of stuff, and it's awesome. Thank you for sharing. STEPH: I will warn you. I wouldn't read about hammerhead worms right before bed. Otherwise, you might have some nightmares because the way that they do prey and consume earthworms or other creatures that they prey on is the stuff of horror movies, which I find happens so much in nature, but them especially they fall into that category. So just be aware if you're reading about hammerhead worms and how they consume their food. Now I feel like everybody's going to go read. But as long as you have that warning, I feel safe sending you in that direction. CHRIS: Yeah, first thing in the morning on a very sunny morning, that is the time to do this research. STEPH: Exactly. He got it. I also learned about marauder ants because apparently, this is the day that I'm having. I'm learning about all these creatures. But I won't go into that one, but they're really interesting. And this one's thanks to someone on Twitter who shared, specifically @Rainmaker1973 is their Twitter handle if you want to go see what they shared about marauder ants. So I'll just leave that one for those that are curious. I won't dive into that one because I don't want to take us in the direction of that we're all about worms and ants now. CHRIS: Not all about worms and ants but definitely some. STEPH: But in technical news, I've got some stuff to share, but I was so excited about worms and ants that now I have to figure out which is the thing that I want to share from the week. So there's a couple of interesting things that I'd love to chat about with you, one of them, in particular, is there's been some interesting conversations going on with my client team around deploys versus releases and how we have changed our deploy structure, and then how that has impacted the rest of the team as they are communicating to customers as to what features are available. And there have been some interesting conversations around how to migrate this process forward. So to provide a bit of context, we were previously having very strict, rigid deploys. So we would plan our deploys typically every Tuesday. It was usually once a week. And then we would make sure that everything had been through QA, things had been reviewed and tested. And then we would have one of those more like grand deploys, things are going out. And then hey, if you need to get something into the deploy, let us know; we need to talk about it. So there was just more process and structure to that. And so deploy really mapped to the idea that if we are doing a deploy, then that means all these feature bug fixes are going out, and this is now the time that we can tell customers, "Hey, this new feature is available or this bug that you reported to us has now been fixed." We have since been moving towards a more continuous deployment structure where we're not quite there where we're doing continuous deploy, but we are deploying at least once a day, so it's a lot more frequent. And so this has changed the way that we really map the idea of the work that's being done versus the work that's actually available to customers. Because as we are merging work into the main branch, and then let's say if I'm working on a feature and then I merge that into the main branch and then push it up staging, we have an overnight QA process. So then overnight QA, if they say, "Hey, there's something that's wrong with this feature. It didn't quite meet the required specs," then they can kick that ticket back to me, but that's not true for my code. We could do a revert and take my code out at that point. But at this point, it's in main, and main may have been deployed at that point. So there have been some interesting strategies around how can we safely continue to deploy while we know we often have a 24-hour wait period for QA and to get sign-off on this work? But we want to keep moving forward and then also communicate that just because the code has been deployed doesn't necessarily mean that it's available to customers. There's a lot there. So I'm going to pause and see if you have questions. CHRIS: Well, first, I'm just super excited to talk about this. This is something that's been very much top of mind for me, and it's a direction that I want to be going more and more, so yeah, excited that you're pushing the boundaries on this. I am intrigued. I'm guessing feature flags is the answer about how you're decoupling that and how you're making it so that you've got that separation of deployment and actual availability of the feature. So, yeah, can you talk more about that? STEPH: Definitely. And yes, you're right. We're using feature flags, so we'll use the same scenario. I'm working on a feature, and I want to be able to release it safely, so I'm going to wrap it in a feature flag. And I'll probably wrap it, and maybe it's like a beta feature flag, something to indicate that this is a feature that's going to be available to all, but we don't actually want to turn it on until we know that it's truly ready to be turned on. So then that way, it's hidden, but then we can still merge it into the main branch. We can still have a deploy even if my code hasn't gone through QA at that point, but we know it's still safe to deploy. And then, QA can go to a staging environment; they can test it. And if they say, "No," it's fine because nothing was churned in production. But then, if it gets approved, then we can turn it on, and then we'll have a follow-up to then remove that feature flag. CHRIS: So some follow-on questions. I'm wondering about the architecture of the application. Is this like traditional Rails app rendering HTML on the server, or do you have any more advanced client-side stuff? And then I'm also wondering what you're using for the actual feature flagging, and those will probably inform each other. But what's the story on both of those fronts? STEPH: It's a traditional Rails application. So we're not using any other client-side application. It is Rails and rendering HTML. As for feature flags, so we're not using something traditional. And by traditional, I mean I typically have reached for Flipper in the past for managing feature flags. We're using more of a hand-rolled approach because there's a lot of context there that I don't know is necessarily helpful. But to answer your question, we essentially do have feature flags as columns in the database, and we can just check if they are enabled or disabled. And then that also allows us to easily turn it on, turn it off as well since it's just a database update. CHRIS: Okay, that makes sense. I think the nature of being a Rails application rendering HTML on the server like what you're doing totally makes sense in that context. I think it becomes a lot harder the more complex the architecture of your application is. So if you've got microservices, then suddenly you've probably got to synchronize across some of them, and that sounds like a whole thing. Or even if you have a client-side application, then suddenly you've got to serialize the feature flag stuff across the boundary or somehow expose that, which really does push the issue of we could just render stuff on the server and send it to the client and let that be good enough, then man, is stuff simpler. But unfortunately, that's not the case in a lot of situations. I'm expecting to be introducing feature flags on the app that I'm working on pretty soon. And again, we've got...so it's a Rails server-side thing. So there's going to be plenty of feature flag logic on that side. And then I'll need to do something to serialize it across the boundary and get it onto the client-side without ballooning every payload and adding complexity, and lookups, and whatnot. I think it's doable. Inertia, again, being the core architecture of the application, I think will make this a little bit easier, but I am interested to see what I'm able to pull off and how happy I am with where I get to. Another question that I have for you then are you testing the various flows? So given a Boolean feature flag, you now have two different possible paths for your code to go through. And then there may be even more than Boolean, or you may have feature flags that sort of interact with each other. And how much complexity are you trying to manage and represent in the test suite? STEPH: Yeah, good question, and we are. So we're testing both flows, especially if it's a new feature, then we are testing when the flag is enabled or disabled. One that's been tricky for me is what about a bug fix? Is that something that should be feature flagged? And I think at the surface level, if you're presuming that it needs to go through QA before this is live on production, then the answer is yes, that then you have to feature flag a bug fix, which feels weird. But then the other consideration would be, well, it is a bug fix. And could we find another way to QA this faster or some other approach so that way we don't have to wrap it in a feature flag? And I don't have a great answer for that one because I can see arguments in favor of either approach. Although wrapping everything in a feature flag does feel tedious, it's something that I'm not accustomed to doing. And it's something that then becomes a process for the team to remind each other that, hey, is this wrapped in a feature flag? Or just being mindful of that as part of our process. And it prompted me to think back on the other projects that I've worked on and how did we manage that flow? How did we go from development to staging to QA and then out to production? And one additional consideration with this flow is that we do have an overnight QA team. So in the past, when I've worked with teams, often product managers or even other developers, we would QA each other's work. So then it was a pretty fast turnaround that then you could get something up on staging. Someone could check it out and say, "Yes" or "No." But then I'm also pretty confident most of the teams that I've worked with we have had a distinct staging branch. So we would often merge work into a staging branch, and then deploy that work, and then get it tested. And then, if it passed everything, then we would essentially cherry-pick that work and move it over into production. And I can see there's a lot of arguments against that, but then I have also experienced that and had a really positive experience where we could test everything and not have to worry about going out to production. We didn't have to wrap everything in feature flags, and it just felt really nice to know that everything in the main or production branch, whatever you call your production branch, that everything in there was deployable versus having to go the feature flag route, or the hey, did this go through QA? I don't know. Let me check. Can I include this? Should I cherry-pick some commits into our actual deployment to avoid stuff that hasn't gone through QA? I've been through that dance before too, and that one's not great. CHRIS: I like the way you're framing the different sort of trade-offs that we have there in velocity or deployment speed and ease of iteration versus confidence as things are going out. I have worked with a staging branch before, and I personally did not find it to be valuable. It ended up adding this indirection. Folks had to know how to use Git in a pretty deep way to be comfortable with that just as a starting point. So it already introduced this hurdle of knowledge, and then beyond that, that idea that you have commits going in in a certain order on the staging branch. But then say we verify the functionality of the third commit in that list, and we want to cherry-pick it across to the main branch. Commits don't actually...you can't just take the thing that you had there. That commit existed in the context of all the others. There are subtleties of how history exists in Git. And I would worry about those edge cases where you're taking a piece of work out of the context of the rest of the commits that were around it or before it is, more importantly…that preceded it in the history on the staging branch, and you're now bringing it across to the main branch. Have you now lost something that was meaningful? Ideally, you would get a conflict if it was really bad, but that's more of like a syntactic diff level thing. It's not a functionality-level thing. So personally, I may be overly cautious around this, but I really like as much as possible to have the very boring linear history in Git and do everything I can such that work happens on feature branches and then gets merged in as a fast forward into the main branch or rather the main branch is fast-forward marched into my feature branch such that I'm never working with code that I haven't fully worked with in an integrated way before. But again, even that, as I'm saying that, I have this topological map of Git in my head as I'm saying all of that, and it's complicated. And having any of that complexity leak out into the way we talk about the work is something that I worry about, but maybe I'm worried about a bunch of things that don't matter. Maybe a staging branch is actually fantastic. STEPH: I think you make a lot of good points. Those are a lot of good concerns that come up with...it comes back to the idea that we want to mimic production as much as possible, and we don't want to lose that parity. So then, by having a staging branch, then it feels that we've lost that parity. There could be stuff that's in staging that's not in production. And so staging could be a little bit of this Wild West area, and then that doesn't fully represent then what's going to production. So I certainly understand and agree with those points that you're making. And to speak specifically to the Git challenges, I agree. It does require some more Git knowledge to be able to make that work. Specifically, I think how we handled it on a previous project is where we'd actually cherry-pick our commits into staging and then deploy that. But we always had the PR issued against main. So then merging into main was often a bit easier. But then you're right; things could get out of sync. And the PR is issued against main, so then you still could run into those oddities where then if you are cherry-picking commits in the staging, but then you have your final draft that's going into main. And then what are the differences between those, and what did you lose along the way? And as I say all of that out loud, I definitely understand the Git concerns. And I don't know; I just feel like there's not a great answer then here, which is shocking to me. I've been doing this for a while, and yet here I am feeling like there's not a great answer to this very vital part of our workflow. And I'm surprised even though that we do have a delayed QA process that this still feels like a painful thing to figure out how do we have a continuous deployment workflow even though we do have that delayed QA process? CHRIS: I think somewhat fundamentally your comment there of "I'm surprised that we don't have a good answer to this is," I'm not surprised, I guess, is my reaction. I don't want to go to the software is bad and broken, and we don't know anything end of the spectrum. But I don't feel like we have great answers to a lot of the things about development. I feel like software is more broken than it should be. It costs more to develop. It is difficult. It's hard to create, and maintain, and build over time. And that's just, to get lofty about it, that's what the entire focus of my career is, is trying to solve that problem. But it's a big, hard problem that I do not think is solved, unlike just about any of the fronts. I know how to put stuff in a database and take it back out. And even that, I'm like, oh yeah, but what if the database gets really big? Or what if the database...everything has complexities and edge cases. STEPH: [laughs] CHRIS: And we've joked a handful of times about the catchphrase of The Bike Shed being it depends, and that really feels true, though. I don't know that that's unique to this industry either. I feel like everything in the world is just more complicated the more you look at it, and there aren't clear, good, obvious answers to just about anything in the world, but that's the human condition. I got weirdly philosophical on this, so we should probably round this out. [laughs] STEPH: Well, I can circle us back because I was providing context, and I went a bit into the deep end providing all of that context. So if I circle back to what I wanted to share with you around deploys and releases, there has been that interesting conversation. Now that we have the context, there has been that interesting conversation around originally; we had this very structured deploys, a deploy map to the fact that features were going out to the world. And now we have this concept of a deploy doesn't necessarily mean that's available to customers. It doesn't mean that the code is running. It is more a deploy represents that we have placed a commit. We have placed code on the server. But that doesn't mean that it is accessible to anyone because it's probably hidden behind a feature flag. But from the perspective of the rest of the team that then is communicating these changes out to customers, they still really need to know, okay, when is something actually available to customers? And we kept using this terminology around deploy. And so Joël Quenneville, another thoughtboter who's on this project with me, has done a lot of great, thoughtful work around how can we help them know when something is truly available versus when something is deployed? Because right now, we're using Jira for our ticket issue tracking. And there's a particular screen in Jira that was showing what's being deployed. And from that screen, you can see the status of the ticket, and you would see stuff like in code review, in QA. So, of course, those looking at the tickets are like, hold up, you're deploying something that's in QA? That sounds really dangerous and risky. Why are you doing that? And then we'd have to explain, well, we're deploying it, but it's not actually live or accessible to anybody, but we want to get close to that continuous deploy cycle. So we have shifted to using the terminology of a release. So a deploy is more for the we're putting the code on the server and then release really represents okay, we have now released these features and these bug fixes, and they're now available all with the goal just to make sure that our teams are working well together. But it's been such an interesting conversation around how tickets move, the fact that they can progress linear and then also get moved backwards. But in continuous deployment, things don't go backwards and then making those things align. Typically, things don't go backwards. Technically, yes. CHRIS: History is a directed acyclic graph that only points forward. The arrow of time is very clear on this matter. Yeah, that really does add one more layer of like; what does it mean to actually be out there in the world? I do wonder if giving view-only visibility to the feature flag dashboard and only when it's fully green does someone think that that's deployed? But if you're putting feature flags around everything, there's complexity. And yeah, it's just one more layer to having to manage all of this. And it sounds like you've gotten to a good place, or at least you're evolving in a way that's enjoyable. But yeah, it's complicated. STEPH: Yeah, it definitely feels like we're moving in the right direction and that this will be a better...I want to say workflow, but it really focuses more around vocabulary and some of the changes to our processes and how we surface tickets in Jira. But it's more focused on how we talk about the changes that are getting shipped and when they're available. So, yeah, that's my story. What's new in your world? CHRIS: Well, I very much appreciate your story. In my world, I am in the thick of the MBP initial drive to get something into production, which is one of my favorite times, especially if everyone's in agreement about what exactly do we mean by MVP? Who are the users going to be? What's it going to look like? What's the bar that we're going to maintain? What features can we drop? What can't we drop? When there's a good collaborative sort of everyone rowing in the same direction set of conversations around that, I just love the energy of that time. So I'm happily in that space hacking away on features building as much as I can as quickly as I can. But as part of that, there are a lot of just initial decisions and things that I have to wire up and stuff that I have to change or configure. Thankfully, Rails makes a lot of that not the case. I can just go with what's there and be happy about that. But there is one thing that I did decide to change just today. But it's interesting; I don't think I've actually ever made this change before. I'm sure I've worked on an app that had this configuration, but typically, a Rails app will store the session in a cookie. So there is a signed HTTP only encrypted. I think those are all the things, but it uses a cookie to store that. And the actual data of the session lives in the payload of that cookie. And so, each time there's a request-response lifecycle, the full payload of that cookie is going up and down from the server to the client and then back and forth with all of the requests. And there's a limit; I think it's 4k is the limit on the cookie session. But there are some limitations to cookie sessions as far as I'm coming to understand them; one is the ability to do replay attacks. So if someone gets a hold of that cookie, then unless you rotate the secret key base, which will have some pretty wide-ranging effects on your application, that cookie can be reused in the future because it basically just has like, this is the user's ID. There you go. And there's no way to revoke that other than rotating the secret key base. Additionally, there are just costs of that payload of data, especially if you're putting a non-trivial amount of stuff. Like, if you're getting close to that 4K limit, then you have 4K of overhead, both on the request and the response of your HTTP requests. So especially in apps that are somewhat chatty and making a bunch of Ajax requests or doing different things, that's some weight that you should consider. So all of those mixed together, more so on the security side, I decided to look into it. And I have now switched from a cookie store, and I went all the way to the ActiveRecord database store. So I skipped over...there's a middle option that you can do with Memcached or Redis. We do have Redis in this particular application. We don't have Memcached yet; we probably will at some point. But you can do a memory store, so do Redis and store the session there, but I opted to go all the way to the database. And my understanding of the benefits here are we have a smaller cookie footprint, so smaller overhead on all the requests because now we're only sending the session ID. And then that references the actual payload of data that's stored in the database. We do have the ability now to invalidate sessions, so we can just truncate that table if we just want to sign all the users out and reset the world, which can be useful at times. We also have the ability...if there's any particular user that's like, "I left myself logged in somewhere," we can…well, I actually don't know how to do this now that I say that. I don't know how to log out a specific user because the sessions don't inherently have the user associated with them. You can have an unauthenticated session, which then transitions to be authenticated when someone signs in, and then the user ID gets installed in there. I would love to have these indexed to users such that I could invalidate and have a button on the admin dashboard that says, "Sign out all instances," and that will revoke all of the sessions or actually delete them from the database table now. I think I would have to add some extra instrumentation to do that. So anytime a user signs in via device, we annotate the session records so that it's got a user ID column and then index on that so that we can look them up efficiently. I think that's how that would work, but that's one of those things that I'm like; I think I should think very hard about this before I do it. It has security implications. It's not part of the default package. There's probably a reason for that. I'm going to do that another day. But yeah, overall, it was a pretty easy upgrade. I think I'm happy with it. It feels like one of those things that it's not clear to me why this isn't the default sort of thing where SQLite is often the database that you use just because it's slightly easier to get up and running? But for any application that we're working on, we're like, no, no, no, we're going to go to Postgres for local development and for everything because obviously, that's what we want to do. And I'm wondering if this should be in that space, like yeah, of course, the session should go in the database. There are so many reasons that it's better that way. I'm wondering if there are some edge cases that I'm not thinking about, but overall it seems cool. Have you ever worked with an alternative to the cookie store? STEPH: I'm thinking back to the recent projects that I've worked on. And it's been a while since I've mucked around with session work specifically. And the more recent projects that I've been on, we've used JWTs, or they're pronounced jots, I found out, which is really surprising. I don't know why, but that's a thing. CHRIS: What? STEPH: [laughs] CHRIS: This doesn't feel true. STEPH: It's JWT, but it's pronounced jot, J-O-T. CHRIS: I think I'm just going to not do that. This is a trend I'm not going to get on board with. [chuckles] STEPH: I don't even know if it's a trend. I'm not sure who decreed this into the world. CHRIS: You're familiar with the great internet war around GIF versus JIF, right? I think there's room for different opinions. STEPH: I mean, it's really not a war. There's a correct side. CHRIS: We're on the same side, right? STEPH: [laughs] And this is how The Bike Shed ended. No, this is perfect for The Bike Shed. What am I talking about? CHRIS: This is perfect for The Bike Shed. I'm just going to need to hear you say the word real quick. [chuckles] STEPH: Oh, it's GIF, absolutely, CHRIS: Okay. All right, phew. Steph, I was worried, I was worried. Also, anyone out there that says JIF, it's fine. These things don't really matter. Although I am surprised when you have an acronym that gets turned into...I think it's an initialism, like jot versus JWT. I forget which is which. I think JWT would be the acronym. But jot, that's not even...I'm going to move on and say...[laughs] And so I think that JWTs, which is what I'm going to call them in this context, are, as far as I understand it, an orthogonal, different sort of thing. Like, you can put a JWT in the session, and the session can be stored in a cookie or in the database or wherever. You can also put JWTs...often, they are in local storage, which my understanding is that's a bad idea. That is a security vulnerability waiting to happen from cross-site scripting, I think, is the one that is coming to mind. But I think that's an independent thing where JWT is this signed assertion that you are someone. But it's coming often from an external system versus I'm using devise in this case on a Rails app and so devise is using the warden session, which is signing and encrypting and a bunch of stuff that I'm not thinking about. But it's not using JWTs at the end of the day. Jot, really, huh? STEPH: [laughs] I like how that's the thing that stuck out to you. CHRIS: Of course it is. STEPH: But it's fair because it did the same to me too, so I had to share it. [laughs] CHRIS: This is The Bike Shed, after all. [laughs] STEPH: So, going back to your question, what you've done sounds very reasonable to me, especially because you wanted to address that possibility of a replay attack. So I like the idea. I'm also intrigued by why it's not the default. What's the reasoning there? And I'm trying to think of a reason that it wouldn't be the default. And I don't have a great answer off the top of my head. Granted, it's also been a while since I've been in this space. But yeah, everything that you've done sounds really reasonable. I like it. I also see how being able to sign out a specific user would be really neat. That seems like a really nice feature. I don't know how often that would get used, but that seems like a really nice thing to be able to do to identify a particular user if they submitted and, I don't know, if some scenario came up and someone was like, "Help, please sign me out," then to have that ability. So I'll be intrigued to hear how this advances if you still really like this approach or if you find that you need to change back to using Memcached or the cookie store. CHRIS: Yeah, I'm in that space where as I'm looking at it, I'm like, I only see upside here. I guess there's a tiny bit of extra complexity. You have to watch that database table and set up a regular recurring job to sort of sweep old sessions that haven't been touched in a while because this is sort of like an append-only store. Every time someone signs in anew, they're getting a new session. So over time, this database table is just going to grow and grow and grow. But it's very easy to stay on top of that if you just set up a recurring job that's cleaning them. It's part of the ActiveRecord session store is the name of the gem. It's under the Rails namespace or the Rails GitHub organization. So that seems manageable. Maybe that's the one complexity is it has this sort of runaway trait to it that you have to stay on top of, whereas the cookie-based sessions don't. But yeah, I'm seeing a lot of upside for us, so I'm going to try it. I think it's going to be good. I'm also unfortunately in that space where I think I see all the moving parts as to how I could implement the sign out a user in all of their sessions. But I'm worried that I'm tricking myself there. It's one of those things it's like this feels like it would be built in if it was that straightforward, or it could easily have subtle...it's like, don't invent your own crypto. Like, I think I know how crypto algorithms work. I can just write one real quick. No, don't do that, definitely don't do that. And this one, it seems clear enough, but it's still in the space of crypto security, et cetera, that I just don't want to mess with without really thoroughly convincing myself that I know what I'm talking about. So maybe six months from now, I will have talked myself into it. Or if anyone out there is listening and knows of a good founded, well-thought-out version of yeah, this is totally a thing that we do; here's what it looks like; I would love to hear that. But otherwise, I'll probably just be happy with the ability to wipe everyone's session as necessary. If any one user leaves themselves logged in at a library and needs me to log them out, I'll just log out every user. That's fine. That's a good enough solution. STEPH: Yeah. All of that makes sense. And also, the part that you highlighted around that there is that additional work of where then you have to make sure that you have a rake task that's running to then sign people out since there's that additional lift that you mentioned. But I'm excited to hear what folks have to say if they're using this approach and what they think about it. It is super interesting. CHRIS: Well, yeah, I am very excited about this new development and the management of sessions. And I will let you know if I make any headway on the signing out a user sort of thing. But I think that covers that topic. As an aside, I just wanted to take a quick moment to ask folks out there; we are getting to the bottom of our listener question queue, and we absolutely love getting listener questions. They really help us find novel things to talk about that whenever we start talking about them, it turns out that we have a lot to say. So please do send in any questions that you have. You can send them to hosts@bikeshed.fm. That's an email option. You can tweet at us; we're @bikeshed, or either of us individually. I'm @christoomey. STEPH: And I'm @SViccari. CHRIS: And we also have a Google Form, which we will link in the show notes of this episode. So any of those versions send us questions. It can be about more tech stuff, more process stuff, more team-building, really anything across the spectrum. But we really do love getting the questions in, and definitely helps provide a little bit more structure to the show. So, with that aside, Steph, what else is going on in your world? STEPH: Yeah, I love when we call from our listener questions, for the reason that you highlighted because it often exposes me to different ways of thinking in topics that I hadn't considered before. And you're right; we're often very opinionated souls. [laughs] And along that note, so I have a question for you. The context is another developer, and I ran into a bug. And when we initially looked at the bug, it was one of those there's no way. There's no way the code is in this state. That does not make sense. And then, of course, it's one of those well, the computer says otherwise, so clearly we're wrong. We just can't see how the code is getting to this place. And what was happening is we were setting a value. We were parsing some JSON. We're looking for a value in that JSON, and we're using dig specifically in Ruby. So if it's the JSON or if it's a hash, and then we're doing dig, and then we're going two layers deep. So let's say we're going foo and then bar, and then dig; if it doesn't find those values, instead of erroring, it's just going to return nil. And then we have an or, and then we have a hard-coded string. So it's like, hey, we want to set this attribute to this value. If it's the hash, then give us back that value; if not, it's going to be nil, and then give us this hard-coded string. What we were seeing in the actual data is that we were getting an empty string. And initially, it was one of those; how are we possibly getting an empty string when we gave you a hard-coded string to give us instead? And it's because empty strings are truthy. When we were performing the dig, it was finding both of those values, but that value was set to an empty string. And because that evaluates to truthy, we weren't getting the hard-coded string, and then we were setting it to an empty string, and then that caused some problems. So then my question to you is should we have truthiness in our code? CHRIS: Oh wow. That's a big question. It's also each language I might have a slightly different version of my answer. Yeah, I'm going to have to go sort of across languages to answer. I think in Ruby, I have generally been happy with Ruby's somewhat conservative implementation of truthiness. Yeah, anything that isn't nil false...is that it? Are those the only falsy values? There's maybe one more, but zero is not a falsy value. Empty string is not a falsy value. They're truthy, to name it in the affirmative. And I like that Ruby has a more conservative view of what things are. And so it can have this other surprising edge. I will say that I do reach for present? in Rails, so present? Present with a question mark at the end, that method in Rails, which I pronounce as present, huh? STEPH: Which is delightful, by the way. CHRIS: Well, thank you. That method I reach for often or presence would be the variant in this case where you can presence or and then chain on the thing that you want, and that gets the value. It will basically do the thing that you want here. And so, I do find myself reaching for that, which does imply that maybe Ruby's default truthiness is not quite what I want. And I want a little more permissive truthiness, a little more like, no, empty strings are not truthy. Empty string is an empty value, so it is empty. But yeah, I think I can always convince myself of the other argument when I'm angrily fighting against a bug that I ran into, and I'm surprised by. Like, I've experienced this from both sides many times in my life. I will say in JavaScript, I am constantly surprised by the very, very permissive type coercion that happens where you compare a string and a number, and suddenly they're both strings, and they get smashed together. It's like, wait, how is that ever the thing that I would want? And so JavaScript's version feels like it is definitively foundationally wrong. Ruby's feels like it's maybe a tiny bit conservative, but I like that as a default and then Rails building on top of that. I think I lean towards that most of the time. I will say at the other end of the spectrum, I've worked with Haskell, and Haskell has I want to say it's like a list of chr, like C-H-R list of characters as the canonical way to do strings. I may be mixing this up. It may be actually the string type, but then there's also a text type, and they're slightly different. Maybe it's UTF. I forget what the distinction was, but they both exist, and they are both often found in libraries and in code. And you end up having to constantly convert back and forth. And there are no subtle equivalents between them or any type coercion between them because it's Haskell, and there isn't really any of that. And this was early on. I never got particularly far in Haskell, but I found that so painful and frustrating. It was just like, come on; they're like strings. Please just do the thing. You know what I mean. And Haskell was like, "I do not. And I require you to be ridiculously specific about it." So that was sort of the high end for me of like nope, definitely not that JavaScript of like anything's anything and it's fine. That feels bad. So somewhere in the middle, Ruby feels like it's a happy in the middle. Maybe Rails is actually where I want to land, but I don't know that there is a good answer to this. I don't know that there's a language that's like, we got it. It's this very specific set of things. It's truthy, and these are falsy, and it's perfect every time. Like, I don't think that can happen. STEPH: As an aside, I like how your Haskell voice had the slight air of pretension that really resonated with me. [laughs] CHRIS: I don't know what you're talking about. That doesn't sound familiar to me at all. [laughs] STEPH: I agree. I don't know that anyone has gotten this perfect. But then again, I also haven't tried all the languages that are out there, so I don't feel like that's really a fair statement for me to make either. Specific to the Ruby world, I do think Boolean coercions are a bit nice because then they do make certain checks easier. So if you are working with an if statement, you can say, "If this, and then do that, else, do this." And that feels like a pretty nice common idiomatic flow that we use in Ruby but then still feels like one of those areas that can really bite you. So while having this conversation with some other thoughtboters, Mike Burns provided a succinct approach to this that I think I really like where he said that he likes the use of truthy and falsy for if statements, Booleans for the and statement, and only truthy falsy for Booleans, so no nulls. So Boolean should not have three states is what that last part is highlighting. It should be just true or false. And then if we're working with the double ampersand and in Ruby, that then if you have that type of conditional that you are conveying, then to use a strict Boolean, be more strict and use the methods that you were referring to earlier, like empty and explicitly checking is this an actual...like turn it into a Boolean instead of relying on that that truthy falsy of is it present? Is it an empty string? Does that count? But then, for the if statements, those can be a little more loose. And actually, now that I'm saying it, that first part, I get it. It's convenient, but I still feel like bugs lie down that path. And so, I think I'm still in favor of being more explicit. If I really care if something is true or false, I want to call out explicitly. I expect this to be true or false versus relying on the fact that I know it will evaluate, although I'm sure I do it all the time, just because that's how you often write idiomatic Ruby. So I'm interested in watching my own behavior now to see how often I'm relying on that truthy, falsy behavior, and then see the areas that I can mitigate that just because yeah, that bug is fresh in my mind, and I'd like to prevent those bugs going forward. CHRIS: I really liked that phrase of that bug is fresh. So that bug is going to own a little bit more mindshare than that old bug that's a bit stale in the back of my brain. I will say as you were talking about idiomatic Ruby, I think you're right that the sort of core or idiomatic way to do it would be if the user or whatever to see is the user here, or are they nil? Did we find one, or did we not? That sort of thing is commonly the way it would be done. I almost always write those as if users are not present? I will convert it into that because A, I'm writing Ruby, and I write Ruby because I want it to sound like the human words that I would say. And so I wouldn't say like, "If user," I would say, "If the user is present, then do the thing." And so I write the code to do that, but I also get the different semantics that present? Brings or blank? Is the counterpart, the other side of it. That seems to be the way that I write my code. That's idiomatic me, Ruby, and I don't know how strongly I hold that belief. But that is definitely how I write those, which I find interesting in contrast to what you were saying. The other thing that came to mind as you were saying this is that particular one of an empty string. I kind of want to force empty strings to not be okay, particularly at the database level. So I'll often have null false on a string column, but then I'll find empty strings in there. And I'm like, well, that's not what I meant. I wanted stuff in there. Database, I want you to stop it if I was just putting in an empty string because you're supposed to be the gatekeeper that keeps me honest. And so I do wonder if there is a Postgres extension that we could have similar to the citexts, citext, which is case-insensitive text. So you can say, "Yeah, store this as it is, but whenever you compare it, compare case-insensitively," because an email is an email. Even if I capitalize the third letter, it doesn't make it a different email. I want a non-empty text as a column type that is both null false but also has a check constraint for an empty string and prevents that. And then similarly, the three-state Boolean thing that you're talking about, I will always do null false on a Boolean column because it's a lie if I ever tell myself. I'm like, yeah, but this Boolean could be null, then you've got something else. Then you've got an ADT, which I also can't represent in my database, and that makes me sad. I guess I can enum those, but it's not quite the same because I can't have additional data attached. That's a separate feeling that I have about databases. I'm going down a rabbit hole here. I wish the database would prevent me from putting in empty strings into null, false string columns. I understand that I'm going to have to do some work on my side to make that happen, but that's the world I want to live in. STEPH: I'm trying to think of a name for when you have a Boolean that's also a potential null value. What do you have? You have nullean at that point? CHRIS: Quantum Boolean. STEPH: Quantum Boolean. [laughs] CHRIS: Spooky Boolean. STEPH: The maybe Boolean? CHRIS: Yeah. STEPH: No, that's worse. [laughs] Yeah, I'm with you. And I like the idiomatic Ruby. I think that is something that I would like to do more of where I'm explicitly checking if user instead of just checking for that presence and allowing that to flow through doing the present check and verifying that yes, we do have a user versus allowing that nil to then evaluate to falsy. That's the type of code that I think I'd like to be more strict about writing. But then it's also interesting as I'm formulating these ideas. Is it one of those if I'm reviewing a PR and I see that someone else didn't do it, am I going to advise like, hey, let's actually check or turn this into a true Boolean versus just relying on the truthy and falsy behavior? And probably not. I don't think I'm there yet. And I think this is more in the space that I'm interested in pursuing and seeing how it benefits the code that I'm writing. But I don't think I'm at the state where then I would advocate, at least not loudly, on other PRs that we do it. If it is, it'd be like a small suggestion, but it wouldn't be something that I would necessarily expect someone else to do. CHRIS: Yeah, definitely the same for me on that, although it's a multi-step plan here, a multi-year plan. First, we say it on a podcast, then we say it again on a podcast, then we change all the hearts and minds, then everyone writes the style, then we're all in agreement that this is the thing that we should do. And then it's reasonable to bring up in a pull request, or even then, I still wouldn't want it. Then it's like standard rb or somebody else's job. That's the level of pull request comment that I'm like, really? Come on. Come on. STEPH: This is a grassroots movement for eradicating truthiness and falsyness. I think we're going to need a lot of help to get this going. [laughs] CHRIS: Thankfully, there are the millions of listeners to this show that will carry this torch forward, I assume. STEPH: Millions. Absolutely. CHRIS: I'm rounding roughly a little. STEPH: There are a couple, yeah. [laughs] I'd be far more nervous if I knew we had millions of people listening. CHRIS: I kind of know that people listen. But at the same time, most of the time, I just entirely forget about that, and I feel like we're just having a conversation, which I think is good. But yeah, the idea that actual humans will listen to this in the future is a weird one that just doesn't do good things in my head. So I just let that go. And you and I are just having a chat, and it's great. STEPH: Yeah. I'm with you. And just to reiterate what you were saying earlier, we love getting listener questions. So if there's anything that you'd like to send our way and have us to chat about or something you'd like to share with us, then please do so. On that note, shall we wrap up? CHRIS: Let's wrap up. The show notes for this episode can be found at bikeshed.fm. STEPH: This show is produced and edited by Mandy Moore. CHRIS: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes, as it really helps other folks find the show. STEPH: If you have any feedback for this or any of our other episodes, you can reach us at @bikeshed or reach me on Twitter @SViccari. CHRIS: And I'm @christoomey. STEPH: Or you can reach us at hosts@bikeshed.fm via email. CHRIS: Thanks so much for listening to The Bike Shed, and we'll see you next week. All: Byeeeeeeeeee. Announcer: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success._

Please note: The audio version doesn't include code or commands. Those parts of the post can be seen in the text version. With its ever-growing database and ease of use, Shodan has become one of the most popular tools used by security researchers for gathering IoT intelligence. Shodan provides a great starting point for researchers performing any information gathering task. By being able to filter data by its location, software version, when it was last seen and much more, Shodan can help researchers target specific research points, making their work easier and more efficient. Shodan is great for marketing teams and software vendors too, allowing you to filter out different versions of software running on a server. Furthermore, with the location filters available, one can also find the number of instances running in a certain country, city or district. Shodan employs cybersecurity fingerprinting as a way to find and tag devices, similar to the way human fingerprints identify a person. Various bits of information and services running on an IP address help identify the device running on that IP address. For example, looking up the issuer of an SSL certificate attached to an IP address can often help identify the manufacturer of the device with which the IP is associated. Today we will explore the top Shodan dorks to find sensitive data from IoT connected devices. Most popular Shodan dorks Thanks to its internet scanning capabilities, and with the numerous data points and filters available in Shodan, knowing a few tricks or "dorks"-like the famous Google Dorks can help filter and find relevant results for your IP intelligence research. To begin using Shodan dorks-in a practice known as "Shodan dorking", you'll first need to log in-or create an account and log in to your Shodan account by clicking on the "Login or Register" button on the right-hand side: After which you can log in to or create your Shodan account. Keep in mind this list is presented in random order. There's no Shodan dork more important than any other; they're merely used for different purposes. Let's begin. Databases Databases often hold critical bits of information. When exposed to the public internet, whether for ease of development access or simply due to misconfiguration, can open up a huge security hole. To find MongoDB database servers which have open authentication over the public internet within Shodan, the following search query can be used: MongoDB also has a web management application similar to phpMyAdmin called Mongo Express Web GUI, which we can find with the following query: Similarly, to find My SQ-L-powered databases: To lookup popular Elastic search-powered instances: And to look up Postgre SQ-L databases: Exposed ports Searching for services running on open ports accessible on the public internet, like FTP servers, SS-H servers and others, is possible by using the following queries. For FT-P, querying for pro ftpd, a popular FT-P server: To look for FT-P servers that allow anonymous logins: To query for Open SS-H, a popular SS-H server: For Telnet, querying for port 23: To look up EXIM-powered mail servers on port 25: Memcached, commonly seen on port 11211, has been a major source of UD-P amplification attacks leading to huge DDoS attacks. Services running Memcached available on the public internet are often exploited for these attacks: Jenkins is a popular automated build, deploy and test tool, often the starting point of any software being built for release. It can be found via the following query: DN-S servers DN-S servers with recursion enabled can be a huge source of network threats. To find these servers, one can use the query: Network infrastructure To find devices running a specific version of a Router-OS operating system that powers routers, switches and other networking equipment from the company MikroTik, we use the following search query: This allows us to find those switches, routers and other networking gear running an older an...

Guest Engineer Kenneth Samonte of YoungCTO Rafi Quisumbing Ken is an Amazon AWS Certified Solutions Architect – Professional, AWS Certified DevOps Professional, Certified Google Cloud Platform (GCP) Engineer, and Red Hat Certified System Administrator (RHCSA). His career started as a full-time on-premises Linux/UNIX system administrator. Now, he's working as a Cloud Architect for Samsung Research and Development Philippines. LINKEDIN: https://www.linkedin.com/in/kenneth-s...​ Ken is also a VMware Certified Professional, IBM AIX Administration Certified has an ITIL v3 Certification. I'm a registered Electronics Engineer and a Cisco Certified Network Associate (CCNA). Ken has also written a book dedicated to AWS DevOps Professionals who want to get AWS Certified. Core Technologies: Cloud: Amazon Web Services, Google Cloud Platform Infrastructure: Terraform, Ansible, AWS, GCP, Docker, Kubernetes, VMWare, Linux, OpenStack, Ceph, Resilience, and Disaster Recovery. CI/CD: Spinnaker, Jenkins, Packer, CircleCI Monitoring: Stackdriver, SumoLogic, DataDog, NewRelic, Elasticsearch, Logstash, Kibana, Grafana, Prometheus, Zabbix. Programming: NodeJs, GraphQL, Java, RDBMS, NoSQL, MySQL, Redis, Memcached, SOAP, Microservices Miscellaneous: WordPress, Zimbra, iRedMail, JIRA, Red Hat, Ubuntu If you want to be a guest here, please reach out to me anywhere. Kahit mag comment lang oks na. YoungCTO Shirt: https://store.awsug.ph/shop/product/1...

Redis is quite a versatile NoSQL, key-value database. Or in-memory cache. Or pub/sub broker. With transactions, stored procedures and fast replication. It's quite universal. Anyway, the main use-case for Redis is caching. Internally the whole dataset must fit in memory. Redis can optionally persist data on disk, but all online operations happen entirely in memory. This makes Redis extremely fast. It's often used as an alternative to the widespread Memcached server. Read more: https://256.nurkiewicz.com/31 Get the new episode straight to your mailbox: https://256.nurkiewicz.com/newsletter

It’s the third in our series of chats called the A-Z of WordPress, where we attempt to cover all the major aspects to building and maintaining sites with WordPress. Today, it’s the letter 'C' for Caching (including server side caching, front end caching (plugins) and CDNs. However, we avoid getting too deeply into page load times and scores, which we cover separately on another podcast. So what is the point of caching? What does it do and how might we do it with our WordPress websites. It's certainly a complex subject, and we don't have all the answers, but we do surface some good plugins that you might not have heard of. Check out the podcast...

Sponsored by Linode! pythonbytes.fm/linode Special guest: Ogi Moore Watch the live stream on YouTube. Michael #1: diskcache via Ian Maurer Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python. The cloud-based computing of 2020 puts a premium on memory. Gigabytes of empty space is left on disks as processes vie for memory. Among these processes is Memcached (and sometimes Redis) which is used as a cache. Wouldn't it be nice to leverage empty disk space for caching? Features: Pure-Python Fully Documented Benchmark comparisons (alternatives, Django cache backends) 100% test coverage Hours of stress testing Performance matters Django compatible API Thread-safe and process-safe Supports multiple eviction policies (LRU and LFU included) Keys support "tag" metadata and eviction Developed on Python 3.8 Tested on CPython 3.5, 3.6, 3.7, 3.8 Tested on Linux, Mac OS X, and Windows Tested using Travis CI and AppVeyor CI Brian #2: TOML is 1.0.0 now. What does that mean for Python? Hopefully, some kind of toml parser will make it into Python core. Any Python access to 1.0.0? Mixed Implementations and TOML version support page lists: pytomlpp supports 1.0.0-rc.3, which is a wrapper around C++ tomlplusplus, which does support 1.0.0. Confusing tomlkit supports 1.0.0-rc.1, so that’s promising toml supports 0.5.0, great name. It’d be cool if it would support 1.0.0 What’s different between 0.5.0 and 1.0.0? Unless I’m mistaken, not much: CHANGELOG 1.0.0-rc1 Leading zeroes in exponent parts of floats are permitted. Allow raw tab characters in basic strings and multi-line basic strings. Allow heterogenous values in arrays. Other than that, lots of “Clarify …”, which I’m not sure how those all affect implementation. I’d love to hear more from people who know more about this Ogi #3: pyqtgraph pyqtgraph - plotting library, for when you need fast/interactive plots Uses qt5 (and soon qt6) bindings to generate plots within Qt applications Fills a niche role, want easy mouse interactivity, running locally on a machine Often used in engineering/scientific applications when looking at a lot of data, and wanting interactivity Michael #4: Parler + Python = Insurrection in public via Jim Kring and Mark Little According to Wikipedia: Parler (/ˈpɑːrlər/) is an American alt-tech microblogging and social networking service. Parler has a significant user base of Donald Trump supporters, conservatives, conspiracy theorists, and right-wing extremists. ArsTechnica article send in by Mark Little Ars: Parler’s amateur coding could come back to haunt Capitol Hill rioters Coding mess A key reason for her success: Parler’s site was a mess. Its public API used no authentication. When users deleted their posts, the site failed to remove the content and instead only added a delete flag to it. Oh, and each post carried a numerical ID that was incremented from the ID of the most recently published one. Another amateur mistake was Parler’s failure to scrub geolocations from images and videos posted online. Some 80 terabytes of posts, 1M videos, many already deleted, preserved for posterity. Catalog and Python pointed out by Shaun King. See the catalog (maybe, it’s the ugly side of people). The gist: https://gist.github.com/kylemcdonald/d8884da1a82ef50754ee49e0b6561071 Partially back online with Russian hosting service? Brian #5: Best-of Web Development with Python Suggested by Douglas Nichols Cool list with nice icons Covers Frameworks, HTTP Clients, Servers Auth tools, HTML Processing, URL utilities OpenAPI, GraphQL, Websocket RPC, Serverless, Content Management Web Testing, Web Forms, Markdown Third-party APIs Email, Web Scraping & Crawling, Monitoring Admin UI API Proxies Flask/FastAPI/Pyramid/Django Utilities Nice to see lots of FastAPI projects: fastapi-sqlalchemy - Adds simple SQLAlchemy support to FastAPI. fastapi-plugins - FastAPI framework plugins. fastapi_contrib - Opinionated set of utilities on top of FastAPI. starlette_exporter - Prometheus exporter for Starlette and FastAPI. fastapi-utils - Reusable utilities for FastAPI. fastapi-code-generator - This code generator creates FastAPI app from an.. slowapi - A rate limiter for Starlette and FastAPI. fastapi-versioning - api versioning for fastapi web applications. fastapi-react - Cookiecutter Template for FastAPI + React Projects. Using.. fastapi_cache - FastAPI simple cache. Ogi #6: Assorted Pyjion - https://github.com/tonybaloney/Pyjion a JIT extension for CPython that compiles python code using .NET 5 CLR CuPy - NumPy compatible multi-dimensional array on CUDA, uses _``*array_function_* (enabled with numpy 1.17) code using numpy to operate directly on CuPy arrays see NEP-18 and CuPy docs compatible with other libraries as well Extras: Michael: Trying Firefox + Brave + VPN Python Web Conf 2021 call for talks, due Jan 29, I’ll be speaking! PyCon US 2021 launched call for proposals: December 22, 2020 — Call for proposals opened February 12, 2021 — Proposals are due March 16, 2021 — Notifications will be sent to presenters March 23, 2021 — Deadline for speakers to confirm participation March 30, 2021 — Schedule is publicly released April 28, 2021 — Deadline to submit pre-recorded presentation (tutorials will be live) May 12-13, 2021 — Tutorial days May 15-16, 2021 — Conference days Apple launching Racial Equity and Justice Initiatives with partners across a broad range of industries and backgrounds — from students to teachers, developers to entrepreneurs, and community organizers to justice advocates Brian: PyCascades 2021 schedule https://2021.pycascades.com/program/schedule/ Ogi: Anthony Explains Video Series Learn X in Y minutes Reading Working in Public by Nadia Eghbal - provides some sanity checks for existing maintainers, might be a fantastic perspective for new contributors to open source Joke Tech Support, 2x Working at the help desk? Get the theme song: Here to help song And help by chat: "Running a successful open source project is just Good Will Hunting in reverse, where you start out as a respected genius and end up being a janitor who gets into fights." - Byrne Hobart

Jonas Lejon is an amazing mind in the Swedish security world. A great entrepreneur, hacker, and security-expert! We had the pleasure of talking with him in this episode of Security Headlines. he wanted to specialize in security so he packed his bag and headed over to the capital city to work more in-dept with security. He wanted to go deeper and deeper, so spent his extra hours learning the assembly programming and getting into the low-level brain of the computer system. He managed to land a job working for the Swedish version of NSA. Jonas now runs his own company called "Triop" and has a lot of fun side projects that we dig into. In this episode we also cover: Micro blogging building search engines bloggz dot se Getting over 20K users within a few weeks Twitter in the early days Building Sweden's biggest micro-blogging platform testing in production WordPress Security bug bounties Finding security holes in Zoom writing about encryption and security fuzzing Hacking Bluetooth ISOC-SE the swedish top level domains .se and .nu the internet in Sweden beatboxing pentesting enumerating existing users based on validation time updated, security by default systems network logging Programming leaving python 2 Customizing Kali linux Time-of-check to time-of-use attacks writing exploits ## External links: https://triop.se https://kryptera.se https://web.archive.org/web/20081102073248/http://bloggz.se/ https://web.archive.org/web/20110630210858/http://bloggy.se/ https://en.wikipedia.org/wiki/Memcached https://wpsec.com/ https://utvbloggen.se/ https://se.linkedin.com/in/jonaslejon https://www.youtube.com/channel/UCI49rLPi_Lbbux5eo8ewLKA https://en.wikipedia.org/wiki/Dave_Aitel https://github.com/SofianeHamlaoui/Spike-Fuzzer https://isoc.se/ https://internetstiftelsen.se/en/ https://www.netnod.se/ https://en.wikipedia.org/wiki/Kali_Linux https://en.wikipedia.org/wiki/Arcade_Fire https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use https://github.com/juliocesarfort/public-pentesting-reports https://www.hackerone.com/ https://www.bugcrowd.com/ https://twitter.com/jonasl

Come si può creare una piattaforma globale nel cloud velocemente? Quali tecnologie e servizi abilitano piccoli team di sviluppatori e data scientist ad innovare velocemente con tecnologie open-source come Kubernetes, Elasticsearch, MySQL e Memcached? In questo episodio intervisto Francesco Delfino, cofondatore di Musixmatch, per fargli qualche domanda sulla loro storia e su come hanno iniziato ad utilizzare AWS dieci anni fa. Parleremo della piattaforma di crowdsourcing e analisi linguistica e musicale che hanno costruito, e che oggi fornisce servizi B2B ad aziende come Apple Music, Amazon Music, Shazam, Vevo e Instagram.

Estamos en una nueva temporada esta vez hablando de Google Cloud Platform. En este episodio hablaremos sobre Memorystore. Una base de datos NoSQL, basada en llave-valor, serverless y podras usarlo con Redis o Memcached. Recuerda compartir si crees en la comunidad.

פרק מספר 68 של באמפרס (393 למניין רברס עם פלטפורמה) - רן, אלון ודותן נפגשים שוב ב-8 ביולי 2020 בעיצומו של הגל השני, מקליטים מהבית דרך Zoom . . . ואף על כן - באמפרס: רן, אלון ודותן עם סידרה של קצרצרים על מה שקרה ברשת, מה עניין אותנו, בלוג-פוסטים מעניינים שנתקלנו בהם, Repos מעניינים ב-GitHub ועוד.אז נצלול . . .רן - חברת Microsoft הוציאה לקוד פתוח את התוכנה שנקראת GW-BASIC - מי זוכר מה זה?מדובר בשכלול קל על ה-Basic הרגיל, הכי בסיסיה-GW-BASIC הייתה אחת הגרסאות הכי פופלאריות של Basic - יכול מאוד להיות שאם אתם מכירים Basic, אז אתם מכירים את הגרסא הזו.למעשה, Microsoft גם הוציאו בלוג-פוסט וגם Repo ב-GitHub, ששם נמצא כל ה-Source Code של GW-BASIC(דותן) שאפו על זה שהם ממש שמו היסטוריה אמיתית ב-Git - יש כאן “38 years ago” . . .(רן) כנראה באמת שיחזרו את ההיסטוריה, כי Git לא היה קיים לפני 38 שנים . . .אתם יכולים לגשת לכל קבצי ה-ASM (הלא הם ה-Assembly!) ולקרוא את הפקודות - אשכרה פקודות-מכונה שבאמצעותן נכתב GW-BASICמרתק למי שבקטע - או סתם נוסטלגיה למי שפחות.(דותן) אתם יודעים מה זה אומר? (אלון) שאפשר להתחיל לכתוב ב-BASIC?(דותן) גם - וגם שצריך להתחיל לפתוח להם Pull-Requests . . . למה אין Source folder?! למה אין Make?!(רן) לגמרי - מבחינת איכות כתיבת הקוד . . .(דותן) אין פה Folders בכלל! מחפש איפה להיכנס ואין לאן.(אלון) אני לא יודע האם לפני 38 שנים Windows ידע לעבוד עם Folders - בעצם זה היה עוד בכלל DOS . . .(דותן) כן, יש פה Code of Conduct ו-Contributing . . . תתרום! אה, בעצם - “Please do not send Pull Requests” . . .(רן) למרות שיש פה ושם עדכונים - ראיתי אחד לפני חודשיים, אז זה לא שזה לגמרי הכל כמו לפני 38 שנים, אבל הרוב כן.(דותן) וכולם כל כך ממושמעים - אין כאן אפילו Pull Request אחד שנפתח, לא Closed, לא כלום . . .(רן) כן, טוב - הם הבעלים של הפלטפורמה, בוא לא נשכח . . .סקר של Stack Overflow שהתפרסם לא מזמן - הסקר השנתי שלהם של שנת 2020הם כל שנה מוציאים סקר וזה תמיד מעניין ונחמד לקרוא את מה שהם כותבים.הפעם הדבר הבולט ביותר בעיני הוא שויזואלית - זה מהמם . . . פשוט מעוצב יפה.יש שם גם הרבה תוכן, אבל הדבר הראשון שבולט לעין (כן . . .) זה שזה מעוצב יפה, עם JavaScript כזה אינטראקטיבי וכל מיני גרפים שזזים.על הסקר ענו 65,000 מפתחים מרחבי העולם - אפשר לראות פרטים דמוגרפיים שלהם וכו’.אני לא זוכר איזשהו אייטם ספציפי לגבי שאלות או תשובות מעניינות שראיתי, אבל יש שם המון אינפורמציה - כל אחד ימצא את מה שמעניין שם.יש המון אינפורמציה על טרנדים דמוגרפיים וטרנדים בתעשייה - אם זה טכנולוגיות ודברים כאלהפשוט כיף לראות את זה, ויזואלית זה מאוד יפה, עם הרבה מאוד אינפו-גרפיקות מכל מיני סוגים.אם אתם זוכרים, באחד הפרקים שעברו דיברתי על זה שאני קורא כמה ספרים ובינתיים לא מצאתי משהו מעניין - אז מצאתי ספר טוב שאני כן רוצה להמליץ עליודותן, זוכר? אמרת שברגע שיהיה משהו להמליץ אז נמליץ? אז הנה - ספר שאני עדיין בעיצומו ולא סיימתי לקרוא אותו ונקרא An Introduction to Machine Learning, שזה תחום שאני עוסק בו בזמן האחרון.הורדתי את הספר אונליין, אני קורא אותו כ eBookמה שאני אוהב בספר הזה זה(1) הוא כתוב בשפה מאוד יפה, זאת אומרת - בניגוד לספרים אחרים שקראתי והייתה בהם אנגלית “קצת שבורה ומעצבנת”, כאן זאת באמת שפה יפה שכיף לקרוא ובנוסף (2) יש בו הרבה מאוד תרגילים - בסוף כל פרק - שמאוד עוזרים להפנים את החומר.יש שלושה סוגי תרגילים - סוג אחד הוא “תרגילי חשיבה”; סוג שני הוא “קח נייר ועפרון ותעשה חישוב” וסוג שלישי של כתיבת תוכניות שמממשות Perceptron או מסווג מסוג כזה או אחר - וזה מאוד עוזר להפנים את החומר.אז הספר נקרא An Introduction to Machine Learning, בהוצאת Springer, המחבר הוא Miroslav Kubat - אמריקאי מאוניברסיטת פלורידה (מיאמי)אם אתם בעניין של לעשות איזושהי הכרות עם Machine Learning אז זו היכרות די מעמיקה, אני חייב להגיד.(דותן) עד כמה הוא פרגמטי? או אם לשאול בצורה אחרת - אתה צריך לדעת אלגברה לינארית לפני כן? להיזכר בכל מיני דברים מהאוניברסיטה, או שהוא מאוד פרגמטי?(רן) הוא לא מאוד פרגמטי . . . הוא לא מדבר על ספריות כמו Pandas או TensorFlow, לא מדבר בכלל על כליםהוא מדבר ברמה התיאורטית - אבל התרגילים הם כן פרקטיים, זאת אומרת שצריך ממש לכתוב תוכנהאני את התרגילים האלה כותב ב Clojure מתוך היצר המזוכיסטי שלי . . .אתה כן מקבל איזשהו ניסיון תכנותי - אבל הוא לא פרגמטי כל כך במובן של “להכיר כלים אמיתיים”.מבחינת ידע ורקע - אני חושב שמתימטיקה ברמה של תואר ראשון זה לגמרי מספיק, כנראה שאפילו פחות, אולי אפילו רק השנה הראשונה של התואר הראשון מספיקה; אלגברה לינארית ברמה לא גבוהה מדי, חשבון אינפיטיסימלי או חדו”א (!) ברמה גם לא-מאוד-גבוהה - צריך להבין מה זו נגזרת, מה זה אינטגרל, דברים כאלה . . . שנה ראשונה באוניברסיטה בכל אחד מהמקצועות המדעיים נותנת לכם רקע מספיק בשביל הדברים האלה, עם קצת הבנה בהסתברות וסטטיסטיקה, אולי קצת הבנה בקומבינטוריקה אבל לא הרבה. זהו . . .זה לא ספר קל, אני חייב להגיד (כי עד עכשיו נשמע סבבה) - דורש קריאה איטית ומחשבה, אז גם אם יש לכם את הרקע, זה לא רומן . . . זה משהו שדורש מחשבה והעמקה ובעיקר תרגול.בכל אופן - אני אוהב את הספר. המלצה!(אלון) טוב לדעת . . . אבל אם לא סיימת, עדיין אפשר לעשות לך ספויילרים על מה קורה בסוף! נגלה לך איזו תוכנה אתה כותב בסוף . . .זה ספר על Machine Learning, מה כבר יכול לקרות?(רן) האם המסווג הוא חיובי או שלילי?נושא אחר אבל קצת דומה (ופרגמטי) - בלוג-פוסט של GitHub שמתאר איך הם עושים MLOps (שזה בעצם Machine Learning Ops) באמצעות GitHub Actionsה - GitHub Actions הוא Feature בן שנה בערך, אולי יותר - ומאפשר לעשות לא רק CI מעל GitHub אלא בכלל איזושהי אוטומציה יותר כלליתלמשל - בכל פעם שעושים Push, אז להריץ איזשהו Pipelineכאן הם מתארים כל מיני משימות סטדנרטיות שיש ב-Machine Learning, שהם מכנים בשם הכללי “MLOps”לא שהם המציאו את השם הזה, הוא היה כבר קייםלמשל - ניקוי Data או Feature Engineering או הרצה של כל מיני Frameworks (במקרה הזה מדברים על binder) - דברים כאלהוכל זה - ב-Pull Request, וזה נחמדהרבה פעמים כשמפתחים איזשהו מודל ורוצים לעשות אופטימיזציות, רוצים לראות שלא עשינו משהו יותר גרוע, שלא שברנו משהו - וזה נחמד שכל הדברים הללו יכולים לקרות בצורה אוטומטית.אתם חושבים ששיפרתם משהו - עשיתם Commit לאיזשהו פרמטר ואז פתאום מגלים ששברתם משהו אחר . . . זה כל ה- Concept מאחורי Contentious Integration.בהקשר הזה - MLOps זו התשובה, והם נותנים דוגמא שלה באמצעות GitHub Actions(אלון) זה נשמע ממש בסיסי . . מה הבשורה שלהם?(רן) כקונספט, לנו כמהנדסים, אין כאן שום דבר חדש - אבל הם כן מראים איך הם עושים אינטגרציה לכלים הרלוונטיים השונים.איך אתה עושה Extraction ל-Data, איך אתה עושה Feature Engineering, איך אתה מריץ את המודל - וכל זה בתוך ה-Containers שלהםלמי שעושה CI כבר שנים אין פה חדש, אני מסכים - זה לא קונספט חדש, אלא משהו יותר פרקטי, מראים את הכלים עצמם(אלון) משעשע שהם משתמשים Argo עבור Workflow, ולא במשהו פנימי . . . לא ידעתי שמישהו משתמש בזה חוץ מאיתנו . . .שפה בשם goplus - וכן, זה “Go עם עוד קצת” . . .זה מעיין Super-set של Go, כשכל תוכנית ב-Go היא גם תוכנית ב-goplus - אלא של-goplus יש גם Syntax נוסף שמאפשר לה להיראות קצת כמו Script, קצת כמו Python באיזשהו מובן.לא חייבים להכריז על פונקציה, אפשר פשוט לכתוב “=:a” ולכתוב לשם איזשהו מערך וכו’ - נותן איזשהו “Feel” של Python (או Ruby או JavaScript), אבל עם Syntax שהוא מאוד Go-י - קצת כמו לקחת את Go ולעשות ממנו Script.כמה פיצ’רים בולטים - אפשר פשוט להריץ את זה כסוג של Script, לא צריך לכתוב פונקציה כדי להריץ משהוכמו ב-Python, יש יכולת לעבוד על List Comprehensions (או Map Comprehension), שכל מי שאוהב את Python בודאי מכיר - For x in . . . where x>3 - אז אפשר לעשות את זה גם למערכים וגם ל-Maps, וזהו מאוד קומפקטי ונחמדזה לגמרי Compatible עם Goויש עוד הרבה פיצ’ריםויש גם Playground - כמו שיש את ה Go Playground, יש גם Go+ Playground, שזה נחמדכל הקונספט של זה, לפי מה שרשום, זה שזה אמור להיות ידידותי ל-Data Science: ה-Tagline הוא The Go+ language for data scienceלמה זה “ידידותי ל-Data Science”? כי Data Scientists בדרך כלל עובדים בתוך Notebooks, כותבים סקריפטים קצרים ורוצים לראות מה התוצאה - ולכתוב תוכנית ב-Go זה לפעמים overhead שהוא קצת פחות מדבר ל-Data Scientists, ובגלל זה Python כל כך קוסמת.אז goplus מביא את חלק מהיתרונות של Python לפהכמובן שהחלק המשמעותי הוא הספריות - שאולי חלק מהן קיימות, אבל זה ממש לא באותה רמה של Python, אבל השפה כבר פה.האם זה חילול הקודש או ברכה? לא יודע, כל אחד עם הטייק שלו . . . מי שאוהב את Go ואוהב אותה כמו שהיא אז עבורו זה כנראה חילול הקודש, אבל למי שרוצה לראות את Go מתפתחת לכל מיני כיוונים אז זה אולי אחד מהכיוונים.דרך אגב - אני לא רואה את המפתחים של Go מאמצים משהו מפה - זו לגמרי שפה אחרת, אפשר לחשוב על זה כמו על C ועל ++C - יש כאלה שפשוט ישארו עם C תמיד ולא ילכו ל++C, וזה לא מתערבב.בכל מקרה - זה מעניין, וזה Repo שהושקעה בו הרבה מאוד עבודה - וגם מאוד פופולארי ב-GitHub(אלון) יש פה כמה קונספטים ממש מעניינים . . . ה-Error-Handling זה משהו שמאוד התחברתי אליו, הוא הרבה יותר הגיוני לדעתי.אני חושב שלקחת את Go ולהביא אותה ל - Data Science זה מעניין, אבל לדעתי זה לא יבוא מ-Go אלא יבוא מ-Rust כי Facebook מאוד דוחפים לזה, אבל זה מעניין, קונספט מעניין ומבורך.(רן) דרך אגב - יש ספריות Data Science ב-Go, הן לא עשירות כמו אלו של Python אבל בהחלט קיימות. בואו נראה . . .גם ב-Rust זה מעניין - יכול להיות שאת ספריות ה-Core, אם היום כותבים אותן ב-++C אז מחר יכתבו אותן ב-Rust, אבל עדיין משתמשי הקצה . . . הרבה מה- Data Scientists לא כותבים ב-++C אלה ב-Python או R, ואני לא רואה אותם עוברים ל-Rust סתם ככה, אלא אם כן הם באמת צריכים לכתוב ממש ספריות, וזה לא רוב הזמן.אלון - נתחיל מאחד הנושאים הפופולאריים - הפגנות Black Life Matter: התחילו לעשות “ניקוי שורות” בכל מיני שפות, נתחיל ב-Go כדי להמשיך את הקו: Pull request של להעיף את כל הרפרנסים ל - White list מול Black list או Master ו-Slave מה-Core Library של Goשמתי את זה בתור אחד מהראשונים שלי, ואז זה התחיל לתפוס פופולאריות בעוד כל מני מקומות, ולהתחיל להעיף איזכורים מעוד כל מיני מקומות.הרעיון הוא ש -whitelist/blacklist זה דבר פוגעני, וצריך להחליף ל Allowlist /Blocklist - שזה גם שמות יותר ברורים, האמת.ואת master/slave ל- Primary / Secondary אני חושב, לא רואה את זה כרגע.בקיצור - הרבה שפות התחילו לשנות, לא רק Go, והמונחים שאנחנו רגילים להשתמש בהם הולכים להשתנות כנראה בתקופה הקרובההדבר היחיד שעוד לא ראיתי ששינו זה את ה Git Repo - ה-Root זה עדיין Master . . . אבל עוד לא ניתקלתי במחאה בכיוון הזה.(דותן) חייב להגיד שאני נפלתי פה - לקחתי את ה-Commits שיש פה, סתם כדי להסתכל, ונפלתי על To-Do - שינו את הטקסט ב To-do, והיה שם Split כדי שאפשר יהיה לעשות allowlist במקום whitelist - אז אם כבר נכנסו ושינו, לא לא כבר עשו את ה-To-do? . . .(אלון) אם אתה הולך נגיד על fmt, אז שינו שם למשל את blacklist ל-blocklist . . .(דותן) כן - אבל יש שם הערה שאומרת “to-do: צריך לממש את זה אחרת”, ואם אתה כבר עושה re-factor ל-Comment אז כבר תעשה מימוש . . .(אלון) תראה, אני לא נכנסתי פה . . .(דותן) אבל אתה כבר שם! שינית את ה- whitelist ל-allowlist . . .(אלון) בסוף זה Copy-Paste-Replace . . . כן, שינו - אתה יכול לעבור על ה-commits, חלקם זה באמת Comments (בתוך ה-GC זה Comment) . . .בתוך loader.go שינו whitelist ל-allowlist(דותן) אז צריכים לעבור קובץ-קובץ ולהכריז . . .(אלון) כן, אין הרבה שינויים - אבל עשו עבודה, וזה לא במקום היחיד שעשו את השינוי הזה.טוויט נחמד שנתקלתי בו - Ashley Willis שאלה What’s the best tech talk you’ve ever seen?מה שמעניין זה שיש פה מאות תשובות עם לינקים להרצאות, שכל אחד טוען שזו ההרצאה הכי טובה שהוא ראהעברתי על זה ברפרוף ואמרתי שאני שומר לעצמי את הלינק הזה - והעבודה הבאה היא לפלטר לי מפה הרצאות ולהכין רשימת צפייה, כי זה בטח שווה משהו, אם כל אחד שם את ההרצאה שהוא חושב שהיא הכי טובה אז בטח יש פה רשימה מכובדת, “חוכמת ההמונים” וכו’.נראה כמו לינק שעבור מי שמחפש הרצאות לראות אז זה יהיה מאוד שימושי עבורו.(דותן) יש על זה כבר Crawling או עוד לא? . . . (אלון) לא . . .הנה , יש לך הזדמנות - שמו לפעמים את אותו לינק פעמיים ואז תדע עם מה להתחיל.(רן) רציתי להגיד שזה מדהים, מבחינת חדשנות ישראלית, איך לכל דבר אנחנו מביאים את ה-Touch האישי שלנו, פשוא מדהים המוח היהודי . . .(דותן) צריך רק למצוא איזו תמונה של מישהו מרצה על איזשהו Slide, ואז כשאתה לוחץ . . .(רן) כן, בשנות התשעים זה היה אחד הטובים(אלון) היית עושה מליונים, הרבה לירות היה יוצא לך מזה . . . בקיצור, יש כאן הרצאות ענתיקות בחלקן וחלקן מהשנים האחרונות, אנשים שמו פה הרצאות גם מ-1900 ומשהו, אני לא יודע אם היה למרצה מחשב באותה תקופה, כל מיני כאלה - וחלק זה ממש מהשלוש-ארבע שנים האחרונות אז כנראה יותר רלוונטי . . . נראה לי מגניב(דותן) אני גם לא רואה כאן את Remembering Joe . . .(רן) של Joe Armstrong? אני חושב שאני מכיר . . .(דותן) זה היה באחד הפרקים (369 הקוסמי!), מה זאת אומרת?!(רן) בסדר, לא כולם מקשיבים (ברור, חלק רק קוראים)(אלון) דווקא חושב שראיתי את Joe Armstrong שם, די בטוח - בקיצור, תעבור, תכין רשימה יותר מצומצמת, ניתן לרן לצמצמם עוד קצת - ואז אני אסתכל(דותן) אי אעשה את הישנים והטובים, אתה תעשה את המודרניים והמגניבים(רן) ואני דורש שיהיו בכל רשימה לפחות חמישה מכנסי רברסים שעברו . . .(אלון) זו הזדמנות להכניס שם ל-List ולהתחיל להפציץ אותו . . . אני מבקש מכל המרצים: כל אחד, שישים את הלינק של עצמו.זו קריאה למרצים! - שימו את הלינק להרצאות שלכם שם, ואז אתה מקפיץ את הכנס כנס? 2020?ספריה ישראלית - golang mediary - של Here Mobilityהוספת interceptors ל-http.Clientשלחו לי - הסתכלתי - נחמד - מפרגן בכיףהרעיון הוא שאפשר להתחבר על ה HTTP Request - לפני ה-Request, אחרי ה-Request, ואז לעשות אינטרפולציות ל-Request עצמו או ל-Responseאפשר להוסיף לוגים או דברים של Security או statsd . . . יש דוגמאות, גם Tracing . . . יכול להיות מענייןנראה חמוד למי שצריך את זה, ספריה צעירה יחסית - שיהיה בהצלחה! אני אהבתיונמשיך עם Go, ככה יצא הפעם - mockery זו ספריה שמאפשר לעשות Mock-ים ב-Goספרייה מאוד פשוטה וחמודה - למי שמחפש לעשות Unit Test ומחפש איך למקמק (create mocks) קוד - שווה להסתכלנחמד, פשוט, קליל, שימושי ונוח.(רן) ואחת הפופולאריות שבהן - יש עוד אחת-שתיים, אבל זו אחת הפופולאריות ביותר(אלון) מה שמפתיע זה שגם הפופולאריות לא פופולאריות . . . פחות מ-2000 Stars זה . . . או שאנשים לא עושים טסטים, גם אופציה(רן) אני חושב שפשוט צריך הרבה פחות Mocks, במיוחד ב-Go, בעיקר בגלל הגישה של ה-Interfaces - פונקציה שמקבלת Interface, אז אם הוא מספיק “רזה” זה כל כך קל למקמק (Mock) בעצמך כך שאתה לא חייב שום Framework.מתי כן צריך Framework? אולי לא צריך - אבל מתי תרצה? או כשה-Interfaces יחסית ארוכים ואתה לא רוצה למקמק הכל בעצמך, או כשאתה רוצה לעשות Spying: לספור את מספר הקריאות או משהו כזה, ואז אתה כבר תלך ותשתמש באיזשהו Frameworkאני, בטסטים שלי, פשוט יוצר Instances של ה-Interfaces בלי להשתמש באף Framework - יותר קומפקטי, יותר מובן, לדעתי, לא מצריך ללמוד עוד Framework - אני חושב שזה לפחות חלק מההסבר(אלון) כן, אבל הרבה פעמים יש דברים מורכבים . . . זה נכון לדברים יותר פשוטים, אבל כשאתה בא לספריית צד-שלישי בדרך כלל, עם כל מיני התחברויות ודברים שקורים . . . זה יותר מורכבאני ניסיתי פעם למקמק ל-S3, וזה לא היה סימפטי(רן) במקרים כאלה אני באמת לא אקח את זה על עצמי ובאמת אשתמש בספרייהאו שאני אשתמש בבדיקות אינטגרציה (Integration Testing), למשל - ארים Container שיש לו Interface של S3 - מכיר את Testcontainers? יש להם מלא קונטיינרים עם כל מיני כלים - S3 זה אחד מהם אם אני לא טועה, יש ל-SQS ולעוד כל מיני דברים כמובן - כל הדברים הסטנדרטיים כמו Databases מסוגים שוניםאז אתה יכול פשוט להרים Container - ודרך אגב יש לזה גם תמיכה ב-Go: אתה יכול לעשות setup לטסט שמרים לך Container בהתחלה ואז מוריד את ה-Container, ולפעמים זה יותר נוח מאשר למקמק (Mock it) את זה בעצמךזה אמנם רץ יותר לאט, אבל מצד שני זה קצת יותר אמין, מבחינת ה-API(אלון) מבחינת טסטים ל-Integration זה הכי נחמד - אבל זה כבר Integration Test ולא Unit Test.(רן) נכון, זה כבר לא Unit Test - אבל אתה כבר עובד עם S3, האם זה עדיין Unit Test? שאלה פילוסופית . . . אם אתה גם ככה כבר עובד עם משהו כבד חיצוני, זה כנראה גם ככה כבר לא ממש Unit Test.(אלון) זה ברור, אני נכנסים פה כבר לפילוסופיה . . .(דותן) זה עניין של טעם, בסוף - טעם ואיזון.(רן) לגמרי - אני לא מנסה להחליט מה זה Integration Test ומה זה Unit Test כי לא נצא מזה בחיים - רק אומר שיש לך כאן כמה אופציות, ואחת מהן זה באמת לעשות Mocking באמצעות mockery או באמצעות כלים אחרים; אופציה שנייה זה לקחת את ה-Interfaces ולממש אותם בעצמך, וזה נוח כשה-Interfaces יחסית “רזים”; ואופציה שלישית זה באמת להרים Service, אם אתה מדבר עם Service - להרים Service ב-Container ליד; או, רחמנא ליצלן! - לדבר עם ה-Service האמיתי (למשל S3 האמיתי), אבל זה ברוב המקרים הכי פחות מומלץ.אם אתה באמת הולך על הגישה של Container - יש Framework כזה שנקרא Testcontainers, שיש לו תמיכה בהמון שפות - Java ו-Go ובטח עוד הרבה - שממש נותנים לכם בזמן ה-Setup של הטסט להרים Container ולהוריד אותו בסוף הטסט, והאינטגרציה הזו מאוד נחמדה.(אלון) זה חמוד ממש - ותמיד יש את ההמלצה הקבועה: הכי טוב זה טסט אמיתי - טסט על Production! למה לא לנצל את זה?(רן) Famous last words . . .דותן - ספריה ש-Apple הוציאה, או יותר כמו Framework, בשם ExposureNotificationאם נחבר את זה לאקטואליה - בעצם הם ייצרו Framework סטנדרטי שממדל חשיפות ל - COVID-19זה חלק מההכרזות שלהם לא מזמן (iOS 13.5 release)- הם ראו שיש כל מיני ממשלות או כל מיני אפליקציות שמנסות למדל חשיפות לקורונה על גבי מפה וכו’ - והם פיתחו עבור זה API סטנדרטיעכשיו אם אתה רוצה לבנות אפליקציה כזו - אתה יכול להשתמש בספרייה הזאת, והיא גם עוזרת לך פה ושם.אני (דותן) נכנסתי לקרוא את ה-Interface, ויש שם כמה חלקים מגניבים, שאולי מגיעים משפות של רפואהלדוגמא, לרגע התבלבלתי כשהיה כתוב שם “Transmission risk level” ו-”Signal” - אני לקחתי את זה לכיוון של רדיו וכו’ . . .(רן) אתה כנראה הסתכלת על טורי פורייה, אבל הכוונה לביולוגיה . . .(דותן) בדיוק . . . הכוונה ל-Transmission של המחלה, אולי ה-Signal של המחלה? בכל אופן - נראה מעניין, לפחות ברמה של ה-API, שאפשר לקרוא איך נראית קורונה דרך API . . . זה מגניב, וכמובן שאם מישהו רוצה לפתח אפליקציה פופולארית ל-App Store, אז זה מקל את הכאב . . .(רן) דרך אגב - לא דיברנו כאן ואולי שווה לדבר על איך עובדות אפליקציות למעקב אחרי קורונה . . . בגדול, לפי מה שאני (רן) יודע, יש שני סוגים - סוג אחד זה לפי קירבה - משתמש ב-Bluetooth ועושה איזשהו מעקב אחרי מי נמצא ליד מי, למשל אם אתם נמצאים במקום ציבורי, אז ה-Bluetooth שלכם “מדבר” עם Bluetooth של אחרים, וככה אתם יודעים אם אתם קרובים למישהו אחר - ואם אחר כך מתגלה שהוא חולה, אז יש את המעקב הזה.איך זה נשמר ואיך באמת עושים את הגלוי? זה כבר סיפור אחר . . . אבל לפחות ברמה העקרונית, ברמה הפיזית, הגילוי הוא באמצעות Bluetooth.שיטה אחרת זה באמצעות מיקום - GPS וכו’למיטב ידיעתי, השיטה של ה-Bluetooth נקראת “השיטה הסינגפורית”, ואותה בסופו של דבר גם Apple וגם Google מאמצים - כשדיברו על זה ש-”Apple ו-Google משלבים ידיים למאמץ משותף” אז מדובר על זה, למיטב ידיעתי, בשיטה שמבוססת על ה-Bluetoothאלא שזה לא יהיה באפליקציה - זה יהיה ממש מוטמע במערכת ההפעלה, וזה יהיה Battery efficient וכל זה.השיטה של האפליקציה הישראלית שנקראת “המגן”, אני מניח שהרבה מכם התקינו אותה - זו דווקא שיטה שמתבססת על מיקום - ולכל אחד מהם יש יתרונות וחסרונות:ל-Bluetooth - מצד אחד הוא באמת יותר אמין - ברזולוציה, Bluetooth אמור לקלוט למרחק של כמה מטרים בודדים, כשהדבקה מוגדרת, אני חושב, כמצב שבו אתה נמצא רבע שעה במרחק של שני מטרים או פחות מבנאדם - ומרחק של שני מטרים או פחות זה משהו שבדרך כלל Bluetooth יודע ו-GPS פחות יודע, כי GPS (אזרחי…) עובד ברזולוציה יותר גבוהה.מצד שני - ל-Bluetooth יש גם יכולת לקלוט מעשרה או עשרים מטרים, תלוי בתנאי מזג האוויר ורעשי רקע ודברים כאלה.לכל אחד מהם יכולים להיות False Positives, ואולי גם False Negatives - אני לא מכיר את המקרים אבל יכול להיות שיש כאלה.זהו - אני חושב שזה מעניין, ככה, קצת לדבר על הטכנולוגיה שמאחורי זה, אבל אני שואל את עצמי האם באמת Apple ו-Google יכולים לקחת את ה-Bluetooth ולהוריד שם את רמת ה-False Positives בצורה משמעותית, כי בשביל להיות מסוגלים לעשות את זה, צריך גישה ממש למערכות הפיסיות, כדי להבין באמת מה עוצמת הסיגנל ומהן רמות ההפרעה וכו’, כדי להבין האם באמת הבנאדם קרוב או רחוק ממני.(דותן) וזו קריאה ל Apple ו-Google - לשלוח מכתב למערכת (AWS מאזינים מזמן . . .), אבל כן - זה מגניב(אלון) קודם כל - שמעתם את זה פה לראשונה, כי אנחנו תמיד חוזים דברים, זה ידועאבל רגע - “לפני מיליון שנה”, כשעבדתי באינטל, היו חיישני Bluetooth והיינו מבינים איפה הדבר נמצא לפי המרחקים ועוצמת ה-Bluetooth - עוד אז רישתנו הכל ב-Bluetooth וידענו להגיד איפה ה-Wafers נמצאים בכל רגע נתון לפי מרחקים - אז זה משהו שכבר קיים, לפי הרבה שנים(דותן) עוצמת הסיגנל של Bluetooth, אם אני זוכר נכון, קיים ב-iOS(רן) כאן, זה קיים - השאלה היא רק מה רמת הדיוק של זה? לפעמים עוצמה היא “5” כשאתה במרחק שני מטרים ולפעמים העוצמה היא “5” כשאתה במרחק של עשרה מטרים . . . זה לא מדויק. אתה יכול אולי באופן יחסי להגיד מי קרוב ומי רחוק(אלון) תראה (תשמע) - אני יכול להגיד לך שאנחנו אולי היינו (Literally) בתנאי מעבדה, אבל בתנאי מעבדה זה היה מאוד יציב . . . היה מאוד ברור וזה עבד מאוד טוב, הזיהוי מרחק של מקומות, זה היה עוד בזמן “Bluetooth 0” או לא יודע איזו טכנולוגיה זה היה, אבל ה-Bluetooth התקדם מאז די הרבה אז יכול להיות שעכשיו זה שונה - אבל בזמנו זה עבד, אז אני לא יודע מה הבעיה . . .(רן) הפיסיקה השתנתה . . . באמת, אין לי ידע עמוק בזה אז אם מישהו מהמאזינים מכיר אז מוזמנים לתקן אותי, למיטב הבנתי זה פשוט מאוד תלוי בתנאי הסביבה, ובאמת יש הבדל מאוד משמעותי אם אתה בתנאי מעבדה או לא - תלוי בלחות, תלוי במכשירים האחרים שנמצאים ליד, ואני מניח שבעוד כמה פרמטרים.אבל שוב - אני בטח לא מומחה לתחום, ואני גם שמעתי או קראתי את זה איפשהו.בכל אופן - אני חושב שזה מעניין עכשיו להגיד שבאמת יש שני מודלים, ויכול להיות שהתשובה היא איזשהו שילוב של שניהם, כדי להגיע לרמה דיוק יותר גבוהה - אבל שני המודלים האלה בגדול הם שאחד מתבסס על שירותי מיקום (כמו באפליקציית המגן הישראלית), והשני מתבסס על Bluetooth, זהו, Se Tu.(אלון) רק אסיים - הפיסיקה אכן השתנתה! בתקופתי העולם היה עגול ועכשיו אומרים שהוא שטוח, אז זה כנראה שינה את כל הפיסיקה(דותן) ואז ניהיה דור 5 . . . ספריה וכלי - streamlitמבוסס Python, או לפחות לקהילת ה-Python או ככה זה נראהלמי שמכיר את Swift Playgrounds - זוכרים שהייתה ההכרזה של Apple על Swift, ואז זה גם הופיע ב-iPad - שאתה צריך לכתוב קוד ומופיעה לך ויזואליזציה של הקוד שלך והכל אינטראקטיבי, אתה יכול להזיז Sliders כאלה, והקוד שלך בעצם משתנה לפי ה-Sliders?אז הם לקחו את הקונספט הזה - ועשו את אותו הדבר ל-Pythonלפחות מה-ReadMe נראה שקהל היעד זה בעיקר Data Scientists ואנשים שמתעסקים עם Data.שיחקתי עם זה קצת וזה אחלה לכל דבר - מספיק שיש לך פה Sliders ו-Controllers אינטראקטיביים, ויש לך איזושהי פונקציה ב-Python שאתה רוצה לשחק איתה, אז זה מהר מאוד יכול להפוך לכלי לימודי, בלי קשר ל-Data Science, אחלה דבר.(רן) אני מחכה לראות את זה נכנס לתוך Jupyter Notebooks, כי זה מתבקש הרבה פעמים רציתי לעשות איזושהי ויזואליזציה (Visualization) עם איזשהו Control של Slider, או משהו כזה - ועד עכשיו לא מצאתי, אז נראה שזו אולי התשובה, רק צריך לעשות לזה אינטגרציה לתוך Jupyter(דותן) לא ראיתי על משהו כזה . . . כן נראה שיש פה חברה מאחורי זה, סוג של . . . אני מניח שהם רצו להחליף או להיות אלטרנטיבה לזה, כי זה נראה קצת כמו Jupyter.קצת בקטע של נוסטלגיה - Cryengine, או Crytek - החברה שמאחורי Cryengine שמאחורי המשחק Crisis - פתחה (Open sourced) את הקוד של המנוע הראשון של Crisis (המשחק)אנחנו לא משחקים עם ה-Crisis הראשון, אבל אני זוכר אותו, כי זה מסוג המשחקים ששינו את העולם ונשארים לך במוח, כמו Doom וכאלה (עד כדי כך?)אז הם פתחו את הקוד ואני קצת רפרפתי - קצת ++C, בגדול, שנראה שנכתב ע”י מפתח אחד או שניים, “במשיכה אחת” מה שנקרא.מעניין למי שאוהב נוסטלגיה - אני אוהב להסתכל לפעמים; לא בניתי, לא קימפלתי וממש גם לא הולך לעשות את זה, אבל לפעמים גם כיף להסתכל על קוד שנכתב באותה תקופה, וזה נחמד.(רן) אני מסתכל על Commits שלהם, ונראה שיש להם מוסכמה מעניינת ל-Commits - נגיד B! או T! או I! . . . מעניין מה זה.(דותן) האמת שראיתי את זה וזה היה נראה לי כמו רעש, אבל אתה נותן פה טוויסט מעניין . . . (רן) כנראה שיש כאן איזושהי קונבנציה (Convention) ל-Commits שאני מנסה לפענח . . לפני איזה שניים נגיד יש XB! (היה בהקלטה לפחות . . .)(אלון) וגם XI! . . . זה מגניב, עכשיו אני חייב להבין מה זה . . . T! זה סתם טקסט, אתה רואה שזה סתם Copyright וכאלה, אז זה כבר מעניין.(רן) אולי B! זה Bug . . . מה זה I! ? . . .(אלון) U! זה בטח User Interface . . . לא, בעצם זה Undo . . . נחמד(דותן) יש כאן עוד כמה דברים מעניינים - יש Commit שמתקן משהו שנראה כמו Bug מלפני חודש - עכשיו, זה Cryengine, זה מ-2004 . . . מה קורה פה?(רן) כנראה עבדו על זה כדי להוציא את זה ל-Open Source(דותן) יכול להיות . . . מעניין; אלו החלקים שאני אוהב לנבור בהם, בקוד מאוד ישן - מגלים כל מיני דברים שהאנושות כבר לא עושה.(אלון) עכשיו רק תחפש פה פרצות אבטחה ונחש מה עבר הלאה לגרסאות החדשות . . .(דותן) כן, הא . . .האייטם הבא הוא backstage - פרוייקט של Spotify שהם החליטו לעשות לו Open-sourceזה בעצם Developer portal Framework, והם מכנים את זה “open platform for building developer portals”אני חייב להגיד שקראתי את זה ומאוד רציתי לדעת מה זה - וכשראיתי אז מאוד לא רציתי לראות מה זה . . .לא יודע, אני עדיין מבשל את זה עם עצמי - זה נראה כמו Wiki משולב ב-Dashboards, והכל מוכוון למפתחים ב-Spotify - אם אתה חבר ב-Squad אז יש לך את ה-Squad metrics מול הפנים; אם אתה רוצה לקרוא חדשות אז יש לך חדשות של Spotify שם; אם אתה לראות Metrics של Services אז זה גם שם - בעצם, כל העולם שלך נמצא בתוך מקום אחד.אולי אני קצת Old school, אבל זה . . . אני קצת פחות התחברתי, זה משדר “רובוט שעובד בשביל חברה”, וכל עולמו נסגר במקום אחד . . . כשאני קראתי את זה, חשבתי שאני הולך לראות Developers Portal במובן של כל הידע של ה-Developers והפרוייקטים והכלים שאני יכול להשתמש בהם כדי להאיץ את העבודה וכו’ - אבל אני בעצם רואה פה סוג של “מנגנון שליטה” או “חוטים סביב הבובה”. אבל תשוטטו בזה, זה מגניב.(אלון) אני עוד לא הבנתי מה אני יכול לעשות עם זה, אם זה טוב או רע - אני צריך לראות את הוידאו, לא נעים לי(דותן) יש לך Gif, לא צריך וידאו . . .(אלון) ה-Gif לא מספר את כל הסיפור . . . ב-Gif זה דווקא נראה חמוד: אתה מכין דשבורדים (Dashboards), יש את כל המטריקות (Metrics) שאתה צריך, אם מעניין אז יש משהו לראות . . . יכול להיות נחמד.(דותן) זה קצת Fallacy, כי קודם כל - אם אתה מאמן או מאלף אנשים להסתכל רק במקום אחד ולא לצאת מהמקום הזה, אז אוקיי, סבבה - יש כאן כל מיני Widgets שאם מישהו שם Widget שאתה אמור להכיר אז עכשיו לא הכרת ולא ידעת אז זה לא קיים.(אלון) אתה יכול לבדוק את ה-CI, לבדוק את המטריקות (Metrics), לבדוק לוגים . . . יש לך מקום אחד במקום להתחיל לטייל, וזה לא רע.(רן) לא - וגם חברות עושות את זה אז בוא - כל חברה בונה את זה לעצמה, כל חברה שאני הייתי בה בנתה אחד כזה, אז זה יכול להיות נחמד להתחיל ממשהו מוכן.אתה יכול לבוא ולהגיד שיש לזה חסרונות, כי ברגע שאתה בונה פורטל כזה לא מסתכלים ימינה ושמאלה - אולי, אבל מצד שני כולם בונים, כי אני חושב שה-Benefit עולה על החסרון הזה.עכשיו - האם זה פורטל טוב? אני לא יודע, אבל האם צריך פורטל? אני חושב שכן, אני די משוכנע שצריך.(דותן) זה תמיד יש - יש לך Jira ויש לך את העולמות שלך . . מה שאני מכיר זה שבונים, אבל בונים בתצורה של כלי, ופה ה-Feel שאני מקבל זה של “זה העולם שלך, וה-Browser שלך נעול לתוך הדבר הזה וזהו”. זה Feel כזה, זה לא באמת . . .(רן) יכול להיות . . . אני מסכים עם זה שנכון שיהיה לו API, שזה לא יהיה UI-First אלה API-First, שכל פעולה שאתה יכול לעשות דרך ה-UI אתה תוכל לעשות גם דרך ה-CLI באמצעות Client וכו’.עדיין, אני חושב שזה נכון שיהיה איזשהו פורטל מפתחים, ששם יהיה את כל מה שהם צריכים - אתה יודע, דברים בסיסיים כמו Service Catalog ו-Metrics ואיך ליצור Service חדש, ומי ה-Owner של כל אחד מה-Services ומה התלויות בינהם ודברים כאלה.דרך אגב - לא הכל כל כך בסיס, חלק מהדברים כן מורכבים, אבל זה הכל שימושי בעיני.כל חברה שהייתי בה בסופו של דבר בנתה לעצמה אחד כזה, אז אני חושב שזה נחמד להתחיל מאיזשהו משהו, אבל אני לא יודע - צריך לעשות לו איזשהו Test Run ולראות האם זה באמת הכלי הנכון בשבילכם.(דותן) לא, עכשיו זה נראה . . פחות, אבל תנסו(אלון) אל תקשיבו! Spotify, אתה לא יכול ללכלך עליהם - הגיע סוף סוף לארץ ה - Spotify Family (קישור לא ממומן . . .), אז אני מבקש - לא ללכלך עליהם!(דותן) לא מלכלך . . . זה אחלה, כלי מדהים!הספרייה הבאה - rich - עושה צבעים ב-Pythonחייב לומר שזו סופסוף ספרייה שנראית טוב, עבור מי שרוצה ליצור Developer experience טיפה מעבר למה שיש בסטנדרט של Python.היא עושה את כל בצבעים, כל הפלטה (palette) - טבלאות ו-Spinners ו-Progress bars, עושה גם Syntax coloring על הטרמינל ועוד ועוד - אפילו מרנדרת markdown מגניב, ברגע שאתה לוקח ספרייה כזו, יש לך את החופש לעשות מה שבא לך, או שבתוך הטרמינל את יכול לרנדר Markdown, יכול להוציא טבלאותאני מניח שכלים מגניבים יבנו מעל הספרייה הזאת ובזכותהממש אהבתי - וגם עושה חשק לבנות כלי Command Line חדשים שנראים טוב ב- Pythonתשתמשו!ספרייה בשם texthero - שעושה עיבוד טקסטהדגש פה הוא על זה שהיא קלה וקלילה - אהבתי את הנקיונות של טקסט שבה, אבל יש בה עוד יכולותאתה מתקין ומיד יש לך כל מיני אלגוריתמים פופולאריים לעבודה על טקסטלא יותר מדי עמוק אבל גם לא יותר מדי - פשוט וממש נחמדלמי שלא אוהב את הדוקומנטציה (Documentation) של Docker, יש docker-cheat-sheet (באתר של Docker)כאן יש את כל הדוקומנטציה שבאתר - משוטח לקובץ Markdown אחד, הכל ב-Repositoryגם נחמד - וגם יותר קל לחפש, וגם יותר נוח להשאיר פתוח כל הזמן . . .(אלון) רשום פה “4 months ago” . . .(דותן) כן, הדוקומנטציה הרשמית כנראה מתעדכנת יותר תדיר, אבל יש פה את הדברים שהם Basic ורוב מה שלפעמים אתה אולי שוכח אז יש לך.עוד ספרייה בשם mimalloc - נושא שהוא קצת יותר Low-level ו-hardcore, דיברנו על זה קצת בעבר - הספרייה היא לשימוש ב-Allocator ש-Microsoft הוציאוהם בעצם הפכו ל-Allocator עם ה-Performance הכי טוב בשוק, פחות או יותרלאן זה רלוונטי? רלוונטי לספריות או כלים שבנויים על ++C, וב-Space האישי שלי - על Rust.אנחנו רואים פה כבר הבדלים שהם יחסית משמעותיים - היא עושה ניהול אלוקציה של זכרון (Memory allocation) פי 5 או פי 6 מהר ממה שיש שיש לך ב-Default.יש פה גם פי 10 ופי 20 לעומת אלטרנטיבות אחרותלמי שעוסק ב-Performance או ש-Performance חשוב לו, ויש לו Code Base שעושה המון אלוקציות והמון עבודה “קשוחה” כזו ב-Rust, יכול להחליף את ה-Allocator שלו ברמה של כמה דקות עבודה ולראות האם זה שיפר לו ביצועים.בשפות אחרות אני מניח שזה גםבשורה התחתונה - הופך להיות משהו שהוא פחות אקספירמנטלי וכבר נראה די טוב לשימוש.עוד אייטם שמצא כן בעיני דווקא בגלל ה-Feel שלו - hackingtool: כלי ל-Hackers כמו בשנות ה-90!מישהו לקח סקריפט ב-Python, ובנה כאלה Prompts ולוגו כזה ענק וכו’ - וזה בסך הכל מפעיל מלא Scripts אחרים, סתם הצחיק אותי(אלון ) רגע . . . עכשיו אנחנו עובדים מהבית, אבל במשרד, עם חלון כזה פתוח באופן קבוע זה . . . שמע - להיט!(דותן) כן, ממש 90’s, ממש הזכיר לי את זה - זה כזה עם תפריטים, שאתה לוחץ ואז מופיע התפריט הבא, ויש כותרת אחרת ועוד תפריט, עד שבסוף אתה מגיע למה שאתה רוצה להפעיל ואומר לו “תפעיל!” . . . ממש s’90 ונוסטלגיהבסוף יש מלא כלי Hacking, ממש המון, אז הוא לקח רק כמה - לא יודע אם זה הכלי הכי טוב ל-Hacking או ל-Pen-Testing, אבל בהחלט הכי מעלה זכרונות(רן) אני זוכר שפעם היו ממש גרסאות Linux שממש היו מיועדות לזה, עם כל הכלים מותקנים . . .(דותן) אה - יש! עדיין יש(רן) עוד עושים כאלה?(דותן) בטח . . . מה שקרה איתן זה שלמשל KALI ו-Backtrack הפכו להיות חברות, באיזושהי דרך, חברות Security שאיכשהו מימנו או קנו, ונוצרה להן מעיין יישות שהיא, מעבר להפצת Linux עם מלא כלי Security, בעצם גם מובילת-דעה בעולם של Pen-Testing, וחלק ממה שהיא עושה זה גם להוציא את ההפצה שנקראת, נגיד, KALI.אז לא רק שהיו - הן גם התרבו ויש כבר די הרבה.ב”ימים של האינטרנט הגרוע” היה לי כזה, בסטנדרט, בתיק - וכשהייתי צריך אינטרנט אז הייתי “משיג” בצורה כזאתגם ה-WiFi של פעם לא היה כזה מתוחכם - לוקח כמה דקות ויש לך סיסמא של מישהו, של ה-WiFi שלו . . .היום זה כבר פחות רלוונטי, זה יותר קשה לעשות(אלון) תגיד - הרצת את זה? יש גם מוסיקה, כמו פעם?(דותן) לא . . . אין מוסיקה, אבל זה אחלה רעיון ל-Pull Request.(רן) זה כולל קפוצ’ון?(אלון) נראה לי שתורנו לקבל קפוצ’ון . . .(דותן) רעיונות מדהים, נראה לי שצריך להוסיף ל Pull Requests - “להוסיף מוסיקה!”ועוד אחד - EasyOCR: מישהו לקח Neural Network, את כל מה שאנחנו מכירים ב-Neural Network ו-Deep Learning וזיהוי טקסט, ארז את זה בספרייה ויצר OCR שמזהה כמה וכמה שפות.אני חושב שהדגש הוא על קלות ההפעלה, או איך שלא נקרא לזהבעצם, בשלוש שורות - יש לך OCR, מה שבדר”כ היינו עושים tesseract כזה, שזה חינמי? אז פה כבר אפשר לקחת, לנסות ולראות אם זה נותן יתרון משמעותי מעל ה-OCR-ים האחרים, החינמיים.(רן) רק נזכיר למי ששכח - OCR זה Optical Character Recognition - היכולת “לקרוא” טקסט(דתן) מקבלים תמונה - מקבלים טקסטואם כבר אנחנו מתמקדים בנושא - ה OCR-ים “מהדור הראשון” לקחו פונטים ואיכשהו היו Coupled לפונטים בדרך שלהם לזהות טקסטהיום זה כבר Neural Network, אז ההבדל הוא די רציניבכל אופן - ה-EasyOCR יודע לעשות את זה גם באנגלית וגם בשפות קצת יותר אקזוטיות: סינית, תאילנדית וכו’. מעניין.אייטם נוסף - gitqlite: אני ראיתי בזה עוד פעם את “איך לא עשו את זה כבר?” - מישהו לקח Git Repo ולקח SQLite . . . היה לנו אייטם כזה פעם, שמישהו לוקח Data, מכניס אותו ל - SQLite ויוצר לו ספריית תחקור . . .אני חושב שזה היה אפילו מישהו ישראלי, זה היה נקרא q, לא? אם אני זוכר נכון . . .(רן) הראל בן עטיה כתב את q, שבאמת לוקח Data, שם אותו בתוך SQLite ואז מתשאל אותו.(דותן) כן, אז שם זה היה JSON אם אני זוכר נכון, וכאן זה Git Commits או Git בכלל - אני מניח שככה הוא בנה את זה: לקח Git Log ועשה לו קצת Parsing או אולי משהו קצת יותר מתוחכם, דחף את זה ל-SQLite לכמה טבלאות, ועכשיו יש לך כלי Command Line שאתה יכול להפעיל שאילתות מעל ה-Repo שלך או מעל ה-Git - שזה די מגניברעיון כזה פשוט ש”איך אף אחד לא חשב על זה קודם?”(רן) במקרה של q, אני חושב שהיו לו כמה סוגים של Inputs - גם JSON וגם CSV וגם Output של פקודות, שהוא היה יכול לפרסר (Parsing) אותן כטבלאות.(דותן) מגניב . . . צריך לבדוק מה הוא עשה ב-gitqlite, אבל אולי אפשר להזרים את זה לתוך q . . . בעצם לא, זה SQLite . . .ואייטם אחרון (כמעט) - practical-python: לא יודע אם זה כזה Highlight כי יש כל כך הרבה resources ללמוד Python, אבל כשהסתכלתי על זה אז משהו קפץ לי פה - השם של מי שעשה את זה הוא David Beazley - וכל מי שעשה Python בשנות ה-2000 מכיר את David Beazley, רן מכיר בטוח . . .(רן) לא מכיר . . .(דותן) הוא עשה את ה-Python Cookbook והיה די חלוץ בעולם ההוראה ה Python-ימה שהוא בעצם עושה זה לפתוח את הקורס שלו, שהוא כתב שהוא העביר יותר מ-400 פעמים, סוג של Training שלו - הוא פותח אותו ועושה אותו חינם ופתוח ב-GitHub, ואפשר ללכת ולעשות את הקורס.יש שם Exercises והוא טוען, ואני מניח שהוא צודק - שהקורס הזה הוא בעצם למידה שלו, שהוא שייף לאורך משהו כמו 20 שנה אחורה.מעניין לפחות להסתכל מה יש שם.ואייטם ממש אחרון - !HEYגובל בדרמה, ואני מניח ששמעתם מה היה עם !HEY . . .(רן) לא - ספר לנו!(דותן) אה, אוקיי . . אז יש את ה-Email החדש שנקרא !HEY, אם אפשר לקרוא לזה ככה, ש DHH . . .(רן) זה Email client?(דותן) לא יודע אם Email client, זה ממש email . . .מחליף את Gmail באיזשהו מובן, ש-DHH ו-Basecamp וכל הקבוצה הזו הוציאו.זה לא של Basecamp, אבל זה חלק מהכלים של Basecamp, נראה לי, בקטע של Productivityמה שהוא אומר זה שהוא הוציא מייל שהוא לא של אף יישות גדולה, לא יודע אם להוסיף “מרושעת” אבל כנראה שזו הכוונה שלו, שהוא תומך ב-Privacy וכו’אבל העניין שהתפתח הוא ש-DHH כהרגלו, יש לו איזשהי מנטרה ל-business שהיא מאוד ידועה, וכשהוא הגיש את האפליקציה של !HEY ל-Apple App Store, אז הוא עבר על ה-Policy של in-app purchase - וקיבלת אפליקציה שאי אפשר להשתמש בה, אלא אם כן את הולך לאתר הנפרד, שלא קשור ל-app Store, של !HEY, ואז אתה משלם ואתה כן יכול להשתמש בה . . . ו-Apple - כמובן שזה נוגד את ה-Term & conditions שלהם, אתה לא יכול לתת אפליקציה שאתה לא יכול להפעיל אותה בלי לשלם, ולשלם בתוך ה-Ecosystem של Apple - אז הם עשו לו Ban לאפליקציה . . .ואז התחילו משהו כמו שבועיים של טרור-טוויטר של של DHH נגד Apple, והתפתחו כל כך הרבה Threads ושיחות מטורפות מעל Twitter וזה די “שבר את Twitter” - ובסוף Apple וויתרו.וזהו - זה היה HEY . . .(רן) רגע - אז הם נותנים לו לעשות Purchase מחוץ ל App Store? בתוך האפליקציה?(דותן) הם סוג-של-וויתרו, וגם הוא סוג-של-וויתר - אבל זה היה . . . אם היית קורא את ה-Twitter בימים האלה אז כאילו נראה היה שיש פה מלחמה ואף אחד לא הולך לרדת מהעץ - אז בסוף הוא עשה גרסא סוג-של-חינמית והם סוג-של-וויתרו על החוקים הנוקשים שלהםאפילו מישהו פתח אתר כזה … היה איזה VP ב-Apple שאמר “You download the app and it doesn’t work”’ ואז מישהו פתח אתר כזה בשם YouDdownloadTheAppAndItDoesntWork.com - ושם היו Screenshot של כל האפלקיציות שאתה מוריד והן לא עובדות.הבדיחה היא שהן לא באמת לא עובדות . . .בין השאר היו גם Spotify ו-Netflix וכו’, וכולן במודל הזה - ב-Apple אמרו שזה Reader וזו לא בדיוק אפליקציה, אבל גם Gmail זה Reader . . . בקיצור, התפתחו שם כל מיני דיונים פילוסופיים מסובכיםיש כאלה שטוענים שזה היה PR Stunt של DHH, כי זה נתן המון פרסום - מעבר ל-Twitter זה עשה המון גלים בכל “אתרי החדשות לגיקים”, אבל זה…מה שנותר לעשות זה לנסות להשתמש בHEY ולנסות להחליף את המיילים שאתם מכירים בחינם - בכסף.(רן) יפה אז סיפקת לנו את הדרמה של היום, בהחלט.(אלון) אני עדיין לא מבין למה אני צריך להחליף את האימייל שלי מכל הסיפור . . . (דותן) אז אמרתי - אתה מוזמן להחליף את האימייל שלך באימייל אחר - בתשלום!(אלון) במקום בחינם?(דותן) כן(רן) אני חושב שזה הקטע שהוא לא מבין, דותן, אבל נסביר לו אח”כ.לאוסף(דותן) סתם - מה שהוא מוכר בסוף זה Privacy - במחיר של $99 לשנה, אתה מקבל Privacy: הוא חוסם לך Trackers וכאלה, ואתה מקבל כתובת אימייל של hey.com, שזה כאילו מגניב . . . אפשר לפתוח לרגע פסקת ציניות? כןלפני ההשקה, כי DHH חימם את כל Twitter, מישהי עשה לו Reply ואמר לו “כבר השגתי כניסה ל-HEY, והכתובת של זה Hey@username” - הפכה את ה-Domain ואת השם, שזה כאילו . . . בסוף את משלם על Domain של שלוש אותיות, זה מה שקורה.(אלון) כן - ואז תתחיל להקריא את זה בשירות שאתה צריך בטלפון: “לאן לשלוח?” - “לAlon@Hey.com” - “מה?! H?” - אנשים לא מבינים, עזוב אותך, למי אכפת שלוש אותיות?(דותן) היה שם קטע נחמד - קיבלתי Invite יחסית מוקדם, אז הדבר הראשון שאתה עושה כשאתה מקבל Invite יותר מוקדם מכולם זה לנסות לתפוס שמות . . .אז יש שם קטע נחמד של מעט אותיות - נגיד, שתי אותיות זה סכום מטורף, אבל שלוש אותיות זה כבר $350 לשנה, נדמה לי - ואז אתה כבר מתחיל לתהות . . .כמובן שניסיתי “DHH” - לא היה . . . ואז ניסיתי DNH, שזה קצת דומה ל-DHH - וכן היה.אז סתם - לידיעתם ה-fisher-ים שם בחוץ, אפשר לעשות דברים מעניינים . . . אבל לא - לא שילמתי(רן) לא שילמת $350?(דותן) לא - לא הלכתי על זה(אלון) היה פעם למישהו סקריפט שתופס שמות קצרים ב-Twitter, אבל בוא נעצור פה.(רן) אני כבר רואה את הבלוג-פוסט הבא: “אתה קונה שם בשלוש אותיות ב-$350 - וזה לא עובד!”(דותן) “com.”(רן) טוב, קצת חרגנו - הגיע הזמן לקטע של המצחיקולים, כדי להקל על האווירה אחרי הדרמה הרצינית הזאת שקרתה פה . . .הראשון - טוויט של bradfitz, אחד המפתחים המפורסמים בעולם - היה בצוות ה-Core של Go, כתב את Memcached בעבר,ועודהוא כתב ב-Twitter שהוא אחרי יום ארוך של ראיונות ורוצה להוציא את התסכול שלו - אז הנה השאלה: “Print the largest even integer in an array of integers.” - וספקו לי אך ורק תשובות שגויותוזה ניהיה מצחיק . . . אנשים הציעו כל מיני רעיונות לאיך להדפיס את המספר הזוגי הגדול ביותר במערך של Integersלמשל - תשובה אחת זה “(print(a” - פשוט להדפיס את כל מערך, והמספר הזוגי הגדול ביותר כנראה יודפס שם . . . זה עובד.תשובה נוספת - לעשות לולאה בין 0 ל-MaxInt ולהדפיס את כל המספרים - וגם במקרה הזה המספר הזוגי הגדול ביותר במערך כנראה יודפס איפשהו שם.בקיצור - היו כל מיני תשובות מתחכמות כמו “קודם כל צריך ליצור מודל ואז לאמן אותו” והייתה תשובה ב-Shell עם Grep ו-Sort . . . בקיצור, כל מיני תשובות מאוד משעשעות, מוזמנים לעבור על ה-Thread ב-Twitterוכן - חלק גם נתנו רפרנסים לתשובות ב-Stack Overflow. . . עשו מזה מטעמים. נחמד, משעשע.אייטם הבא - ypp, או: Yid++ כמו שהם כותבים - the oylem’s first programming shprachמי שיודע פה יידיש - מוזמנים לתרגם . . .וכן - Yid++ זה בעצם Compiler מיידיש ל-++C, אם אני לא טועהזה למעשה ה-Compiler הראשון בעולם, או משהו כזהאתם מוזמניםללכת לקרוא Source Code של Yid++, למשל - למשל - be_soymech_on זה (Include (iostreamו- holding shitta std זה (namespace(std - למי שזוכר את ה-++C בטח יראה את הדמיוןיש גם -bli_ayin_hara main () bh שזה בעצם (void (main, והוא מחזיר בעצם “bh”, שאני מניח שזה “בעזרת השם”ולמעלה כמובן כתוב בגדול “BSD” - שזה “בסיעתה דשמייא” כמובן . . .(דותן) זה גם מבלבל מבחינה לגאלית . . .(רן) אני בטוח שזה לא יד המקרה . . .(אלון) מעניין האם זה מתקמפל בשבת . . .(דותן) לקחת לי! אני כבר מחכה להגיד את זה!(אלון) סליחה, אתה יכול למחוק את המשפט האחרון שלי? (לא) - דותן, מה רצית להגיד?(דותן) האם זה מתקמפל בשבת? האם ה-Compiler יעבוד בשבת?(רן) בואו נקרא עוד קצת פנינים מהשפה - למשל - >>be_machriz זה >>cout, להדפסהיש פה עוד איזו מילה ביידיש שאני לא מזהה . . .בקיצור - משעשע(דותן) בינתיים אני גם מסתכל בקוד - וצריך לפרגן פה לבנאדם שכתב את זה: בחור בשם משה שור מחיפה, מהטכניון - קל”ב אליך . . .יש פה גם דברים מגניבים בקוד, כמו קובץ ++C שנקרא ani_maymin.cpp . . . בקיצור, גם הקוד עמוס בדברים כאלהזה כל כך חזק, שאני מאמין כבר שזה אמיתי . . . אני רואה שיש כאן הכשר מאיזשהו רב ל - Code base . . . זה מתחיל להיות כבר . . . צריך לבדוק את זה איתו.(רן) יש תעודת הכשר לקוד, יפה, הלך עם זה עד הסוף - כל הכבוד, משה!(רן) אני רואה שיש פה שניים - משה וגם עוד מישהו שתרם - יחיאל קלמנסון, שהוא דווקא מניו-יורק(דותן) אני חושב שזה אמיתי, זה נראה לי אמיתי, זו באמת שפה כשרה . . .(רן) לגמרי - סחטיין על העבודה חברים, אם אתם שומעים אותנווזהו - אחלה צחוק, תקראו קצת את הקוד, אני בטוח שתזהו הרבה יידיש גם אם אתם לא דוברי יידיש שוטף, בטוח שתזהו הרבה.זהו - זה הכל, כאן אנחנו מסיימים.תודה לכם אלון ודותן, היה משעשע ומחכים כרגיל - נתראה בפעם הבאה.הקובץ נמצא כאן, האזנה נעימה ותודה רבה לעופר פורר על התמלול

Brian Dorsey and Mark Mirchandani team up this week to speak with Thomas Jung and Lucia Subatin about SAP. SAP, the company that builds software and other technology components, is probably best known for their Enterprise Resource Planning software that helps businesses with everything from accounting to order management. Their Customer Relationship Management software helps companies with things like marketing campaign management and sales, while SAP’s Supplier Relationship Management software helps clients manage large supply chains. Thomas explains how companies can get started with SAP products and integrate them into their current systems. Once SAP products are employed, clients benefit from a better understanding of their complete business and a more efficient company. SAP developers must have great communication skills, as they take SAP products and tailor them to each individual company, whether on-prem or in the cloud. Later, Lucia and Thomas describe instances when clients may want to take their core businesses and enhance them with technologies like AI and how this is possible. To wrap up the episode, Thomas and Lucia introduce us to SAP products like HANA, their Cloud Application Programming Model, and security measures. Thomas Jung Thomas Jung is Head of Developer Advocacy - a team within the SAP Developer & Community Relations organization. The Developer Advocates inform and educate about SAP and related development technologies and also act as the voice of external developers within the SAP organization. Lucia Subatin Chocolate, cats, computers. Lucia spends a lot of time with the computer, either solving problems or starting trouble. She likes to architect technology solutions to help enterprises run more efficiently. Her cats and chocolate help her bring innovations. Cool things of the week Google Cloud learning resources at no cost for 30 days blog Powering up caching with Memorystore for Memcached blog Interview SAP site OData site Pub/Sub site Cloud Knative site SAP TechEd site Cloud Run site Google Cloud Platform Podcast Episode 166: SAP HANA with Lucia Subatin and Kevin Nelson podcast Qwiklabs site Question of the week How do I get started with caching? Why should I cache? Brian tells us more about caching, Memcached, and Redis. Where can you find us next? Brian is taking it day by day right now. Mark will be making more videos!

Jellyfin has its biggest release yet, and Memcached sees a significant update. Plus the Debian Project Leadership contest has begun, and the report spreading misleading information about the distro.

Jellyfin has its biggest release yet, and Memcached sees a significant update. Plus the Debian Project Leadership contest has begun, and the report spreading misleading information about the distro.

פרק מספר 65 של באמפרס (385 למניין רברס עם פלטפורמה) - רן, אלון ודותן עם סקירה של טכנולוגיות ודברים מעניינים מהזמן האחרון.רן - חברת Shopify (חברת eCommerce קנדית) מציגה קונספט מעניין של Dev Degree: מציעים למפתחים בתחילת דרכם (לפני השלמת התואר) לעבוד אצלם, כשהם הם מממנים את הסיפור הזה.שיתןף פעולה עם כמה אוניברסיטאות באיזור, כשהסטודנטים לומדים ועובדים במקביל, במימון החברה.עוד סינגל לכך שחברות צמאות לכוח אדם ומוכנות לשלם הרבה עבור זה - גם בכסף וגם בזמן - Internship מאוד ארוך, במקרה הזה של 4 שנים.בעיית כוח האדם קיימת בכל העולם ומעניין לראות פתרונות יצירתיים - יש הרבה תוכניות של העסקת סטודנטים אבל זה בהחלט די קיצוני.מעניין אם יוזמה כזו תוכל להתרומם גם בישראל.אחד השחקנים המובילים של Go - איש בשם Brad Fitzpatrick - עוזב את Google ולמעשה עוזב את ה Core Development Team של Go.מדובר במפתח עם הרבה מאוד קרדיט בעולם של Go וגם הרבה לפני כן.משתמשים ב Memcached? אז זה קוד שלו מאחת החברות הקודמות.כתב עוד לא מעט מערכות מבוזרות מעניינות.עכשיו החליט שהספיק לו - אחרי 12 שנים וחמישה חודשים . . .הבלוג כולל עוד כל מיני סטטיסטיקות מעניינות שהוא אסף על התקופה הזו - Commits, Code reviews ועוד.אחד האנשים שכתב הרבה מהסביבות הפנימיות, ומשאיר את ה-Community במצב מאוד טוב ובריא - אומר שיהיה נחמד להשתמש בשפה ולא רק לפתח אותה.מצד אחד עצוב, מצד שני מגיע לו הרבה קרדיט על תרומה משמעותית.ולענייני Security - רן קיבל לפני כמה שבועות שיחה מעניינת ממספר פרטי . . .שיחה באנגלית, לא נשמע בריטי או אמריקאי, מבטא כבד . . . נשמע כמו התחלה של שיחה על כנס כלשהו.הדובר עדכן על כך ש”מחשב ה-Windows” ככל הנראה נפרץ ומבצע “פעילות חשודה”. האדם שהתקשר אמר שהוא “מטעם Windows” וביקש לבצע מספר פעולות.בהתחלה נשמע הזוי ו”על מי מנסים לעבוד?” - ומצד שני אולי יהיה נחמד לשחק עם זה קצת… לא שווה את השיחה?רן המשיך ומדי פעם שאל שוב מהיכן מגיעה השיחה (מה זאת אומרת? “מ-Windows!”, בסיאטל…)מישהו ב-Microsoft כנראה היה עונה Redmond“איך אתה יודע שהמחשב שלי?” - “הוא רשום על שמך”. מוזר, לא היה כנראה מחשב Windows רשום על רן כבר הרבה שנים…ואז היה צריך לגשת לאיזשהו אתר - ואז הגיעה שיחה אחרת יותר חשובה והיה צריך להפסיק את המשחק.כנראה שהשלב הבא היה להוריד משהו ולהריץ אותו - ואז אכן המחשב יהיה פרוץ. סוג של חיזוי עתידות…לעיניינו - השקעה די מרשימה: מצאו שם וטלפון, חייגו מחוץ לישראל (לא בטוח - בהמשך האייטם)האם זה משתלם? מה אחוזי ההצלחה ומה עושים עם זה?(דותן) קרה לנו מקרה דומה לפני שנה ולפני שנתיים - ותמיד אלו היו שיחות אל מישהו שמתעסק ב-DevOps או Infrastructure או משהו קשור - יכול להיות שמטרגטים מקצועות?ולמשהו יותר חמור - פוסט של רן בר זיק, שבו הוא מתאר את הכשלים שהיו באפליקצית אלקטור היה הרבה בחדשות, אז רק תקציר - מדובר באפליקציה שנמצאת בשימוש מפלגת הליכוד על מנת לנהל את הקמפיין ולהיות בקשר עם הבוחריםמדובר במעיין פורטל / אפליקציית ווב שבמאגרי המידע שלה יש גישה לכל מאגר הבוחרים בישראל - משהו כמו 6.5 מיליון אזרחים, כתובות מספרי טלפון . . .  כל זה כל כך קל לפריצה שזה די גובל בפשע.שימוש בכלי הפריצה “דפדפן”, כפתור-ימיני View source ומשם מהר מאוד אל סיסמאות ה-Admin . . . זה ספציפית כבר לא קיים (?), אבל כנראה היה פתוח לא מעט זמן ואפשר פשוט להוריד את כל פנקס הבוחרים של ישראל.יש על זה כמובן פרק ב-CyberCyber - האגרון 2: פנקס הבוחרים המלא דלף לרשת דרך הליכודאולי משם הגיעו הפרטים של רן מהאייטם הקודם?בכל אופן, רמת החשיפה של כל אזרחי ישראל הרבה יותר חמורה כרגע, כולל לא מעט פרטים - וזה כשל אבטחתי נוראי ברמה הלאומית, גם ברמת המידע שנחשף וגם בקלות בה ניתן היה להשיג אותו.כל כך קל שזה מכעיס. רק צריך ללבוש קפוצ’ון . . .עכשיו זו פשוט פצצת זמן שממתינה.(דותן) רגע - אז אפשר להצביע דרך הדבר הזה? להכניס רשומה . . .גאוני.אפשר לחשוב רגע על מגוון המחדלים שכנראה היו פה - כל מאגר הבוחרים נמצא במקום שמוגן בסך הכל ע”י סיסמא (ב-Clear text כמובן)ה-URL של הסיסמאות פשוט זמין בתוך אחד מעמודי ה-Web הלא מוגניםבגדול - ביטוי המפתח הוא “מישהו תכנן”. לא נראה ככה . . .המון דברים שגם אם עושים מהר וחפיף - מי שומר סיסמאות ב-Plain text בדף Web? מסתבר שיש.תעודת עניות להמון גורמים בשרשרת.חשבתם שזה נגמר? אחרי החשיפה: פרטי 6.4 מ' ישראלים ממשיכים לדלוףוזה פשוט לא מפסיק . . .לנשום עמוק . . .בשבוע שעבר התקיים כנס GopherconILהכנס התקיים כבר, אבל תוכלו לצפות בהרצאות המוקלטות, לכשיפורסמו.גם רן וגם אלון דיברו בכנסרן דיבר על go-grpc-channelz (דיברנו על זה בבאמפרס 61) - מספק visibility לתוך gRPC, שהיא מערכת RPC רובסטית וטובה, אבל גם מורכבת - והפרויקט מספק עוד Visibility, בעיקר לתוך ה-Clients (לאן מחובר, איך עובד ה-Load balancing, וכו’), והכל בממשק וובי די פשוט.מעיין הרצאת Ignite של 10 דקותאלון דיבר על Go is Getting Rusty - לאורך השנים דיברנו הרבה כאן על Go ו-Rustהאם צריך בכלל להשוות בינהן, קצת על מה זה Rust ולמה זה מעניין . . בקרוב הוידאו.אחת הדמויות הבולטות בעולם ה-Go נכח גם - Dave Cheneyביקר בישראל, היה ה-Keynote speaker וגם העביר סדנא שנזכיר עוד מעט.בעניין דומה ל Go ו-Rust - פוסט של חברת Discord שמדבר על למה הם עברו מ-Go ל-Rust באחד ה-Services שלהם.כתוב יפה, מסביר כל מיני מגבלות (ה-Garbage collection של Go…)הנושא לא מפתיע - כל מי שעובד עם שפה שהיא Garbage-collected חייב להיות מודע למגבלות של ה-Garbage collector. נכון שיש כאלו כמו של JVM שמאפשרות קצת Tuning ואולי במקרה של  Discord היה אפשר להשמש ב-Java או ב-Kotlin כדי לספק את הצרכים שלהם עם ה - Garbage collector, מה שב-Go כמעט שאי אפשר לעשות.בסופו של דבר, אם חשוב לכם Raw Performance ושלא יהיו עיכובים אפילו לא באחוזון ה-99 ועם Latency מאוד נמוך (פחות מ-1mS) - יכול להיות ששפה שהיא Garbage-collected לא תתאים לכם.צריך להכיר את המגבלות של הכלים - שפות שהן Garbage-collected בדרך כלל יותר קלות לפיתוח אבל יש להן מגבלות, וכאן הן מוצגות מאוד יפה ונראה שהמעבר ל-Rust מוצדק.(אלון) אני חושב שחלק מהעניין נובע מההשוואה של Go ל-++C, וזה לא כל כך נכון - אולי לפעמים מגיעים ל-Performance שמתקרב ל-++C, אבל זה לא שם בהרבה מקרים ולא באמת תחליף בהרבה מקומות.(רן) תלוי בהקשר - במקומות שבהם אתה כותב ++C אבל לא מגרד את יכולות הניהול זכרון אז זה לא כל כך משנה; במקרה הזה היה להם הרבה מאוד דאטה - Caches גדולים ואינטנסיביים אבל עם מעט מאוד מקרי Revocation - ואז ה - Garbage collector עדיין נאלץ לעבור על כל האובייקטים וזה לקח הרבה זמן.אלון - בהמשך ל Dave Cheney ו - GopherconIL - הוא העביר גם Workshop (יותר הרצאה-מאוד-ארוכה)כתב על זה- Practical Go: Real world advice for writing maintainable Go programsחלק מספר שהוא כותב - ואלו משהו כמו עשרת הפרקים הראשונים.מגוון נושאים - הצהרה על משתנים ושמות משתנים, מתי להשתמש במה, הערות, Style . . .מסביר איך עובד nil, שזה משהו קצת הזוי ב-Go (מסתבר שלא כל ה nil-ים זהים…)ועוד רשימה ארוכה . . . מאוד ארוך אבל מעיין - נכנס לאיך הוא רואה את הדברים וזו נקודת פתיחה מעניינת לכל מי שכותב ב-Go.(רן) גם השתתפתי בסדנא וקראתי (באמת ארוך) - כתוב יפה מאוד ומסביר דברים יפה ולא חוסך בדוגמאות.מה שכן - לא בהכרח הייתי ממליץ למי שרק עכשיו לומד את השפה אלא למי שכותב כבר כמה חודשים ואז הדברים יהיו יותר הגיוניים.הוא כותב על דברים שהם Battle-tested, ומי שרק לומד את השפה לא בהכרח צריך להבין את כל מקרי הקצה של טיפול ב-Channels למשל, אבל בהחלט קריאה מומלצת.יש כאן ענייני שמות משתנים, החלק של ה-nil ועוד כמה דברים שיכולים כן להיות רלוונטיים, ואולי אחר כך רק מי שכותב ממש ב-Go יכול להמשיך.שפת תכנות חדשה שמיקרוסופט הוציאו בשם veronaמוגדרת כ “Research programming language for concurrent ownership”לא ברור מה המוטיבציה לשפת Research חדשה, אבל לקחו הרבה עקרונות מ-Rust וניסו לבנות שפת Research.מדובר ב-Research במובן של שפות או בהקשר של Data Science? לא לגמרי ברור(דותן) נראה כמו מחקר על שפות תכנות - עוד סוג של Meta (שפת תכנות שחוקרת שפות תכנות, ד”ש ל Inception)(רן) חושב ש-Scala הייתה קצת כזו בתחילת הדרך, ומחבר השפה (Martin Odersky) אמר שיש בה כל מיני דברים שהוא תמיד רצה לנסות בשפות ועכשיו יש לו מגרש משחקים לנסות . . . אולי זה מסביר את זה.בכל מקרה, לא ברור אם יצא מזה משהו, אבל יש כאן הרבה עקרונות מ-Rust (וגם מ-Pony, שאנחנו פחות מכירים - מפתחים שהיו פעם חלק מחברה ואז המשיכו לבד, משהו כזה)המימוש של verona עצמה הוא ב++C.שירות של AWS בשם Amazon Braket - והפעם: Explore and experiment with quantum computingמה אפשר לעשות היום, כשחישוב קוונטי זה עדיין לא משהו עם הרבה יישומים בימים אלו?לא לגמרי ברור, אבל אולי מישהו מהתחום יביןנסו את Lex Fridman

Since re:Invent 2018, the Amazon ElastiCache team has been hard at work innovating on behalf of customers. In this session, we review the work that has been done in 2019 to make sure that ElastiCache is the most cost-effective and best performing Redis- and Memcached-compatible cloud service.

Simon and Nicki share a broad range of interesting updates! 00:48 Storage 01:43 Compute 05:27 Networking 16:07 Databases 12:03 Developer Tools 13:18 Analytics 19:06 IoT 20:42 Customer Engagement 21:03 End User Computing 22:31 Machine Learning 25:27 Application Integration 27:35 Management and Governance 29:17 Media 30:53 Security 32:56 Blockchain 33:14 Quick Starts 33:51 Training 36:11 Public Datasets 37:12 Robotics Shownotes: Topic || Storage AWS Snowball Edge now supports offline software updates for Snowball Edge devices in air-gapped environments | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-snowball-edge-now-supports-offline-software-updates-for-snowball-edge-devices-in-air-gapped-environments/ Topic || Compute Now Available: Amazon EC2 High Memory Instances with up to 24 TB of memory, Purpose-built to Run Large In-memory Databases, like SAP HANA | https://aws.amazon.com/about-aws/whats-new/2019/10/now-available-amazon-ec2-high-memory-instances-purpose-built-run-large-in-memory-databases/ Introducing Availability of Amazon EC2 A1 Bare Metal Instances | https://aws.amazon.com/about-aws/whats-new/2019/10/introducing-availability-of-amazon-ec2-a1-bare-metal-instances/ Windows Nodes Supported by Amazon EKS | https://aws.amazon.com/about-aws/whats-new/2019/10/windows-nodes-supported-by-amazon-eks/ Amazon ECS now Supports ECS Image SHA Tracking | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-ecs-now-supports-ecs-image-sha-tracking/ AWS Serverless Application Model feature support updates for Amazon API Gateway and more | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-serverless-application-model-feature-support-updates-for-amazon-api-gateway-and-more/ Queuing Purchases of EC2 RIs | https://aws.amazon.com/about-aws/whats-new/2019/10/queuing-purchases-of-ec2-ris/ Topic || Network AWS Direct Connect Announces the Support for Granular Cost Allocation and Removal of Payer ID Restriction for Direct Connect Gateway Association. | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-direct-connect-aws-direct-connect-announces-the-support-for-granular-cost-allocation-and-removal-of-payer-id-restriction-for-direct-connect-gateway-association/ AWS Direct Connect Announces Resiliency Toolkit to Help Customers Order Resilient Connectivity to AWS | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-direct-connect-announces-resiliency-toolkit-to-help-customers-order-resilient-connectivity-to-aws/ Amazon VPC Traffic Mirroring Now Supports AWS CloudFormation | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-vpc-traffic-mirroring-now-supports-aws-cloudformation/ Application Load Balancer and Network Load Balancer Add New Security Policies for Forward Secrecy with More Stringent Protocols and Ciphers | https://aws.amazon.com/about-aws/whats-new/2019/10/application-load-balancer-and-network-load-balancer-add-new-security-policies-for-forward-secrecy-with-more-strigent-protocols-and-ciphers/ Topic || Databases Amazon RDS on VMware is now generally available | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-rds-on-vmware-is-now-generally-available/ Amazon RDS Enables Detailed Backup Storage Billing | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-rds-enables-detailed-backup-storage-billing/ Amazon RDS for PostgreSQL Supports Minor Version 11.5, 10.10, 9.6.15, 9.5.19, 9.4.24, adds Transportable Database Feature | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-rds-for-postgresql-supports-minor-version-115-1010-9615-9515-9424-adds-transportable-database-feature/ Amazon ElastiCache launches self-service updates for Memcached and Redis Cache Clusters | https://aws.amazon.com/about-aws/whats-new/2019/10/elasticache-memcached-self-service-updates/ Amazon DocumentDB (with MongoDB compatibility) adds additional Aggregation Pipeline Capabilities including $lookup | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-documentdb-add-additional-aggregation-pipeline-capabilities/ Amazon Neptune now supports Streams to capture graph data changes | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-neptune-now-supports-streams-to-capture-graph-data-changes/ Amazon Neptune now supports SPARQL 1.1 federated query | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-neptune-now-supports-SPARQL-11-federated-query/ Topic || Developer Tools AWS CodePipeline Enables Setting Environment Variables on AWS CodeBuild Build Jobs | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-codepipeline-enables-setting-environment-variables-on-aws-codebuild-build-jobs/ AWS CodePipeline Adds Execution Visualization to Pipeline Execution History | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-codepipeline-adds-execution-visualization-to-pipeline-execution-history/ Topic || Analytics Amazon Redshift introduces AZ64, a new compression encoding for optimized storage and high query performance | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-redshift-introduces-az64-a-new-compression-encoding-for-optimized-storage-and-high-query-performance/ Amazon Redshift Improves Performance of Inter-Region Snapshot Transfers | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-redshift-improves-performance-of-inter-region-snapshot-transfers/ Amazon Elasticsearch Service provides option to mandate HTTPS | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-elasticsearch-service-provides-option-to-mandate-https/ Amazon Athena now provides an interface VPC endpoint | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-athena-now-provides-an-interface-VPC-endpoint/ Amazon Kinesis Data Firehose adds cross-account delivery to Amazon Elasticsearch Service | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-kinesis-data-firehose-adds-cross-account-delivery-to-amazon-elasticsearch-service/ Amazon Kinesis Data Firehose adds support for data stream delivery to Amazon Elasticsearch Service 7.x clusters | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-kinesis-data-firehose-adds-support-data-stream-delivery-amazon-elasticsearch-service/ Amazon QuickSight announces Data Source Sharing, Table Transpose, New Filtering and Analytical Capabilities | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-quicksight-announces-data-source-sharing-table-transpose-new-filtering-analytics-capabilities/ AWS Glue now provides ability to use custom certificates for JDBC Connections | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-glue-now-provides-ability-to-use-custom-certificates-for-jdbc-connections/ You can now expand your Amazon MSK clusters and deploy new clusters across 2-AZs | https://aws.amazon.com/about-aws/whats-new/2019/10/now-expand-your-amazon-msk-clusters-and-deploy-new-clusters-across-2-azs/ Amazon EMR Adds Support for Spark 2.4.4, Flink 1.8.1, and the Ability to Reconfigure Multiple Master Nodes | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-emr-adds-support-for-spark-2-4-4-flink-1-8-1-and-ability-to-reconfigure-multiple-master-nodes/ Topic || IoT Two New Solution Accelerators for AWS IoT Greengrass Machine Learning Inference and Extract, Transform, Load Functions | https://aws.amazon.com/about-aws/whats-new/2019/10/two-new-solution-accelerators-for-aws-iot-greengrass-machine-lea/ AWS IoT Core Adds the Ability to Retrieve Data from DynamoDB using Rule SQL | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-iot-core-adds-ability-to-retrieve-data-from-dynamodb-using-rule-sql/ PSoC 62 Prototyping Kit is now qualified for Amazon FreeRTOS | https://aws.amazon.com/about-aws/whats-new/2019/10/psoc-62-prototyping-kit-qualified-for-amazon-freertos/ Topic || Customer Engagement Amazon Pinpoint Adds Support for Message Templates | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-pinpoint-adds-support-for-message-templates/ Topic || End User Computing Amazon AppStream 2.0 adds support for 4K Ultra HD resolution on 2 monitors and 2K resolution on 4 monitors | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-appstream-2-adds-support-for-4k-ultra-hd-resolution-on-2-monitors-and-2k-resolution-on-4-monitors/ Amazon AppStream 2.0 Now Supports FIPS 140-2 Compliant Endpoints | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-appstream-2-now-supports-fips-140-2-compliant-endpoints/ Amazon Chime now supports screen sharing from Mozilla Firefox and Google Chrome without a plug-in or extension | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-chime-now-supports-screen-sharing-from-mozilla-firefox-and-google-chrome-without-a-plug-in-or-extension/ Topic || Machine Learning Amazon Translate now adds support for seven new languages - Greek, Romanian, Hungarian, Ukrainian, Vietnamese, Thai, and Urdu | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-translate-adds-support-seven-new-languages/ Introducing Amazon SageMaker ml.p3dn.24xlarge instances, optimized for distributed machine learning with up to 4x the network bandwidth of ml.p3.16xlarge instances | https://aws.amazon.com/about-aws/whats-new/2019/10/introducing-amazon-sagemaker-mlp3dn24xlarge-instances/ SageMaker Notebooks now support diffing | https://aws.amazon.com/about-aws/whats-new/2019/10/sagemaker-notebooks-now-support-diffing/ Amazon Lex Adds Support for Checkpoints in Session APIs | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-lex-adds-support-for-checkpoints-in-session-apis/ Amazon SageMaker Ground Truth Adds Built-in Workflows for the Verification and Adjustment of Data Labels | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-sagemaker-ground-truth-adds-built-in-workflows-for-verification-and-adjustment-of-data-labels/ AWS Chatbot Now Supports Notifications from AWS Config | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-chatbot-now-supports-notifications-from-aws-config/ AWS Deep Learning Containers now support PyTorch | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-deep-learning-containers-now-support-pytorch/ Topic || Application Integration AWS Step Functions expands Amazon SageMaker service integration | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-step-functions-expands-amazon-sagemaker-service-integration/ Amazon EventBridge now supports AWS CloudFormation | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-eventbridge-supports-aws-cloudformation/ Amazon API Gateway now supports access logging to Amazon Kinesis Data Firehose | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-api-gateway-now-supports-access-logging-to-amazon-kinesis-data-firehose/ Topic || Management and Governance AWS Backup Enhances SNS Notifications to filter on job status | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-backup-enhances-sns-notifications-to-filter-on-job-status/ AWS Managed Services Console now supports search and usage-based filtering to improve change type discovery | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-managed-services-console-now-supports-search-and-usage-based-filtering-to-improve-change-type-discovery/ AWS Console Mobile Application Launches Federated Login for iOS | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-console-mobile-application-launches-federated-login-for-ios/ Topic || Media Announcing New AWS Elemental MediaConvert Features for Accelerated Transcoding, DASH, and AVC Video Quality | https://aws.amazon.com/about-aws/whats-new/2019/10/announcing-new-aws-elemental-mediaconvert-features-for-accelerated-transcoding-dash-and-avc-video-quality/ Topic || Security Amazon Cognito Increases CloudFormation Support | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-cognito-increases-cloudformation-support/ Amazon Inspector adds CIS Benchmark support for Windows 2016 | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-inspector-adds-cis-benchmark-support-for-windows-2016/ AWS Firewall Manager now supports management of Amazon VPC security groups | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-firewall-manager-now-supports-management-of-amazon-vpc-security-groups/ Amazon GuardDuty Adds Three New Threat Detections | https://aws.amazon.com/about-aws/whats-new/2019/10/amazon-guardduty-adds-three-new-threat-detections/ Topic || Block Chain New Quick Start deploys Amazon Managed Blockchain | https://aws.amazon.com/about-aws/whats-new/2019/10/new-quick-start-deploys-amazon-managed-blockchain/ Topic || AWS Quick Starts New Quick Start deploys TIBCO JasperReports Server on AWS | https://aws.amazon.com/about-aws/whats-new/2019/10/new-quick-start-deploys-tibco-jasperreports-server-on-aws/ Topic || Training New Training Courses Teach New APN Partners to Better Help Their Customers | https://aws.amazon.com/about-aws/whats-new/2019/10/new-training-courses-teach-new-apn-partners-to-better-help-their-customers/ New Courses Available to Help You Grow and Accelerate Your AWS Cloud Skills | https://aws.amazon.com/about-aws/whats-new/2019/10/new-courses-available-to-help-you-grow-and-accelerate-your-aws-cloud-skills/ New Digital Course on Coursera - AWS Fundamentals: Migrating to the Cloud | https://aws.amazon.com/about-aws/whats-new/2019/10/new-digital-course-on-coursera-aws-fundamentals-migrating-to-the-cloud/ Topic || Public Data Sets New AWS Public Datasets Available from Audi, MIT, Allen Institute for Cell Science, Finnish Meteorological Institute, and others | https://aws.amazon.com/about-aws/whats-new/2019/10/new-aws-public-datasets-available/ Topic || Robotics AWS RoboMaker introduces support for Robot Operating System 2 (ROS2) in beta release | https://aws.amazon.com/about-aws/whats-new/2019/10/aws-robomaker-introduces-support-robot-operating-system-2-beta-release/

@TheBrianFink -- LINKEDIN:  With 10+ years of specialized recruiting experience, I've helped a wide range of professionals ranging from entry level candidates (new grads) to C-Suite executives take the next step up in their careers and stretch their professional capabilities. As a true "head-hunter"?, I excel at direct recruiting, direct sourcing, and aligning opportunities to professionals. Working with client companies, I specializing in finding outstanding recruiting, sales, marketing, and technical professionals. My abilities include: * C-Suite (CEO, CAO, CIO, CTO, COO) recruiting and research; * Staffing companies at scale and building to fit culture; * Retained executive and performance research based full-cycle recruiting; * Design and implement national staffing strategies & candidate assessment profiles; * Understanding hiring needs and pairing them with recruiting strategies; * Designing Campus/University relations & recruiting; * Building & training highly effective recruiting teams; * Increasing company brand exposure; * Research and candidate identification/sourcing methods. * Found Carmen Sandiego! * I've been told I execute a mean high-five and I'm passionate about the adverse qualities of glitter =D Keywords: Java, JavaScript, NoSQL (Solr, Memcached, Voldemort, Cassandra, Redis), SQL, Struts, Caching (Akamai, Apache Traffic Server), Kafka, Spark, R, Servlets, WebObjects, Junit, XML, JSP, Ruby, Clojure, Velocity, REST, APIs, UI, Framework, React, Ember, Angular, Node.js, Freemarker, CSS, HTML, Mac OS X, Ubuntu, CentOS, Red Hat, Windows.   

Fixes for 19 different vulnerabilities across MySQL, Dovecot, Memcached and others, plus we talk to Joe McManus about the recent iLnkP2P IoT hack and the compromise of DockerHub's credentials database and more.

Followup Benchmarks of Memcached on multiple SSDs EC2 Graviton benchmarks - Phoronix Some DL problems are intractable 7nm Complete Coffee Lake CPU list - Anandtech Available in 2019 iMacs Jetson Nano Google Stadia - The Verge Netflix Open Connect ESRGAN texture packs Playable raytraced Quake 2 Article HN Discussion Battlefield V RTX Demo RTX On/Off Meme, fellow kids ARM Helium vector extensions Article GradientFlow - Training ImageNet in 1.5 Minutes on 512 GPUs Thinkpad X210 Aftershow Microsft Wall of GPU History Farming Simulator peripherals Cup throwing robots

In this session, we provide a behind the scenes peek to learn about the design and architecture of Amazon ElastiCache. See common design patterns with our Redis and Memcached offerings and how customers use them for in-memory data processing to reduce latency and improve application throughput. We review ElastiCache best practices, design patterns, and anti-patterns. Complete Title: AWS re:Invent 2018: [REPEAT 1] ElastiCache Deep Dive: Design Patterns for In-Memory Data Stores (DAT302-R1)

16.06.2018 BSidesKharkiv https://kharkiv.securitybsides.org.ua/ 07.06.2018 OWASP Odesa https://www.facebook.com/events/2104923576405410/ 07.07.2018 BSidesOdessa https://odessa.securitybsides.org.ua/ Kostiantyn Korsun про NoNameCon https://www.facebook.com/kostiantyn.korsun/posts/840821456102957 EFAIL https://efail.de/ Efail: Breaking S/MIME and OpenPGP Email Encryption using Exfiltration Channels (draft 0.9.1) https://efail.de/efail-attack-paper.pdf ProtonMail is safe against the efail PGP vulnerability. https://twitter.com/ProtonMail/status/995996112526954496 Efail or OpenPGP is safer than S/MIME https://lists.gnupg.org/pipermail/gnupg-users/2018-May/060315.html Digital Photocopiers Loaded With Secrets https://www.cbsnews.com/news/digital-photocopiers-loaded-with-secrets/ Throwhammer: Rowhammer Attacks over the Network and Defenses https://www.cs.vu.nl/~herbertb/download/papers/throwhammer_atc18.pdf Rowhammer strikes networks, Bolton strikes security jobs, and Nigel Thornberry strikes Chrome, and more http://www.theregister.co.uk/2018/05/12/security_roundup/ Memcached https://memcached.org/ 7-Zip: From Uninitialized Memory to Remote Code Execution https://landave.io/2018/05/7-zip-from-uninitialized-memory-to-remote-code-execution/ IBM bans all removable storage, for all staff, everywhere http://www.theregister.co.uk/2018/05/10/ibm_bans_all_removable_storage_for_all_staff_everywhere/ Second wave of Spectre-like CPU security flaws won't be fixed for a while http://www.theregister.co.uk/2018/05/09/spectr_ng_fix_delayed/ Every major OS maker misread Intel's docs. Now their kernels can be hijacked or crashed http://www.theregister.co.uk/2018/05/09/intel_amd_kernel_privilege_escalation_flaws/ Ex-CIA man fingered as prime suspect in Vault 7 spy tool manuals leak http://www.theregister.co.uk/2018/05/15/vault_7_leak/ DHCP Client Script Code Execution Vulnerability - CVE-2018-1111 https://access.redhat.com/security/vulnerabilities/3442151 Securit13 Patreon https://www.patreon.com/securit13 Keygen Music [2+ hour Mix] https://www.youtube.com/watch?v=cYkaG5CT53I

Allan’s recap of the ZFS User conference, first impressions of OmniOS by a BSD user, Nextcloud 13 setup on FreeBSD, OpenBSD on a fanless desktop computer, an intro to HardenedBSD, and DragonFlyBSD getting some SMP improvements. Headlines ZFS User Conference Recap Attendees met for breakfast on the fourth floor, in a lunchroom type area just outside of the theatre. One entire wall was made of lego base plates, and there were buckets of different coloured lego embedded in the wall. The talks started with Matt Ahrens discussing how the 2nd most requested feature of ZFS, Device Removal, has now landed, then pivoting into the MOST requested feature, RAID-Z expansion, and his work on that so far, which included the first functional prototype, on FreeBSD. Then our friend Calvin Hendryx-Parker presented how he solves all of his backup headaches with ZFS. I provided him some helpful hints to optimize his setup and improve the throughput of his backups Then Steven Umbehocker of OSNEXUS talked about their products, and how they manage large numbers of ZFS nodes After a very nice lunch, Orlando Pichardo of Micron talked about the future of flash, and their new 7.5TB SATA SSDs. Discussion of these devices after the talk may lead to enhancements to ZFS to better support these new larger flash devices that use larger logical sector sizes. Alek Pinchuk of Datto talked about Pool Layout Considerations then Tony Hutter of LLNL talked about the release process for ZFS on Linux Then Tom Caputi of Datto presented: Helping Developers Help You, guidance for users submitting bug reports, with some good and bad examples Then we had a nice cocktail party and dinner, and stayed late into the night talked about ZFS The next day, Jervin Real of Percona, presented: ZFS and MySQL on Linux, the Sweet Spots. Mostly outlining some benchmark they had done, some of the results were curious and some additional digging may turn up enhancements that can be made to ZFS, or just better tuning advice for high traffic MySQL servers. Then I presented my ZSTD compression work, which had been referenced in 2 of the previous talks, as people are anxious to get their hands on this code. Lastly, Eric Sproul of Circonus, gave his talk: Thank You, ZFS. It thanked ZFS and its Community for making their companies product possible, and then provided an update to his presentation from last year, where they were having problems with extremely high levels of ZFS fragmentation. This also sparked a longer conversation after the talk was over. Then we had a BBQ lunch, and after some more talking, the conference broke up. Initial OmniOS impressions by a BSD user I had been using FreeBSD as my main web server OS since 2012 and I liked it so much that I even contributed money and code to it. However, since the FreeBSD guys (and gals) decided to install anti-tech feminism, I have been considering to move away from it for quite some time now. As my growing needs require stronger hardware, it was finally time to rent a new server. I do not intend to run FreeBSD on it. Although the most obvious choice would be OpenBSD (I run it on another server and it works just fine), I plan to have a couple of databases running on the new machine, and database throughput has never been one of OpenBSD's strong points. This is my chance to give illumos another try. As neither WiFi nor desktop environments are relevant on a no-X11 server, the server-focused OmniOS seemed to fit my needs. My current (to be phased out) setup on FreeBSD is: apache24 with SSL support, running five websites on six domains (both HTTP and HTTPS) a (somewhat large) Tiny Tiny RSS installation from git, updated via cronjob sbcl running a daily cronjob of my Web-to-RSS parser an FTP server where I share stuff with friends an IRC bouncer MariaDB and PostgreSQL for some of the hosted services I would not consider anything of that too esoteric for a modern operating system. Since I was not really using anything mod_rewrite-related, I was perfectly ready to replace apache24 by nginx, remembering that the prepackaged apache24 on FreeBSD did not support HTTPS out of the box and I had ended up installing it from the ports. That is the only change in my setup which I am actively planning. So here's what I noticed. First impressions: Hooray, a BSD boot loader! Finally an operating system without grub - I made my experiences with that and I don't want to repeat them too often. It is weird that the installer won't accept "mydomain.org" as a hostname but sendmail complains that "mydomain" is not a valid hostname right from the start, OmniOS sent me into Maintenance Mode to fix that. A good start, right? So the first completely new thing I had to find out on my new shiny toy was how to change the hostname. There is no /etc/rc.conf in it and hostname mydomain.org was only valid for one login session. I found out that the hostname has to be changed in three different files under /etc on Solaris - the third one did not even exist for me. Changing the other two files seems to have solved this problem for me. Random findings: ~ I was wondering how many resources my (mostly idle) new web server was using - I always thought Solaris was rather fat, but it still felt fast to me. Ah, right - we're in Unixland and we need to think outside of the box. This table was really helpful: although a number of things are different between OmniOS and SmartOS, I found out that the *stat tools do what top does. I could probably just install top from one of the package managers, but I failed to find a reason to do so. I had 99% idle CPU and RAM - that's all I wanted to know. ~ Trying to set up twtxt informed me that Python 3.6 (from pkgin) expects LANG and LC_ALL to be set. Weird - did FreeBSD do that for me? It's been a while ... at least that was easy to fix. ~ SMF - Solaris's version of init - confuses me. It has "levels" similar to Gentoo's OpenRC, but it mostly shuts up during the boot process. Stuff from pkgsrc, e.g. nginx, comes with a description how to set up the particular service, but I should probably read more about it. What if, one day, I install a package which is not made ready for OmniOS? I'll have to find out how to write SMF scripts. But that should not be my highest priority. ~ The OmniOS documentation talks a lot about "zones" which, if I understand that correctly, mostly equal FreeBSD's "jails". This could be my chance to try to respect a better separation between my various services - if my lazyness won't take over again. (It probably will.) ~ OmniOS's default shell - rather un-unixy - seems to be the bash. Update: I was informed about a mistake here: the default shell is ksh93, there are bogus .bashrc files lying around though. ~ Somewhere in between, my sshd had a hiccup or, at least, logging into it took longer than usual. If that happens again, I should investigate. Conclusion: By the time of me writing this, I have a basic web server with an awesome performance and a lot of applications ready to be configured only one click away. The more I play with it, the more I have the feeling that I have missed a lot while wasting my time with FreeBSD. For a system that is said to be "dying", OmniOS feels well-thought and, when equipped with a reasonable package management, comes with everything I need to reproduce my FreeBSD setup without losing functionality. I'm looking forward to what will happen with it. DigitalOcean http://do.co/bsdnow [Open Source Hardware Camp 2018 — Sat 30/06 & Sun 01/07, Lincoln, UK (includes 'Open-source RISC-V core quickstart' and 'An introductory workshop to NetBSD on embedded platforms')](http://oshug.org/pipermail/oshug/2018-April/000635.html) ``` Hi All, I'm pleased to announce that we have 10 talks and 7 workshops confirmed for Open Source Hardware Camp 2018, with the possibility of one or two more. Registration is now open! For the first time ever we will be hosting OSHCamp in Lincoln and a huge thanks to Sarah Markall for helping to make this happen. As in previous years, there will be a social event on the Saturday evening and we have a room booked at the Wig and Mitre. Food will be available. There will likely be a few of us meeting up for pre-conference drinks on the Friday evening also. Details of the programme can be found below and, as ever, we have an excellent mix of topics being covered. Cheers, Andrew ``` Open Source Hardware Camp 2018 On the 30th June 2018, 09:00 Saturday morning - 16:00 on the Sunday afternoon at The Blue Room, The Lawn, Union Rd, Lincoln, LN1 3BU. Registration: http://oshug.org/event/oshcamp2018 Open Source Hardware Camp 2018 will be hosted in the historic county town of Lincoln — home to, amongst others, noted engine builders Ruston & Hornsby (now Siemens, via GEC and English Electric). Lincoln is well served by rail, reachable from Leeds and London within 2-2.5 hours, and 4-5 hours from Edinburgh and Southampton. There will be a social at the Wig and Mitre on the Saturday evening. For travel and accommodation information information please see the event page on oshug.org. News Roundup Nextcloud 13 on FreeBSD Today I would like to share a setup of Nextcloud 13 running on a FreeBSD system. To make things more interesting it would be running inside a FreeBSD Jail. I will not describe the Nextcloud setup itself here as its large enough for several blog posts. Official Nextcloud 13 documentation recommends following setup: MySQL/MariaDB PHP 7.0 (or newer) Apache 2.4 (with mod_php) I prefer PostgreSQL database to MySQL/MariaDB and I prefer fast and lean Nginx web server to Apache, so my setup is based on these components: PostgreSQL 10.3 PHP 7.2.4 Nginx 1.12.2 (with php-fpm) Memcached 1.5.7 The Memcached subsystem is least important, it can be easily changed into something more modern like Redis for example. I prefer not to use any third party tools for FreeBSD Jails management. Not because they are bad or something like that. There are just many choices for good FreeBSD Jails management and I want to provide a GENERIC example for Nextcloud 13 in a Jail, not for a specific management tool. Host Lets start with preparing the FreeBSD Host with needed settings. We need to allow using raw sockets in Jails. For the future optional upgrades of the Jail we will also allow using chflags(1) in Jails. OpenBSD on my fanless desktop computer You asked me about my setup. Here you go. I’ve been using OpenBSD on servers for years as a web developer, but never had a chance to dive in to system administration before. If you appreciate the simplicity of OpenBSD and you have to give it a try on your desktop. Bear in mind, this is a relatively cheap ergonomic setup, because all I need is xterm(1) with Vim and Firefox, I don’t care about CPU/GPU performance or mobility too much, but I want a large screen and a good keyboard. Item Price, USD Zotac CI527 NANO-BE $371 16GB RAM Crucial DDR4-2133 $127 250GB SSD Samsung 850 EVO $104 Asus VZ249HE 23.8" IPS Full HD $129 ErgoDox EZ V3, Cherry MX Brown, blank DCS $325 Kensington Orbit Trackball $33 Total $1,107 OpenBSD I tried few times to install OpenBSD on my MacBooks—I heard some models are compatible with it,—but in my case it was a bit of a fiasco (thanks to Nvidia and Broadcom). That’s why I bought a new computer, just to be able to run this wonderful operating system. Now I run -stable on my desktop and servers. Servers are supposed to be reliable, that’s obvious, why not run -current on a desktop? Because -stable is shipped every six months and I that’s is often enough for me. I prefer slow fashion. iXsystems iX Ad Spot NAB 2018 – Michael Dexter’s Recap Introduction to HardenedBSD World HardenedBSD is a security enhanced fork of FreeBSD which happened in 2014. HardenedBSD is implementing many exploit mitigation and security technologies on top of FreeBSD which all started with implementation of Address Space Layout Randomization (ASLR). The fork has been created for ease of development. To cite the https://hardenedbsd.org/content/about page – “HardenedBSD aims to implement innovative exploit mitigation and security solutions for the FreeBSD community. (…) HardenedBSD takes a holistic approach to security by hardening the system and implementing exploit mitigation technologies.” Most FreeBSD enthusiasts know mfsBSD project by Martin Matuska – http://mfsbsd.vx.sk/ – FreeBSD system loaded completely into memory. The mfsBSD synonym for the HardenedBSD world is SoloBSD – http://www.solobsd.org/ – which is based on HardenedBSD sources. One may ask how HardenedBSD project compared to more well know for its security OpenBSD system and it is very important question. The OpenBSD developers try to write ‘good’ code without dirty hacks for performance or other reasons. Clean and secure code is most important in OpenBSD world. The OpenBSD project even made security audit of all OpenBSD code available, line by line. This was easier to achieve in FreeBSD or HardenedBSD because OpenBSD code base its about ten times smaller. This has also other implications, possibilities. While FreeBSD (and HardenedBSD) offer many new features like mature SMP subsystem even with some NUMA support, ZFS filesystem, GEOM storage framework, Bhyve virtualization, Virtualbox option and many other new modern features the OpenBSD remains classic UNIX system with UFS filesystem and with very ‘theoretical’ SMP support. The vmm project tried to implement new hypervisor in OpenBSD world, but because of lack of support for graphics its for OpenBSD, Illumos and Linux currently, You will not virtualize Windows or Mac OS X there. This is also only virtualization option for OpenBSD as there are no Jails on OpenBSD. Current Bhyve implementation allows one even to boot latest Windows 2019 Technology Preview. A HardenedBSD project is FreeBSD system code base with LOTS of security mechanisms and mitigations that are not available on FreeBSD system. For example entire lib32 tree has been disabled by default on HardenedBSD to make it more secure. Also LibreSSL is the default SSL library on HardenedBSD, same as OpenBSD while FreeBSD uses OpenSSL for compatibility reasons. Comparison between LibreSSL and OpenSSL vulnerabilities. https://en.wikipedia.org/wiki/LibreSSL#Security https://wiki.freebsd.org/LibreSSL#LibreSSL.28andOpenSSL.29SecurityVulnerabilities One may see HardenedBSD as FreeBSD being successfully pulled up to the OpenBSD level (at least that is the goal), but as FreeBSD has tons more code and features it will be harder and longer process to achieve the goal. As I do not have that much competence on the security field I will just repost the comparison from the HardenedBSD project versus other BSD systems. The comparison is also available here – https://hardenedbsd.org/content/easy-feature-comparison – on the HardenedBSD website. Running my own git server Note: This article is predominantly based on work by Hiltjo Posthuma who you should read because I would have spent far too much time failing to set things up if it wasn’t for their post. Not only have they written lots of very interesting posts, they write some really brilliant programs Since I started university 3 years ago, I started using lots of services from lots of different companies. The “cloud” trend led me to believe that I wanted other people to look after my data for me. I was wrong. Since finding myself loving the ethos of OpenBSD, I found myself wanting to apply this ethos to the services I use as well. Not only is it important to me because of the security benefits, but also because I like the minimalist style OpenBSD portrays. This is the first in a mini-series documenting my move from bloated, hosted, sometimes proprietary services to minimal, well-written, free, self-hosted services. Tools & applications These are the programs I am going to be using to get my git server up and running: httpd(8) acme-client(1) git(1) cgit(1) slowcgi(8) Setting up httpd Ensure you have the necessary flags enabled in your /etc/rc.conf.local: Configuring cgit When using the OpenBSD httpd(8), it will serve it’s content in a chrooted environment,which defaults to the home directory of the user it runs as, which is www in this case. This means that the chroot is limited to the directory /var/www and it’s contents. In order to configure cgit, there must be a cgitrc file available to cgit. This is found at the location stored in $CGIT_CONFIG, which defaults to /conf/cgitrc. Because of the chroot, this file is actually stored at /var/www/conf/cgitrc. Beastie Bits My Penguicon 2018 Schedule sigaction: see who killed you (and more) Takeshi steps down from NetBSD core team after 13 years DragonFlyBSD Kernel Gets Some SMP Improvements – Phoronix Writing FreeBSD Malware Tarsnap ad Feedback/Questions Troels - Question regarding ZFS xattr Mike - Sharing your screen Wilyarti - Adlocking on FreeBSD Brad - Recommendations for snapshot strategy Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

White House blasts Russia for NotPetya cyberattack https://edition.cnn.com/2018/02/15/politics/white-house-russia-notpetya/index.html  Memcached servers can be hijacked for massive DDoS attacks https://www.networkworld.com/article/3258772/security/memcached-servers-can-be-hijacked-for-massive-ddos-attacks.html  Memcrashed - Major amplification attacks from UDP port 11211 https://blog.cloudflare.com/memcr ashed-major-amplification-attacks-from-port-11211/ GITHUB SURVIVED THE BIGGEST DDOS ATTACK EVER RECORDED https://www.wired.com/story/github-ddos-memcached/amp  NETSCOUT Arbor Confirms 1.7 Tbps DDoS Attack; The Terabit Attack Era Is Upon Us https://www.arbornetworks.com/blog/asert/netscout-arbor-confirms-1-7-tbps-ddos-attack-terabit-attack-era-upon-us/  У Харкові засуджено підозрюваного за продаж клієнтської бази поштового перевізника https://cyberpolice.gov.ua/news/u-xarkovi-zasudzheno-pidozryuvanogo-za-prodazh-kliyentskoyi-bazy-poshtovogo-pereviznyka-6604/  Speculative Execution Bounty Launch https://blogs.technet.microsoft.com/msrc/2018/03/14/speculative-execution-bounty-launch/  Frequently Asked Questions about Microsoft Bug Bounty Programs https://technet.microsoft.com/en-us/security/dn425055.aspx  AMD allegedly has its own Spectre-like security flaws https://www.cnet.com/google-amp/news/amd-has-a-spectre-meltdown-like-security-flaw-of-its-own/  ​Linus Torvalds slams CTS Labs over AMD vulnerability report http://www.zdnet.com/article/linus-torvalds-slams-cts-labs-over-amd-vulnerability-report/  Intel: Our next chips won't have data leak flaws we told you totally not to worry about https://www.theregister.co.uk/2018/03/15/intel_spectre_mitigation/  Intel ships (hopefully stable) microcode for Skylake, Kaby Lake, Coffee Lake https://arstechnica.com/gadgets/2018/02/intel-ships-hopefully-stable-microcode-for-skylake-kaby-lake-coffee-lake/  Samba settings SNAFU lets any user change admin passwords https://www.theregister.co.uk/2018/03/14/samba_password_bug/  Zero-day vulnerability in Telegram https://securelist.com/zero-day-vulnerability-in-telegram/83800/  Plugins for Popular Text Editors Could Help Hackers Gain Elevated Privileges https://thehackernews.com/2018/03/text-editors-extensibility.html  В Исландии похитили 600 серверов для добычи Bitcoin https://www.ixbt.com/news/2018/03/06/v-islandii-pohitili-600-serverov-dlja-dobychi-bitcoin.html  CBM - Car Backdoor Maker https://www.kitploit.com/2018/03/cbm-car-backdoor-maker.html  Let's Encrypt updates certificate automation, adds splats https://www.theregister.co.uk/2018/03/14/lets_encrypt_updates_certificate_automation_adds_splats/  CEO of smartmobe outfit Phantom Secure cuffed after cocaine sting, boast of murder-by-GPS http://www.theregister.co.uk/2018/03/13/phantom_secure_ceo_arrested/  Keygen Music [2+ hour Mix] https://www.youtube.com/watch?v=cYkaG5CT53I 

In the news, Memcrashed Memcached DDoS exploit tool, Flash, Windows Users: It's Time to Patch, VMware releases security updates, what happens when Bitcoin miners take over your town, and more on this episode of Paul's Security Weekly! Full Show Notes: https://wiki.securityweekly.com/Episode551 Subscribe to our YouTube channel: https://www.youtube.com/securityweekly Visit our website: http://securityweekly.com Follow us on Twitter: https://www.twitter.comsecurityweekly

In the news, Memcrashed Memcached DDoS exploit tool, Flash, Windows Users: It's Time to Patch, VMware releases security updates, what happens when Bitcoin miners take over your town, and more on this episode of Paul's Security Weekly! Full Show Notes: https://wiki.securityweekly.com/Episode551 Subscribe to our YouTube channel: https://www.youtube.com/securityweekly Visit our website: http://securityweekly.com Follow us on Twitter: https://www.twitter.comsecurityweekly

Flashpoint Editorial Director Mike Mimoso talks to Chad Seaman and Lisa Beegle of the security intelligence response team at Akamai about the recent and record-setting memcached DDoS attacks. The attacks leveraged memcached servers that were exposed to the internet and topped out at well more than a terabyte of traffic used to take down targets. The volume of traffic used in these attacks are the highest seen in publicly reported attacks and Chad and Lisa were among the first to investigate and report on them.

Spécial intelligence artificielle avec Samuel St-Pierre

Digital Shadows’ Research team discusses record DDoS attacks using Memcached servers, disinformation campaigns, a proof of concept exploit for the Spectre vulnerability, and new details of a historical network intrusion affecting the German government.

Midnight Disease‘s Brenden K Jones on creepy random Alexa laughter. People attack driverless cars with their bodies. Quantum computers might start using giant atoms. Jon Favreau. Evel Knievel’s place has rattlesnake sausage pizza which tastes like the Mojave. Memcached servers exploited for massive ddos cyber attacks with a fury of 51,000x. Rhode Island wants to charge... The post talk show EPISODE 4, season 2 appeared first on Greasy Conversation.

Recorded 3/2/18 - Craig is out this week, but the rest of the crew goes through COINHORDER and Memcached and takes a deeper look at authentication and passwords. We cover an overview of reflection attacks and how some passwords schemes that are meant to protect, actually cause harm. We also bid you farewell, since our next episode is supposed to be live after the crew hosts a meeting that stands a not-insignificant chance of getting us all fired. Wish us luck - and send us questions that can make Craig pose to really important Cisco executives.

This week, we discuss the biggest distributed denial-of-service attacks on record, another 2.4 million people affected by the Equifax data breach, and Alexa's sense of humour.

Topics: Secure Framework documents Modifying chromebooks so you can use Debian/Ubuntu Memcached is the new DDoS hotness Announcement of the next BrakeSec Training Class (see Show Notes below for more info) Link to secure framework document: https://drive.google.com/open?id=1xLfY4uI88K2AiA1mosWJ7jFyP100Jv5d Tickets are already on sale for "Hack in the Box" in Amsterdam from 9-13 April 2018, and using the checkout code 'brakeingsecurity' discount code gets you a 10% discount". Register at https://conference.hitb.org/hitbsecconf2018ams/register/    #Spotify: https://brakesec.com/spotifyBDS #RSS: https://brakesec.com/BrakesecRSS #Youtube Channel:  http://www.youtube.com/c/BDSPodcast #iTunes Store Link: https://brakesec.com/BDSiTunes #Google Play Store: https://brakesec.com/BDS-GooglePlay Our main site:  https://brakesec.com/bdswebsite   Join our #Slack Channel! Email us at bds.podcast@gmail.com or DM us on Twitter @brakesec #iHeartRadio App:  https://brakesec.com/iHeartBrakesec #SoundCloud: https://brakesec.com/SoundcloudBrakesec Comments, Questions, Feedback: bds.podcast@gmail.com Support Brakeing Down Security Podcast by using our #Paypal: https://brakesec.com/PaypalBDS OR our #Patreon https://brakesec.com/BDSPatreon #Twitter: @brakesec @boettcherpwned @bryanbrake @infosystir #Player.FM : https://brakesec.com/BDS-PlayerFM #Stitcher Network: https://brakesec.com/BrakeSecStitcher #TuneIn Radio App: https://brakesec.com/TuneInBrakesec   --Show Notes-- Announcements: Matt Miller’s class on Assembly and Reverse engineering Starts 2 April - 6 sessions 2nd Class - 6 sessions, beginning 21 May Beginner course on Assembly Advanced course, dealing with more advanced topics $150 for each class, or a $250 deal if you sign up for both classes paypal.me/BDSPodcast/150USD - Specify in the NOTES if you want the “Beginner” or “Advanced” course paypal.me/BDSPodcast/250USD - If you want both courses We need a minimum of 10 students per class   Projects: Chromebook with Debian Bit of a pain, if I could be honest.. Needed USB hub with eth0, and a USB soundcard USB3 low profile thumbdrives would be better https://www.amazon.com/gp/product/B01K5EBCES/ref=oh_aui_detailpage_o01_s00?ie=UTF8&psc=1 https://www.securecontrolsframework.com/ ←--well well worth the signup https://drive.google.com/open?id=1xLfY4uI88K2AiA1mosWJ7jFyP100Jv5d - ‘secure.xlsx’ http://www.dummies.com/programming/certification/security-control-frameworks/ Numerous security frameworks already exist: Cisco NiST CoBIT ITIL (can be utilized) SWIFT  https://www.accesspay.com/wp-content/uploads/2017/09/SWIFT_Customer_Security_Controls_Framework.pdf “My weird path to #infosec” on twitter https://en.wikipedia.org/wiki/Hydrocolloid_dressing

Cybersécurité et budget fédéral, les Russes (encore) dans les élections et memcached

In this session, we provide a peek behind the scenes to learn about Amazon ElastiCache's design and architecture. See common design patterns with our Redis and Memcached offerings and how customers have used them for in-memory operations to reduce latency and improve application throughput. During this session, we review ElastiCache best practices, design patterns, and anti-patterns.

Tung Nguyen (Twitter tongueroo | Github tongueroo ) is an elite software engineer who as first technical hire at Bleacher Report, led and executed a technology roadmap supporting the company’s growth from an obscure sports blog to one of the world’s most popular sport news sites today. He has a tremendous amount of experience using cloud infrastructure like Amazon Web Services (AWS) and a wide skill set in using open source tools to quickly scale. We covered everything from how “Lebron Decision” presented a major technical challenge, how bad 90s electronic music set a precedent for focusing in a noisy environment, how to get things done working from home with four daughters, how important mentors are in shaping your career, morning habits that power you to get things done, why Richard Branson wasted his time in a hot air balloon instead of witnessing the birth of his child, and so much more. Enjoy the full podcast to immerse yourself into a well rounded discussion on how to architect your stack for massive scalability, home balance, nutrition, and the future of software engineering. If you’re looking for a quick hit, skip to this highlights: How humble beginnings helped Tung to achieve so much more 13:00 How a 9 month hard knock job got his foot into the big league 14:00 First experience coding 16:00 First day and first hire at Bleacher Report 20:00 How to make the a career decision and transition 24:00 Expanding B/R at scale 25:00 Solving engineering challenges caused by the “Janet Jackson Halftime,” Super Bowl power out,” and “Lebron decision” 26:00 Mitigating risks by benchmarking traffic 30:00 Testing on production on a static S3 site vs staging 31:00 Finding and addresses bottlenecks 35:00 Architecting for 4 billion push notifications 36:00 Tools and technologies including Redis, MemcacheD, Rub, Node.js, Amazon Web Services and much more 37:00 On the challenges for building a cloud agnostic platform 38:00 Must have services on Amazon Web Services 40:00 Naming conventions and policies for building 45:00 Handling failure: learning from success vs failing fast 47:00 Handling the NFL crash with a slow query 48:00 Brutal honesty and doing better 50:00 Team building also involves after hours relationship building 51:00 When engineers speaking to end users make sense 53:00 Goal setting and payscale 55:00 Lessons learned from terminating staff 56:00 Will microservices solve all of your problems? 59:00 Creating an environment to get stuff done 1:04 The dark side of coding: dealing with constant change 1:08 Impact of old age: how to keep up as you get older in your career 1:09 Favorite books 1:11 Eliminating distraction and reigning in the focus 1:13 How a house of four daughters influences music choices 1:14 Office vs home environment to get things done (home with 4 daughters!) 1:17 Importance of mentors and the how to build a skill set with them 1:20 Is waking up at 5:00 a game changer in getting things done? 1:23 Balancing nutrition and sleep to perform better in software engineering 1:27 Making time time for kids 1:34 Effect of hacking elementary English by learning characters by site vs phonetics 1:35 Challenge of prioritizing family, kids, and work 1:38 Separating work from home vs. going all in 1:41 What was Richard Branson was doing what while missing the birth of his child? 1:42 Who wears the pants in a house where Tung is the only male? 1:45 Severless architecture and containers 1:52 Devops engineering success in the enterprise is a cultural or engineering challenge? 2:04 Advice for beginners: you already know more than you think you know 2:06 The benevolent dictator and the commitment for life 2:09 A books mentioned in the show Growth Mindset Never Eat Alone How to Win Friends and Influence People The 7 Habits of Highly Effective People Losing My Virginity: How I Survived, Had Fun, and Made a Fortune Doing Business My Way

In this session, we provide a peek behind the scenes to learn about Amazon ElastiCache's design and architecture. See common design patterns with our Redis and Memcached offerings and how customers have used them for in-memory operations to reduce latency and improve application throughput. During this session, we review ElastiCache best practices, design patterns, and anti-patterns.

We’ll share an overview of leveraging serverless architectures to support high performance data intensive applications. Fulfillment by Amazon (FBA) built the Seller Inventory Authority Platform (IAP) using Amazon DynamoDB Streams, AWS Lambda functions, Amazon Elasticsearch Service, and Amazon Redshift to improve results and reduce costs. Scopely will share how they used a flexible logging system built on Kinesis, Lambda, and Amazon Elasticsearch to provide high-fidelity reporting on hotkeys in Memcached and DynamoDB, and drastically reduce the incidence of hotkeys. Both of these customers are using managed services and serverless architecture to build scalable systems that can meet the projected business growth without a corresponding increase in operational costs.

O que o Oracle, o MS SQL Server, o MySQL, o Postgre e tantos outros não fazem que popularizou o uso de outras soluções, conhecidas como NoSQL? Como é utilizar uma dessas soluções, como o MongoDB? Minha base de dados não vai ficar um grande bagunça para dar manutenção? Bem, ela já é uma bagunça sem isso :). Participantes: Paulo Silveira, host do hipsters Mauricio Linhares, o cohost que realmente conhece uma penca de assuntos David Paniz, trabalhou no NuBank onde atuou bastante com NoSQL e é autor do livro de NoSQL da Casa do Código Gleicon Moraes, engenheiro de software da LuizaLabs, autor do livro Ferramentas DevOps Links citados no episódio e extras: MongoDB :P NoSQL Summer: uma sugestão de papers importantes Paper do Teorema CAP Palestra sobre NoSQL do David Paniz NoSQL Tapes: estudos de caso e entrevistas sobre NoSQL, assim como esse tape do Startup Tapes.  Apresentação do Brad Fitzpatrick sobre Memcached no Livejournal. Aurora Deep Dive do AWS Reinvent 2015, muito bom para entender os componentes de um DB e como chegaram a solução de um MySQL escalável Deep Dive do DynamoDB - para entender quais parametros o usuário tem que se preocupar e como um desenho de tabela incompatí vel com as queries pode custar caro Cursos bacanas relacionados: Curso de Redis Curso de MongoDB Produção e conteúdo: Alura Cursos online de Tecnologia - https://www.alura.com.br === Caelum Ensino e Inovação Edição e sonorização: Radiofobia Podcast e Multimídia

O que o Oracle, o MS SQL Server, o MySQL, o Postgre e tantos outros não fazem que popularizou o uso de outras soluções, conhecidas como NoSQL? Como é utilizar uma dessas soluções, como o MongoDB? Minha base de dados não vai ficar um grande bagunça para dar manutenção? Bem, ela já é uma bagunça sem isso :). Participantes: Paulo Silveira, host do hipsters Mauricio Linhares, o cohost que realmente conhece uma penca de assuntos David Paniz, trabalhou no NuBank onde atuou bastante com NoSQL e é autor do livro de NoSQL da Casa do Código Gleicon Moraes, engenheiro de software da LuizaLabs, autor do livro Ferramentas DevOps Links citados no episódio e extras: MongoDB :P NoSQL Summer: uma sugestão de papers importantes Paper do Teorema CAP Palestra sobre NoSQL do David Paniz NoSQL Tapes: estudos de caso e entrevistas sobre NoSQL, assim como esse tape do Startup Tapes.  Apresentação do Brad Fitzpatrick sobre Memcached no Livejournal. Aurora Deep Dive do AWS Reinvent 2015, muito bom para entender os componentes de um DB e como chegaram a solução de um MySQL escalável Deep Dive do DynamoDB - para entender quais parametros o usuário tem que se preocupar e como um desenho de tabela incompatí vel com as queries pode custar caro Cursos bacanas relacionados: Curso de Redis Curso de MongoDB Produção e conteúdo: Alura Cursos online de Tecnologia - https://www.alura.com.br === Caelum Ensino e Inovação Edição e sonorização: Radiofobia Podcast e Multimídia

O que o Oracle, o MS SQL Server, o MySQL, o Postgre e tantos outros não fazem que popularizou o uso de outras soluções, conhecidas como NoSQL? Como é utilizar uma dessas soluções, como o MongoDB? Minha base de dados não vai ficar um grande bagunça para dar manutenção? Bem, ela já é uma bagunça sem isso :). Participantes: Paulo Silveira, host do hipsters Mauricio Linhares, o cohost que realmente conhece uma penca de assuntos David Paniz, trabalhou no NuBank onde atuou bastante com NoSQL e é autor do livro de NoSQL da Casa do Código Gleicon Moraes, engenheiro de software da LuizaLabs, autor do livro Ferramentas DevOps Links citados no episódio e extras: MongoDB :P NoSQL Summer: uma sugestão de papers importantes Paper do Teorema CAP Palestra sobre NoSQL do David Paniz NoSQL Tapes: estudos de caso e entrevistas sobre NoSQL, assim como esse tape do Startup Tapes.  Apresentação do Brad Fitzpatrick sobre Memcached no Livejournal. Aurora Deep Dive do AWS Reinvent 2015, muito bom para entender os componentes de um DB e como chegaram a solução de um MySQL escalável Deep Dive do DynamoDB - para entender quais parametros o usuário tem que se preocupar e como um desenho de tabela incompatí vel com as queries pode custar caro Cursos bacanas relacionados: Curso de Redis Curso de MongoDB Produção e conteúdo: Alura Cursos online de Tecnologia - https://www.alura.com.br === Caelum Ensino e Inovação Edição e sonorização: Radiofobia Podcast e Multimídia

Fredrik talks to James Turnbull of Kickstarter, Docker and several other companies. Topics range from switching between types of companies and solutions to writing books, documentation and contributing to software in ways other than code. Of course, we also discuss Docker, whether it’s succeeded in various ways and where it might be going. Who should be thinking about Docker? How to start thinking about it? Where do you start picking on your monolith to start bringing it into the container future? This episode was recorded during the developer conference Øredev 2015, where James gave a presentation on Orchestrating Docker. Thank you Cloudnet for sponsoring our VPS! Comments, questions or tips? We are @kodsnack, @tobiashieta, @oferlund och @bjoreman on Twitter, have a page on Facebook and can be emailed on info@kodsnack.se if you want to write something longer. We read everything you send. If you like Kodsnack we would love a review in iTunes! Links James Turnbull Docker - the company and the software solution Puppet labs Public-benefit corporation Immutable infrastructure Docker swarm Docker compose Kubernetes Mesos Mesosphere Elasticsearch Memcached Redis Amazon cloudformation James' books William Gibson Dennis Ritchie Daniel Friedman Vagrant Jekyll Titles A lot of similar paralells The unit of the container A unit of compute I want my code to run somewhere where it makes me money A new way of thinking about architecture On the periphery of the monolith Useful information trapped in the heads of smart people My commits tend to be more documentation than code Aspects of being an engineer A higher level of tolerance and precision

Visit This Week on developerWorks at: http://ibm.com/developerworks/thisweek Links to articles mentioned on this episode are at: https://ibm.biz/twodw20140127

Brian talks with Ofer Bengal (Co-Founder/CEO @Garantiadata) about the challenges and realities of NoSQL databases in the cloud and the value of cloud-enabling services such as Memcached and Redis. We also talked about backing up NoSQL databases and the challenges of transitioning to DevOps models.

Continuando con la temática web del episodio 007, en este programa hacemos con Ramón (@m3drano) un completo análisis de todas las técnicas de optimización que podemos aplicar a una aplicación web, técnicas que podremos aplicar sea cual sea el framework o la plataforma que usemos en servidor, ya sea PHP, J2EE, .NET, Python o cualquier [...]

NoSQL is a terrible term for a collection of widely varied databases. You have key-value stores like Redis, Tokyo Cabinet, Memcached, etc. You also have document databases like couchDB and mongoDB. Finally you have column based systems like Cassandra. And still others like HBase. In a lot of cases, there are gems for these. I've used several of them such as the gems for managing Cassandra, couchDB, and mongoDB.

In this episode we discussed: MongoDB Standardization of NoSQL databases Portability between non-relational databases CouchDB PostgreSQL AGPL license PostgreSQL license (like the BSD license) MySQL is GPLv2 Drizzle has rewritten their MySQL driver so it’s not GPL Oracle’s behavior toward products they own that compete InnoDB MySQL engine Microsoft SQL – The price hike and bug report that drove Josh to PostgreSQL Customer expectations vs Intended functionality GreenPlum Alexa Implementing the minimum feature set and getting feedback. Transactional DDL – All operations are transactional except create database. Database Migrations – PostgreSQL can do migrations with no downtime. Memcached Redis Solr ElasticSearch Foreign Data Wrappers – a driver for external data sources that can then be managed through PostgreSQL Lucene Hadoop HBase Cassandra Project Voldemort HyperTable Riak Amazon Cap Theorem Papers VoltDB

Find out what NoSQL is and isnt. News and Follow/Ups – 02:24 Google Sunsets Translate API “Why would anyone ever use your api’s again?” and Google is not your daddy Could they have made it profitable though? Google Wallet Geek Tools – 13:15 Supergoop! SPF 30 Sunscreen Swipes with Zinc for Sensitive Skin Supergoop! SPF 30 […]

Heute reden wir über die Cloud im täglichen Leben sowie einige Neuerungen aus der MySQL-Ecke. Im Studio: Johannes Schlüter, Andreas Huber, Marc Baumann und Moderator Constantin Gonzalez.

Enregitree le 9 Fevrier 2010, reenregistree le 17 fevrier 2010 mieux vaut deux fois qu'une   Guillaume Bort http://twitter.com/PlayFramework http://www.zenexity.fr/ Nicolas Leroux http://twitter.com/nicolasleroux http://www.lunatech-research.com/   Note: Nicolas était dans l'interview originale que j'ai malheureusement perdu. Guillaume et moi avons essayé de reproduire autant que possible l'atmosphère de l'interview originale. Donc Nicolas était là sans être là dans la seconde :)   Play! Framework http://www.playframework.org/ Hibernate https://www.hibernate.org/ Ruby on Rails http://rubyonrails.org/ django http://www.djangoproject.com/ Grails http://www.grails.org/ JSF 2 https://javaserverfaces.dev.java.net/ Seam http://seamframework.org/ Java EE http://java.sun.com/javaee/ Wicket http://wicket.apache.org/ REST http://en.wikipedia.org/wiki/Representational_State_Transfer Groovy http://groovy.codehaus.org/ Quartz http://www.quartz-scheduler.org/ Amazon Simple Queue Service http://aws.amazon.com/sqs/  ZooKeeper http://hadoop.apache.org/zookeeper/      Hadoop http://hadoop.apache.org/ Memcached http://memcached.org/ GORM http://www.grails.org/GORM Mina http://mina.apache.org/ Grizzly https://grizzly.dev.java.net/ Netty http://www.jboss.org/netty Servlet 3 http://java.sun.com/products/servlet/  JBoss AS http://www.jboss.org/jbossas/ Active MQ http://activemq.apache.org/  HornetQ http://www.jboss.org/hornetq ETag http://en.wikipedia.org/wiki/HTTP_ETag jQuery http://jquery.com/ prototype http://www.prototypejs.org/  jQuery in Action http://www.manning.com/bibeault/ Sass http://sass-lang.com/