POPULARITY
12 episodes with Sandboxing
3 episodes with Sandboxing
4 episodes with Sandboxing
2 episodes with Sandboxing
7 episodes with Sandboxing
2 episodes with Sandboxing
2 episodes with Sandboxing
2 episodes with Sandboxing
2 episodes with Sandboxing
The project, spearheaded by Greek non-profit child welfare organisation The Smile of the Child, will be co-created by children and young people to ensure their voices are heard. The ISPCC is honoured to announce its participation in a worldwide project designed to transform how we prevent and respond to online child sexual exploitation and abuse. Safe Online, a global fund dedicated to eradicating online child sexual exploitation and abuse, is funding the project called "Sandboxing and Standardising Child Online Redress". The COR Sandbox project will establish a first-of-its-kind mechanism to advance child online safety through collaboration across sectors, borders and generations. The project is led by The Smile of the Child, Greece's premier child welfare organisation and ISPCC is a partner alongside The Young and Resilient Research Centre at Western Sydney University, Child Helpline International and the Centre for Digital Policy at University College Dublin. Sandboxes bring together industry, regulators and customers in a safe space to test innovative products and services without incurring regulatory sanctions and they are mainly used in the finance sector to test new services. The EU is increasingly encouraging the use of sandboxes in the field of high technology and artificial intelligence. Through the participation of youth, platforms, regulators and online safety experts, this first regulatory sandbox for child digital wellbeing will provide for consistent, systemic care and redress for children from online harm, based on their rights under the United Nations Convention on the Rights of the Child (UNCRC). Getting reporting and redress right means that we can keep track of harms and be able to identify systemic risk. Co-designing the reporting and redress process with young people as equitable participants can help us understand what they expect from the reporting process and what remedies are fair for them putting Article 12 of the UNCRC into action. The project also benefits from the guidance of renowned digital safety experts, including Project Lead and Scientific Coordinator Ioanna Noula, PhD, an international expert on tech policy and children's rights; pioneering online safety and youth rights advocate Anne Collier; youth rights and participation expert Amanda Third, PhD, of the Young and Resilient Research Centre; international innovation management consultant Nicky Hickman; IT innovation and startup founder Jez Goldstone; and leading child online wellbeing scholar Tijana Milosevic, PhD. ISPCC Head of Policy and Public Affairs Fiona Jennings said: "This project is a wonderful example of what we can achieve when we collaborate and listen to children and young people. Having robust online reporting mechanisms in place is a key policy objective for ISPCC and this project will go a long way towards making the online world safer for children and young people to participate in." Project lead Ioanna Noula said: "ISPCC's contribution to a project, which seeks to build coherence around the issue of online redress, will be a catalyst for real and substantial change in the area of online reporting. Helplines play a key role in flagging illegal and/or harmful content. As the experts in listening and responding to children, ISPCC can provide insight from an Irish context to help spearheading the implementation of the Digital Services Act and the wellbeing of children online." See more stories here. More about Irish Tech News Irish Tech News are Ireland's No. 1 Online Tech Publication and often Ireland's No.1 Tech Podcast too. You can find hundreds of fantastic previous episodes and subscribe using whatever platform you like via our Anchor.fm page here: https://anchor.fm/irish-tech-news If you'd like to be featured in an upcoming Podcast email us at Simon@IrishTechNews.ie now to discuss. Irish Tech News have a range of services available to help promote your business. Why not drop us a line at Info@IrishTechNews.ie now to ...
Psychologische Techniken der Manipulation sind allgegenwärtig unter uns. Jeder wendet sie an, auf jeden von uns werden sie angewandt.
More info: https://docs.anthropic.com/en/docs/claude-code/overview The AI coding wars have now split across four battlegrounds: 1. AI IDEs: with two leading startups in Windsurf ($3B acq. by OpenAI) and Cursor ($9B valuation) and a sea of competition behind them (like Cline, Github Copilot, etc). 2. Vibe coding platforms: Bolt.new, Lovable, v0, etc. all experiencing fast growth and getting to the tens of millions of revenue in months. 3. The teammate agents: Devin, Cosine, etc. Simply give them a task, and they will get back to you with a full PR (with mixed results) 4. The cli-based agents: after Aider's initial success, we are now seeing many other alternatives including two from the main labs: OpenAI Codex and Claude Code. The main draw is that 1) they are composable 2) they are pay as you go based on tokens used. Since we covered all three of the first categories, today's guests are Boris and Cat, the lead engineer and PM for Claude Code. If you only take one thing away from this episode, it's this piece from Boris: Claude Code is not a product as much as it's a Unix utility. This fits very well with Anthropic's product principle: “do the simple thing first.” Whether it's the memory implementation (a markdown file that gets auto-loaded) or the approach to prompt summarization (just ask Claude to summarize), they always pick the smallest building blocks that are useful, understandable, and extensible. Even major features like planning (“/think”) and memory (#tags in markdown) fit the same idea of having text I/O as the core interface. This is very similar to the original UNIX design philosophy: Claude Code is also the most direct way to consume Sonnet for coding, rather than going through all the hidden prompting and optimization than the other products do. You will feel that right away, as the average spend per user is $6/day on Claude Code compared to $20/mo for Cursor, for example. Apparently, there are some engineers inside of Anthropic that have spent >$1,000 in one day! If you're building AI developer tools, there's also a lot of alpha on how to design a cli tool, interactive vs non-interactive modes, and how to balance feature creation. Enjoy! Timestamps [00:00:00] Intro [00:01:59] Origins of Claude Code [00:04:32] Anthropic's Product Philosophy [00:07:38] What should go into Claude Code? [00:09:26] Claude.md and Memory Simplification [00:10:07] Claude Code vs Aider [00:11:23] Parallel Workflows and Unix Utility Philosophy [00:12:51] Cost considerations and pricing model [00:14:51] Key Features Shipped Since Launch [00:16:28] Claude Code writes 80% of Claude Code [00:18:01] Custom Slash Commands and MCP Integration [00:21:08] Terminal UX and Technical Stack [00:27:11] Code Review and Semantic Linting [00:28:33] Non-Interactive Mode and Automation [00:36:09] Engineering Productivity Metrics [00:37:47] Balancing Feature Creation and Maintenance [00:41:59] Memory and the Future of Context [00:50:10] Sandboxing, Branching, and Agent Planning [01:01:43] Future roadmap [01:11:00] Why Anthropic Excels at Developer Tools
Vitalik Buterin is the creator of Ethereum, but he's also a true Bitcoin maximalist. In this episode, Vitalik tells his story as a bitcoiner, explains why he built Ethereum, and makes use of his knowledge to predict the future of the two networks. Time stamps: Introducing Vitalik (00:01:00) Vitalik's Early Involvement with Bitcoin (00:02:22) Writing for Bitcoin Weekly (00:03:01) Bitcoin's Early Fees and Transaction Model (00:06:45) Evolving Understanding of Bitcoin (00:09:15) Bitcoin Cash and the Scaling Debate (00:10:25) Dark Wallet Project (00:14:06) Coinjoin and Privacy Innovations (00:16:41) Colored Coins and Bitcoin 2.0 (00:21:05) Transition to Ethereum Development (00:21:58) Current Layer Two Innovations (00:24:11) Scaling and Privacy Innovations (00:25:55) Ethereum's Early Criticism (00:27:05) EVM's Role in Smart Contracts (00:28:11) Challenges of Parallelization (00:29:23) Sandboxing and Security (00:30:24) Future Scaling Ideas (00:34:49) Ethereum vs NXT vs Counterparty vs Omni/Mastercoin (00:35:37) Lessons from Ethereum's Success (00:37:07) The DAO Hack and Community Resilience (00:43:16) Ethereum's Network Effect (00:45:43) Ethereum's Ecosystem Resilience (00:49:35) Decentralization vs. Scalability (00:50:41) Critique of Ethereum Killers (00:51:21) Layer One and Layer Two Dynamics (00:52:53) SideShift (00:53:21) How Vitalik Cancelled Craig Wright (00:54:51) Current Characters in Bitcoin (00:58:03) Daniel Kravisz's Views on Craig Wright (00:59:04) Manipulative Tactics in Dating Advice (01:00:34) NoOnes: Marketplace for Global South (01:01:19) Bitcoin.com News Evolution (01:02:40) Bitcoin Magazine is Now Pro Trump (01:04:37) Libertarian Shifts in Crypto (01:05:03) Ethereum Domain Name Registrations (01:06:09) Layer Two Scaling Decision (01:08:08) Hardware Requirements for Ethereum Node (01:10:45) Philosophical Questions on Scaling (01:12:01) The Dystopia Scenario (01:13:03) Importance of Full Nodes (01:14:24) Technological Innovations (01:15:27) Running Full Nodes in Ethereum (01:16:30) Privacy and RPC Trust (01:17:28) Adapting Ethereum to New Cryptography (01:19:53) Scaling Debate in Ethereum (01:22:04) Respect for Ethereum's Approach (01:23:15) Zcash and Ethereum Collaboration (01:25:00) Challenges for Zcash (01:27:04) Impact of Developer Actions (01:28:01) Scaling Solutions in Bitcoin and Ethereum (01:30:43) Defining Rollups vs. Sidechains (01:31:40) Security Implications of Drivechains (01:34:03) Transition to Proof of Stake (01:36:19) ZK Coins and Shielded Client Side Validation (01:37:53) Thoughts on TheStandard.io (01:40:03) Backing Up Coins and Holding Keys (01:42:11) Evolution of Multi-Sig Technology (01:46:43) Privacy (01:48:14) Concerns About Centralized Data Collection (01:51:10) Impact of Snowden Revelations (01:53:35) Privacy as a Key Aspect of Decentralization (01:55:49) Ethereum's Cypherpunk Roots (01:57:07) Feedback from Cypherpunks on Ethereum (02:00:42) The Inspiration Behind DAOs (02:02:07) AI and DAOs (02:02:40) Vitalik's Public Image and Price Pressure (02:02:55) Media Attention and Its Impact (02:03:43) Decentralization and Attention (02:04:03) Price Influence and Market Dynamics (02:04:59) Focus on Ethereum's Values (02:06:01) Historical Use Cases of Ethereum (02:08:28) Next Bull Market Narrative (02:09:38) DeFi Ecosystem as a Proven Use Case (02:09:45) Political Instability and Financial Security (02:12:05) Polymarket, Prediction Markets and Mainstream Adoption (02:12:20) Zero Knowledge Proofs and Privacy (02:14:20) Roger Ver (02:15:23) Principles of Freedom and Privacy (02:22:57) Critique of Blockstream's Liquid (02:24:00) Bitcoin's Role in Decentralization (02:26:15) Transition to RISC-V (02:27:37) Adoption of RISC-V (02:28:36) Redesigning Ethereum in A Time Travel Scenario (02:31:30) Challenges in Ethereum's Development (02:32:45) Ethereum and Bitcoin Relationship (02:37:02) Complementarity of Bitcoin and Ethereum (02:38:40) Does Vitalik Still Use Bitcoin? (02:41:21) Lightning Network (02:42:06) Standardization of LN Invoies (02:43:20) Privacy Concerns with Bitcoin (02:45:42) Running Lightning Nodes (02:46:52) Home-Based Bitcoin Solutions (02:48:12) Tribalism in Crypto Communities (02:48:53) Ethereum's Evolution and Ideals (02:50:06) Collaboration Between Bitcoin and Ethereum (02:51:10) Diverse Blockchain Future (02:51:45) Is Vitalik a Bitcoin Maximalist? (02:52:59) Community Values and Challenges (02:53:45) Cultural Dynamics in Cryptocurrencies (02:56:05) Layer Two Solutions for Bitcoin (02:59:31) Vitalik's Online Presence (03:00:25) Closing Remarks and Future Guests (03:01:36)
video: https://youtu.be/iZX7YFt66zs This week we are LIVE! Celebrating our 400th episode. We have a lot of content planned for you but first, we just want to thank all of you for supporting us by watching, listening, however you do it. 400 episodes is because of you. The financial support because of our sponsors and Patrons is how we survived. We're so glad you enjoy this show and hope to see you for 400 more. Welcome to Destination Linux, where we discuss the latest news, hot topics, gaming, mobile, and all things Open Source & Linux. Now let's get this show on the road toward Destination Linux! Forum Discussion Thread (https://forum.tuxdigital.com/t/400-celebrating-400-episodes-of-dl-with-a-jills-treasure-hunt-gaming-cybersecurity-and-more/6527) Download as MP3 (https://aphid.fireside.fm/d/1437767933/32f28071-0b08-4ea1-afcc-37af75bd83d6/759da82c-4d2d-4148-bc28-c776bd6aa630.mp3) Support the show by becoming a patron at tuxdigital.com/membership (https://tuxdigital.com/membership) or get some swag at tuxdigital.com/store (https://tuxdigital.com/store) Hosted by: Ryan (DasGeek) = dasgeek.net (https://dasgeek.net) Jill Bryant = jilllinuxgirl.com (https://jilllinuxgirl.com) Michael Tunnell = michaeltunnell.com (https://michaeltunnell.com) Chapters: 00:00:00 Intro 00:04:35 Community Feedback 00:08:48 Pizza Debates: Water, Pineapple, and Everything In-Between 00:15:09 Privacy-Friendly Streaming Options 00:18:22 Memes Born from DL 00:23:50 Sandfly Security 00:28:02 Jill's Treasure Hunt Begins 00:29:18 E.T.'s Backstory and Jill's Nostalgia 00:30:59 Unboxing Jill's Treasure: Atari E.T. Cartridge 00:32:04 The Video Game Crash: Blame E.T.? 00:33:54 E.T.: Not the Worst Game Ever? 00:37:31 First Playthrough: E.T. Frustrations 00:40:34 Fatal Flaw: The Game's Big Bug 00:44:23 To the Landfill: E.T.'s Infamous Fate 00:45:52 In the History Books: E.T.'s Legacy 00:48:14 Wrapping Up Jill's Treasure Hunt 00:49:37 The Big Hack - Salt Typhoon 00:50:55 Off-Topic Fun in the Rabbit Hole 00:53:07 The Targets of Salt Typhoon 00:55:09 How the Hack Worked 00:56:46 What the Hackers Took 00:57:57 How Salt Typhoon Was Discovered 00:59:33 Lessons Learned from the Hack 01:01:43 The FBI's Encryption ‘Solution' 01:05:59 Mozilla's Rebrand: Michael's Verdict 01:25:24 Can't Stop Talking 01:25:56 Gaming: Vessel of Hatred 01:31:45 Showcasing Geek Cred: Community Picks 01:33:48 Software Spotlight: Gear Lever 01:36:49 App Packages: Order of Priority 01:38:08 AppImage Sandbox 01:38:57 What is Sandboxing? 01:41:26 Scam Education: Holiday Safety Tips 01:47:44 Support the Show 01:49:13 Outro Links: Community Feedback https://destinationlinux.net/comments (https://destinationlinux.net/comments) https://destinationlinux.net/forum (https://destinationlinux.net/forum) Privacy-Friendly Streaming Options Apple TV = https://www.apple.com/apple-tv-4k/ (https://www.apple.com/apple-tv-4k/) NVidia Shield = https://www.nvidia.com/en-us/shield/ (https://www.nvidia.com/en-us/shield/) Amazon Fire Stick = https://amzn.to/3DlBUfC (https://amzn.to/3DlBUfC) Sandfly Security [ad] https://destinationlinux.net/sandfly (https://destinationlinux.net/sandfly) Jill's Treasure Hunt https://destinationlinux.net/jillstreasurehunt (https://destinationlinux.net/jillstreasurehunt) https://en.wikipedia.org/wiki/E.T.theExtra-Terrestrial(videogame) (https://en.wikipedia.org/wiki/E.T._the_Extra-Terrestrial_(video_game)) https://www.youtube.com/watch?v=QmrQkQsM9FU (https://www.youtube.com/watch?v=QmrQkQsM9FU) The Big Hack - Salt Typhoon https://hacked.com/chinese-salt-typhoon-hack-reveals-flaws-in-our-data-privacy/ (https://hacked.com/chinese-salt-typhoon-hack-reveals-flaws-in-our-data-privacy/) https://www.msn.com/en-us/money/other/fbi-says-everyone-should-use-encryption-apps-for-their-calls-and-texts-after-china-hack-encryption-is-your-friend/ar-AA1vshO0?ocid=BingNewsVerp (https://www.msn.com/en-us/money/other/fbi-says-everyone-should-use-encryption-apps-for-their-calls-and-texts-after-china-hack-encryption-is-your-friend/ar-AA1vshO0?ocid=BingNewsVerp) Mozilla's Rebrand: Michael's Verdict https://blog.mozilla.org/en/mozilla/mozilla-brand-next-era-of-tech/ (https://blog.mozilla.org/en/mozilla/mozilla-brand-next-era-of-tech/) Gaming: Diablo IV Vessel of Hatred https://store.steampowered.com/app/3043530/DiabloIVVesselofHatred/ (https://store.steampowered.com/app/3043530/Diablo_IV_Vessel_of_Hatred/) https://www.protondb.com/app/2344520 (https://www.protondb.com/app/2344520) Software Spotlight: Gear Lever https://flathub.org/apps/it.mijorus.gearlever (https://flathub.org/apps/it.mijorus.gearlever) https://mijorus.it/projects/gearlever/ (https://mijorus.it/projects/gearlever/) https://www.omgubuntu.co.uk/2024/07/gear-lever-appimage-app-for-gnome (https://www.omgubuntu.co.uk/2024/07/gear-lever-appimage-app-for-gnome) Support the Show https://tuxdigital.com/membership (https://tuxdigital.com/membership) https://store.tuxdigital.com (https://store.tuxdigital.com) https://tuxdigital.com/discord (https://tuxdigital.com/discord)
We have announced our first speaker, friend of the show Dylan Patel, and topic slates for Latent Space LIVE! at NeurIPS. Sign up for IRL/Livestream and to debate!We are still taking questions for our next big recap episode! Submit questions and messages on Speakpipe here for a chance to appear on the show!The vibe shift we observed in July - in favor of Claude 3.5 Sonnet, first introduced in June — has been remarkably long lived and persistent, surviving multiple subsequent updates of 4o, o1 and Gemini versions, for Anthropic's Claude to end 2024 as the preferred model for AI Engineers and even being the exclusive choice for new code agents like bolt.new (our next guest on the pod!), which unlocked so much performance from Claude Sonnet that it went from $0 to $4m ARR in 4 weeks when it launched last month.Anthropic has now raised an additional $4b from Amazon and made an incredibly well received update of Claude 3.5 Sonnet (and Haiku), making significant improvements in performance over its predecessors:Solving SWE-BenchAs part of the October Sonnet release, Anthropic teased a blink-and-you'll miss it result:The updated Claude 3.5 Sonnet shows wide-ranging improvements on industry benchmarks, with particularly strong gains in agentic coding and tool use tasks. On coding, it improves performance on SWE-bench Verified from 33.4% to 49.0%, scoring higher than all publicly available models—including reasoning models like OpenAI o1-preview and specialized systems designed for agentic coding. It also improves performance on TAU-bench, an agentic tool use task, from 62.6% to 69.2% in the retail domain, and from 36.0% to 46.0% in the more challenging airline domain. The new Claude 3.5 Sonnet offers these advancements at the same price and speed as its predecessor.This was followed up by a blogpost a week later from today's guest, Erik Schluntz, the engineer who implemented and scored this SOTA result using a simple, non-overengineered version of the SWE-Agent framework (you can see the submissions here). We have previously covered the SWE-Bench story extensively:* Speaking with SWEBench/SWEAgent authors at ICLR* Speaking with Cosine Genie, the previous SOTA (43.8%) on SWEBench Verified (with brief update at DevDay 2024)* Speaking with Shunyu Yao on SWEBench and the ReAct paradigm driving SWE-AgentOne of the notable inclusions in this blogpost are the tools that Erik decided to give Claude, e.g. the “Edit Tool”:The tools teased in the SWEBench submission/blogpost were then polished up and released with Computer Use…And you can also see even more computer use tools given in the new Model Context Protocol servers:Claude Computer UseBecause it is one of the best received AI releases of the year, we recommend watching the 2 minute Computer Use intro (and related demos) in its entirety:Eric also worked on Claude's function calling, tool use, and computer use APIs, so we discuss that in the episode.Erik [00:53:39]: With computer use, just give the thing a browser that's logged into what you want to integrate with, and it's going to work immediately. And I see that reduction in friction as being incredibly exciting. Imagine a customer support team where, okay, hey, you got this customer support bot, but you need to go integrate it with all these things. And you don't have any engineers on your customer support team. But if you can just give the thing a browser that's logged into your systems that you need it to have access to, now, suddenly, in one day, you could be up and rolling with a fully integrated customer service bot that could go do all the actions you care about. So I think that's the most exciting thing for me about computer use, is reducing that friction of integrations to almost zero.As you'll see, this is very top of mind for Erik as a former Robotics founder who's company basically used robots to interface with human physical systems like elevators.Full Video episodePlease like and subscribe!Show Notes* Eric Schluntz* “Raising the bar on SWE-Bench Verified”* Cobalt Robotics* SWE-Bench* SWE-Bench Verified* Human Eval & other benchmarks* Anthropic Workbench* Aider* Cursor* Fireworks AI* E2B* Amanda Askell* Toyota Research* Physical Intelligence (Pi)* Chelsea Finn* Josh Albrecht* Eric Jang* 1X* Dust* Cosine Episode* Bolt* Adept Episode* TauBench* LMSys EpisodeTimestamps* [00:00:00] Introductions* [00:03:39] What is SWE-Bench?* [00:12:22] SWE-Bench vs HumanEval vs others* [00:15:21] SWE-Agent architecture and runtime* [00:21:18] Do you need code indexing?* [00:24:50] Giving the agent tools* [00:27:47] Sandboxing for coding agents* [00:29:16] Why not write tests?* [00:30:31] Redesigning engineering tools for LLMs* [00:35:53] Multi-agent systems* [00:37:52] Why XML so good?* [00:42:57] Thoughts on agent frameworks* [00:45:12] How many turns can an agent do?* [00:47:12] Using multiple model types* [00:51:40] Computer use and agent use cases* [00:59:04] State of AI robotics* [01:04:24] Robotics in manufacturing* [01:05:01] Hardware challenges in robotics* [01:09:21] Is self-driving a good business?TranscriptAlessio [00:00:00]: Hey everyone, welcome to the Latent Space Podcast. This is Alessio, partner and CTO at Decibel Partners. And today we're in the new studio with my usual co-host, Shawn from Smol AI.Swyx [00:00:14]: Hey, and today we're very blessed to have Erik Schluntz from Anthropic with us. Welcome.Erik [00:00:19]: Hi, thanks very much. I'm Erik Schluntz. I'm a member of technical staff at Anthropic, working on tool use, computer use, and Swebench.Swyx [00:00:27]: Yeah. Well, how did you get into just the whole AI journey? I think you spent some time at SpaceX as well? Yeah. And robotics. Yeah. There's a lot of overlap between like the robotics people and the AI people, and maybe like there's some interlap or interest between language models for robots right now. Maybe just a little bit of background on how you got to where you are. Yeah, sure.Erik [00:00:50]: I was at SpaceX a long time ago, but before joining Anthropic, I was the CTO and co-founder of Cobalt Robotics. We built security and inspection robots. These are sort of five foot tall robots that would patrol through an office building or a warehouse looking for anything out of the ordinary. Very friendly, no tasers or anything. We would just sort of call a remote operator if we saw anything. We have about 100 of those out in the world, and had a team of about 100. We actually got acquired about six months ago, but I had left Cobalt about a year ago now, because I was starting to get a lot more excited about AI. I had been writing a lot of my code with things like Copilot, and I was like, wow, this is actually really cool. If you had told me 10 years ago that AI would be writing a lot of my code, I would say, hey, I think that's AGI. And so I kind of realized that we had passed this level, like, wow, this is actually really useful for engineering work. That got me a lot more excited about AI and learning about large language models. So I ended up taking a sabbatical and then doing a lot of reading and research myself and decided, hey, I want to go be at the core of this and joined Anthropic.Alessio [00:01:53]: And why Anthropic? Did you consider other labs? Did you consider maybe some of the robotics companies?Erik [00:02:00]: So I think at the time I was a little burnt out of robotics, and so also for the rest of this, any sort of negative things I say about robotics or hardware is coming from a place of burnout, and I reserve my right to change my opinion in a few years. Yeah, I looked around, but ultimately I knew a lot of people that I really trusted and I thought were incredibly smart at Anthropic, and I think that was the big deciding factor to come there. I was like, hey, this team's amazing. They're not just brilliant, but sort of like the most nice and kind people that I know, and so I just felt like I could be a really good culture fit. And ultimately, I do care a lot about AI safety and making sure that I don't want to build something that's used for bad purposes, and I felt like the best chance of that was joining Anthropic.Alessio [00:02:39]: And from the outside, these labs kind of look like huge organizations that have these obscureSwyx [00:02:44]: ways to organize.Alessio [00:02:45]: How did you get, you joined Anthropic, did you already know you were going to work on of the stuff you publish or you kind of join and then you figure out where you land? I think people are always curious to learn more.Erik [00:02:57]: Yeah, I've been very happy that Anthropic is very bottoms up and sort of very sort of receptive to whatever your interests are. And so I joined sort of being very transparent of like, hey, I'm most excited about code generation and AI that can actually go out and sort of touch the world or sort of help people build things. And, you know, those weren't my initial initial projects. I also came in and said, hey, I want to do the most valuable possible thing for this company and help Anthropic succeed. And, you know, like, let me find the balance of those. So I was working on lots of things at the beginning, you know, function calling, tool use. And then sort of as it became more and more relevant, I was like, oh, hey, like, let's it's time to go work on encoding agents and sort of started looking at SWE-Bench as sort of a really good benchmark for that.Swyx [00:03:39]: So let's get right into SWE-Bench. That's one of the many claims to fame. I feel like there's just been a series of releases related with Cloud 3.5 Sonnet around about two or three months ago, 3.5 Sonnet came out and it was it was a step ahead in terms of a lot of people immediately fell in love with it for coding. And then last month you released a new updated version of Cloud Sonnet. We're not going to talk about the training for that because that's still confidential. But I think Anthropic's done a really good job, like applying the model to different things. So you took the lead on SWE-Bench, but then also we're going to talk a little bit about computer use later on. So maybe just give us a context about why you looked at SWE-Bench Verified and you actually came up with a whole system for building agents that would maximally use the model well. Yeah.Erik [00:04:28]: So I'm on a sub team called Product Research. And basically the idea of product research is to really understand what end customers care about and want in the models and then work to try to make that happen. So we're not focused on sort of these more abstract general benchmarks like math problems or MMLU, but we really care about finding the things that are really valuable and making sure the models are great at those. And so because I've been interested in coding agents, I knew that this would be a really valuable thing. And I knew there were a lot of startups and our customers trying to build coding agents with our models. And so I said, hey, this is going to be a really good benchmark to be able to measure that and do well on it. And I wasn't the first person at Anthropic to find SWE-Bench, and there are lots of people that already knew about it and had done some internal efforts on it. It fell to me to sort of both implement the benchmark, which is very tricky, and then also to sort of make sure we had an agent and basically like a reference agent, maybe I'd call it, that could do very well on it. Ultimately, we want to provide how we implemented that reference agent so that people can build their own agents on top of our system and get sort of the most out of it as possible. So with this blog post we released on SWE-Bench, we released the exact tools and the prompt that we gave the model to be able to do well.Swyx [00:05:46]: For people who don't know, who maybe haven't dived into SWE-Bench, I think the general perception is they're like tasks that a software engineer could do. I feel like that's an inaccurate description because it is basically, one, it's a subset of like 12 repos. It's everything they could find that every issue with like a matching commit that could be tested. So that's not every commit. And then SWE-Bench verified is further manually filtered by OpenAI. Is that an accurate description and anything you'd change about that? Yes.Erik [00:06:14]: SWE-Bench is, it certainly is a subset of all tasks. It's first of all, it's only Python repos, so already fairly limited there. And it's just 12 of these popular open source repos. And yes, it's only ones where there were tests that passed at the beginning and also new tests that were introduced that test the new feature that's added. So it is, I think, a very limited subset of real engineering tasks. But I think it's also very valuable because even though it's a subset, it is true engineering tasks. And I think a lot of other benchmarks are really kind of these much more artificial setups of even if they're related to coding, they're more like coding interview style questions or puzzles that I think are very different from day-to-day what you end up doing. I don't know how frequently you all get to use recursion in your day-to-day job, but whenever I do, it's like a treat. And I think it's almost comical, and a lot of people joke about this in the industry, is how different interview questions are.Swyx [00:07:13]: Dynamic programming. Yeah, exactly.Erik [00:07:15]: Like, you code. From the day-to-day job. But I think one of the most interesting things about SWE-Bench is that all these other benchmarks are usually just isolated puzzles, and you're starting from scratch. Whereas SWE-Bench, you're starting in the context of an entire repository. And so it adds this entirely new dimension to the problem of finding the relevant files. And this is a huge part of real engineering, is it's actually pretty rare that you're starting something totally greenfield. You need to go and figure out where in a codebase you're going to make a change and understand how your work is going to interact with the rest of the systems. And I think SWE-Bench does a really good job of presenting that problem.Alessio [00:07:51]: Why do we still use human eval? It's like 92%, I think. I don't even know if you can actually get to 100% because some of the data is not actuallySwyx [00:07:59]: solvable.Alessio [00:08:00]: Do you see benchmarks like that, they should just get sunsetted? Because when you look at the model releases, it's like, oh, it's like 92% instead of like 89%, 90% on human eval versus, you know, SWE-Bench verified is you have 49%, right? Which is like, before 45% was state of the art, but maybe like six months ago it was like 30%, something like that. So is that a benchmark that you think is going to replace human eval, or do you think they're just going to run in parallel?Erik [00:08:27]: I think there's still need for sort of many different varied evals. Like sometimes you do really care about just sort of greenfield code generation. And so I don't think that everything needs to go to sort of an agentic setup.Swyx [00:08:39]: It would be very expensive to implement.Erik [00:08:41]: The other thing I was going to say is that SWE-Bench is certainly hard to implement and expensive to run because each task, you have to parse, you know, a lot of the repo to understand where to put your code. And a lot of times you take many tries of writing code, running it, editing it. It can use a lot of tokens compared to something like human eval. So I think there's definitely a space for these more traditional coding evals that are sort of easy to implement, quick to run, and do get you some signal. Maybe hopefully there's just sort of harder versions of human eval that get created.Alessio [00:09:14]: How do we get SWE-Bench verified to 92%? Do you think that's something where it's like line of sight to it, or it's like, you know, we need a whole lot of things to go right? Yeah, yeah.Erik [00:09:23]: And actually, maybe I'll start with SWE-Bench versus SWE-Bench verified, which is I think something I missed earlier. So SWE-Bench is, as we described, this big set of tasks that were scraped.Swyx [00:09:33]: Like 12,000 or something?Erik [00:09:34]: Yeah, I think it's 2,000 in the final set. But a lot of those, even though a human did them, they're actually impossible given the information that comes with the task. The most classic example of this is the test looks for a very specific error string. You know, like assert message equals error, something, something, something. And unless you know that's exactly what you're looking for, there's no way the model is going to write that exact same error message, and so the tests are going to fail. So SWE-Bench verified was actually made in partnership with OpenAI, and they hired humans to go review all these tasks and pick out a subset to try to remove any obstacle like this that would make the tasks impossible. So in theory, all of these tasks should be fully doable by the model. And they also had humans grade how difficult they thought the problems would be. Between less than 15 minutes, I think 15 minutes to an hour, an hour to four hours, and greater than four hours. So that's kind of this interesting sort of how big the problem is as well. To get to SWE-Bench verified to 90%, actually, maybe I'll also start off with some of the remaining failures that I see when running our model on SWE-Bench. I'd say the biggest cases are the model sort of operates at the wrong level of abstraction. And what I mean by that is the model puts in maybe a smaller band-aid when really the task is asking for a bigger refactor. And some of those, you know, is the model's fault, but a lot of times if you're just sort of seeing the GitHub issue, it's not exactly clear which way you should do. So even though these tasks are possible, there's still some ambiguity in how the tasks are described. That being said, I think in general, language models frequently will produce a smaller diff when possible, rather than trying to do a big refactor. I think another area, at least the agent we created, didn't have any multimodal abilities, even though our models are very good at vision. So I think that's just a missed opportunity. And if I read through some of the traces, there's some funny things where, especially the tasks on matplotlib, which is a graphing library, the test script will save an image and the model will just say, okay, it looks great, you know, without looking at it. So there's certainly extra juice to squeeze there of just making sure the model really understands all the sides of the input that it's given, including multimodal. But yeah, I think like getting to 92%. So this is something that I have not looked at, but I'm very curious about. I want someone to look at, like, what is the union of all of the different tasks that have been solved by at least one attempt at SWE-Bench Verified. There's a ton of submissions to the benchmark, and so I'd be really curious to see how many of those 500 tasks at least someone has solved. And I think, you know, there's probably a bunch that none of the attempts have ever solved. And I think it'd be interesting to look at those and say, hey, is there some problem with these? Like, are these impossible? Or are they just really hard and only a human could do them?Swyx [00:12:22]: Yeah, like specifically, is there a category of problems that are still unreachable by any LLM agent? Yeah, yeah. And I think there definitely are.Erik [00:12:28]: The question is, are those fairly inaccessible or are they just impossible because of the descriptions? But I think certainly some of the tasks, especially the ones that the human graders reviewed as like taking longer than four hours are extremely difficult. I think we got a few of them right, but not very many at all in the benchmark.Swyx [00:12:49]: And did those take less than four hours?Erik [00:12:51]: They certainly did less than, yeah, than four hours.Swyx [00:12:54]: Is there a correlation of length of time with like human estimated time? You know what I mean? Or do we have sort of more of X paradox type situations where it's something super easy for a model, but hard for a human?Erik [00:13:06]: I actually haven't done the stats on that, but I think that'd be really interesting to see of like how many tokens does it take and how is that correlated with difficulty? What is the likelihood of success with difficulty? I think actually a really interesting thing that I saw, one of my coworkers who was also working on this named Simon, he was focusing just specifically on the very hard problems, the ones that are said to take longer than four hours. And he ended up sort of creating a much more detailed prompt than I used. And he got a higher score on the most difficult subset of problems, but a lower score overall on the whole benchmark. And the prompt that I made, which is sort of much more simple and bare bones, got a higher score on the overall benchmark, but lower score on the really hard problems. And I think some of that is the really detailed prompt made the model sort of overcomplicate a lot of the easy problems, because honestly, a lot of the suite bench problems, they really do just ask for a bandaid where it's like, hey, this crashes if this is none, and really all you need to do is put a check if none. And so sometimes trying to make the model think really deeply, it'll think in circles and overcomplicate something, which certainly human engineers are capable of as well. But I think there's some interesting thing of the best prompt for hard problems might not be the best prompt for easy problems.Alessio [00:14:19]: How do we fix that? Are you supposed to fix it at the model level? How do I know what prompt I'm supposed to use?Swyx [00:14:25]: Yeah.Erik [00:14:26]: And I'll say this was a very small effect size, and so I think this isn't worth obsessing over. I would say that as people are building systems around agents, I think the more you can separate out the different kinds of work the agent needs to do, the better you can tailor a prompt for that task. And I think that also creates a lot of like, for instance, if you were trying to make an agent that could both solve hard programming tasks, and it could just write quick test files for something that someone else had already made, the best way to do those two tasks might be very different prompts. I see a lot of people build systems where they first sort of have a classification, and then route the problem to two different prompts. And that's sort of a very effective thing, because one, it makes the two different prompts much simpler and smaller, and it means you can have someone work on one of the prompts without any risk of affecting the other tasks. So it creates like a nice separation of concerns. Yeah.Alessio [00:15:21]: And the other model behavior thing you mentioned, they prefer to generate like shorter diffs. Why is that? Like, is there a way? I think that's maybe like the lazy model question that people have is like, why are you not just generating the whole code instead of telling me to implement it?Swyx [00:15:36]: Are you saving tokens? Yeah, exactly. It's like conspiracy theory. Yeah. Yeah.Erik [00:15:41]: Yeah. So there's two different things there. One is like the, I'd say maybe like doing the easier solution rather than the hard solution. And I'd say the second one, I think what you're talking about is like the lazy model is like when the model says like dot, dot, dot, code remains the same.Swyx [00:15:52]: Code goes here. Yeah. I'm like, thanks, dude.Erik [00:15:55]: But honestly, like that just comes as like people on the internet will do stuff like that. And like, dude, if you're talking to a friend and you ask them like to give you some example code, they would definitely do that. They're not going to reroll the whole thing. And so I think that's just a matter of like, you know, sometimes you actually do just, just want like the relevant changes. And so I think it's, this is something where a lot of times like, you know, the models aren't good at mind reading of like which one you want. So I think that like the more explicit you can be in prompting to say, Hey, you know, give me the entire thing, no, no elisions versus just give me the relevant changes. And that's something, you know, we want to make the models always better at following those kinds of instructions.Swyx [00:16:32]: I'll drop a couple of references here. We're recording this like a day after Dario, Lex Friedman just dropped his five hour pod with Dario and Amanda and the rest of the crew. And Dario actually made this interesting observation that like, we actually don't want, we complain about models being too chatty in text and then not chatty enough in code. And so like getting that right is kind of a awkward bar because, you know, you, you don't want it to yap in its responses, but then you also want it to be complete in, in code. And then sometimes it's not complete. Sometimes you just want it to diff, which is something that Enthopic has also released with a, you know, like the, the fast edit stuff that you guys did. And then the other thing I wanted to also double back on is the prompting stuff. You said, you said it was a small effect, but it was a noticeable effect in terms of like picking a prompt. I think we'll go into suite agent in a little bit, but I kind of reject the fact that, you know, you need to choose one prompt and like have your whole performance be predicated on that one prompt. I think something that Enthopic has done really well is meta prompting, prompting for a prompt. And so why can't you just develop a meta prompt for, for all the other prompts? And you know, if it's a simple task, make a simple prompt, if it's a hard task, make a hard prompt. Obviously I'm probably hand-waving a little bit, but I will definitely ask people to try the Enthopic Workbench meta prompting system if they haven't tried it yet. I went to the Build Day recently at Enthopic HQ, and it's the closest I've felt to an AGI, like learning how to operate itself that, yeah, it's, it's, it's really magical.Erik [00:17:57]: Yeah, no, Claude is great at writing prompts for Claude.Swyx [00:18:00]: Right, so meta prompting. Yeah, yeah.Erik [00:18:02]: The way I think about this is that humans, even like very smart humans still use sort of checklists and use sort of scaffolding for themselves. Surgeons will still have checklists, even though they're incredible experts. And certainly, you know, a very senior engineer needs less structure than a junior engineer, but there still is some of that structure that you want to keep. And so I always try to anthropomorphize the models and try to think about for a human sort of what is the equivalent. And that's sort of, you know, how I think about these things is how much instruction would you give a human with the same task? And do you, would you need to give them a lot of instruction or a little bit of instruction?Alessio [00:18:36]: Let's talk about the agent architecture maybe. So first, runtime, you let it run until it thinks it's done or it reaches 200k context window.Swyx [00:18:45]: How did you come up? What's up with that?Erik [00:18:47]: Yeah.Swyx [00:18:48]: Yeah.Erik [00:18:49]: I mean, this, so I'd say that a lot of previous agent work built sort of these very hard coded and rigid workflows where the model is sort of pushed through certain flows of steps. And I think to some extent, you know, that's needed with smaller models and models that are less smart. But one of the things that we really wanted to explore was like, let's really give Claude the reins here and not force Claude to do anything, but let Claude decide, you know, how it should approach the problem, what steps it should do. And so really, you know, what we did is like the most extreme version of this is just give it some tools that it can call and it's able to keep calling the tools, keep thinking, and then yeah, keep doing that until it thinks it's done. And that's sort of the most, the most minimal agent framework that we came up with. And I think that works very well. I think especially the new Sonnet 3.5 is very, very good at self-correction, has a lot of like grit. Claude will try things that fail and then try, you know, come back and sort of try different approaches. And I think that's something that you didn't see in a lot of previous models. Some of the existing agent frameworks that I looked at, they had whole systems built to try to detect loops and see, oh, is the model doing the same thing, you know, more than three times, then we have to pull it out. And I think like the smarter the models are, the less you need that kind of extra scaffolding. So yeah, just giving the model tools and letting it keep sample and call tools until it thinks it's done was the most minimal framework that we could think of. And so that's what we did.Alessio [00:20:18]: So you're not pruning like bad paths from the context. If it tries to do something, it fails. You just burn all these tokens.Swyx [00:20:25]: Yes.Erik [00:20:26]: I would say the downside of this is that this is sort of a very token expensive way to doSwyx [00:20:29]: this. But still, it's very common to prune bad paths because models get stuck. Yeah.Erik [00:20:35]: But I'd say that, yeah, 3.5 is not getting stuck as much as previous models. And so, yeah, we wanted to at least just try the most minimal thing. Now, I would say that, you know, this is definitely an area of future research, especially if we talk about these problems that are going to take a human more than four hours. Those might be things where we're going to need to go prune bad paths to let the model be able to accomplish this task within 200k tokens. So certainly I think there's like future research to be done in that area, but it's not necessary to do well on these benchmarks.Swyx [00:21:06]: Another thing I always have questions about on context window things, there's a mini cottage industry of code indexers that have sprung up for large code bases, like the ones in SweetBench. You didn't need them? We didn't.Erik [00:21:18]: And I think I'd say there's like two reasons for this. One is like SweetBench specific and the other is a more general thing. The more general thing is that I think Sonnet is very good at what we call agentic search. And what this basically means is letting the model decide how to search for something. It gets the results and then it can decide, should it keep searching or is it done? Does it have everything it needs? So if you read through a lot of the traces of the SweetBench, the model is calling tools to view directories, list out things, view files. And it will do a few of those until it feels like it's found the file where the bug is. And then it will start working on that file. And I think like, again, this is all, everything we did was about just giving Claude the full reins. So there's no hard-coded system. There's no search system that you're relying on getting the correct files into context. This just totally lets Claude do it.Swyx [00:22:11]: Or embedding things into a vector database. Exactly. Oops. No, no.Erik [00:22:17]: This is very, very token expensive. And so certainly, and it also takes many, many turns. And so certainly if you want to do something in a single turn, you need to do RAG and just push stuff into the first prompt.Alessio [00:22:28]: And just to make it clear, it's using the Bash tool, basically doing LS, looking at files and then doing CAD for the following context. It can do that.Erik [00:22:35]: But it's file editing tool also has a command in it called view that can view a directory. It's very similar to LS, but it just sort of has some nice sort of quality of life improvements. So I think it'll only do an LS sort of two directories deep so that the model doesn't get overwhelmed if it does this on a huge file. I would say actually we did more engineering of the tools than the overall prompt. But the one other thing I want to say about this agentic search is that for SWE-Bench specifically, a lot of the tasks are bug reports, which means they have a stack trace in them. And that means right in that first prompt, it tells you where to go. And so I think this is a very easy case for the model to find the right files versus if you're using this as a general coding assistant where there isn't a stack trace or you're asking it to insert a new feature, I think there it's much harder to know which files to look at. And that might be an area where you would need to do more of this exhaustive search where an agentic search would take way too long.Swyx [00:23:33]: As someone who spent the last few years in the JS world, it'd be interesting to see SWE-Bench JS because these stack traces are useless because of so much virtualization that we do. So they're very, very disconnected with where the code problems are actually appearing.Erik [00:23:50]: That makes me feel better about my limited front-end experience, as I've always struggled with that problem.Swyx [00:23:55]: It's not your fault. We've gotten ourselves into a very, very complicated situation. And I'm not sure it's entirely needed. But if you talk to our friends at Vercel, they will say it is.Erik [00:24:04]: I will say SWE-Bench just released SWE-Bench Multimodal, which I believe is either entirely JavaScript or largely JavaScript. And it's entirely things that have visual components of them.Swyx [00:24:15]: Are you going to tackle that? We will see.Erik [00:24:17]: I think it's on the list and there's interest, but no guarantees yet.Swyx [00:24:20]: Just as a side note, it occurs to me that every model lab, including Enthopic, but the others as well, you should have your own SWE-Bench, whatever your bug tracker tool. This is a general methodology that you can use to track progress, I guess.Erik [00:24:34]: Yeah, sort of running on our own internal code base.Swyx [00:24:36]: Yeah, that's a fun idea.Alessio [00:24:37]: Since you spend so much time on the tool design, so you have this edit tool that can make changes and whatnot. Any learnings from that that you wish the AI IDEs would take in? Is there some special way to look at files, feed them in?Erik [00:24:50]: I would say the core of that tool is string replace. And so we did a few different experiments with different ways to specify how to edit a file. And string replace, basically, the model has to write out the existing version of the string and then a new version, and that just gets swapped in. We found that to be the most reliable way to do these edits. Other things that we tried were having the model directly write a diff, having the model fully regenerate files. That one is actually the most accurate, but it takes so many tokens, and if you're in a very big file, it's cost prohibitive. There's basically a lot of different ways to represent the same task. And they actually have pretty big differences in terms of model accuracy. I think Eider, they have a really good blog where they explore some of these different methods for editing files, and they post results about them, which I think is interesting. But I think this is a really good example of the broader idea that you need to iterate on tools rather than just a prompt. And I think a lot of people, when they make tools for an LLM, they kind of treat it like they're just writing an API for a computer, and it's sort of very minimal. It's sort of just the bare bones of what you'd need, and honestly, it's so hard for the models to use those. Again, I come back to anthropomorphizing these models. Imagine you're a developer, and you just read this for the very first time, and you're trying to use it. You can do so much better than just sort of the bare API spec of what you'd often see. Include examples in the description. Include really detailed explanations of how things work. And I think that, again, also think about what is the easiest way for the model to represent the change that it wants to make. For file editing, as an example, writing a diff is actually... Let's take the most extreme example. You want the model to literally write a patch file. I think patch files have at the very beginning numbers of how many total lines change. That means before the model has actually written the edit, it needs to decide how many numbers or how many lines are going to change.Swyx [00:26:52]: Don't quote me on that.Erik [00:26:54]: I think it's something like that, but I don't know if that's exactly the diff format. But you can certainly have formats that are much easier to express without messing up than others. And I like to think about how much human effort goes into designing human interfaces for things. It's incredible. This is entirely what FrontEnd is about, is creating better interfaces to kind of do the same things. And I think that same amount of attention and effort needs to go into creating agent computer interfaces.Swyx [00:27:19]: It's a topic we've discussed, ACI or whatever that looks like. I would also shout out that I think you released some of these toolings as part of computer use as well. And people really liked it. It's all open source if people want to check it out. I'm curious if there's an environment element that complements the tools. So how do you... Do you have a sandbox? Is it just Docker? Because that can be slow or resource intensive. Do you have anything else that you would recommend?Erik [00:27:47]: I don't think I can talk about sort of public details or about private details about how we implement our sandboxing. But obviously, we need to have sort of safe, secure, and fast sandboxes for training for the models to be able to practice writing code and working in an environment.Swyx [00:28:03]: I'm aware of a few startups working on agent sandboxing. E2B is a close friend of ours that Alessio has led around in, but also I think there's others where they're focusing on snapshotting memory so that it can do time travel for debugging. Computer use where you can control the mouse or keyboard or something like that. Whereas here, I think that the kinds of tools that we offer are very, very limited to coding agent work cases like bash, edit, you know, stuff like that. Yeah.Erik [00:28:30]: I think the computer use demo that we released is an extension of that. It has the same bash and edit tools, but it also has the computer tool that lets it get screenshots and move the mouse and keyboard. Yeah. So I definitely think there's sort of more general tools there. And again, the tools we released as part of SweetBench were, I'd say they're very specific for like editing files and doing bash, but at the same time, that's actually very general if you think about it. Like anything that you would do on a command line or like editing files, you can do with those tools. And so we do want those tools to feel like any sort of computer terminal work could be done with those same tools rather than making tools that were like very specific for SweetBench like run tests as its own tool, for instance. Yeah.Swyx [00:29:15]: You had a question about tests.Alessio [00:29:16]: Yeah, exactly. I saw there's no test writer tool. Is it because it generates the code and then you're running it against SweetBench anyway, so it doesn't really need to write the test or?Swyx [00:29:26]: Yeah.Erik [00:29:27]: So this is one of the interesting things about SweetBench is that the tests that the model's output is graded on are hidden from it. That's basically so that the model can't cheat by looking at the tests and writing the exact solution. And I'd say typically the model, the first thing it does is it usually writes a little script to reproduce the error. And again, most SweetBench tasks are like, hey, here's a bug that I found. I run this and I get this error. So the first thing the model does is try to reproduce that. So it's kind of been rerunning that script as a mini test. But yeah, sometimes the model will like accidentally introduce a bug that breaks some other tests and it doesn't know about that.Alessio [00:30:05]: And should we be redesigning any tools? We kind of talked about this and like having more examples, but I'm thinking even things of like Q as a query parameter in many APIs, it's like easier for the model to like re-query than read the Q. I'm sure it learned the Q by this point, but like, is there anything you've seen like building this where it's like, hey, if I were to redesign some CLI tools, some API tool, I would like change the way structure to make it better for LLMs?Erik [00:30:31]: I don't think I've thought enough about that off the top of my head, but certainly like just making everything more human friendly, like having like more detailed documentation and examples. I think examples are really good in things like descriptions, like so many, like just using the Linux command line, like how many times I do like dash dash help or look at the man page or something. It's like, just give me one example of like how I actually use this. Like I don't want to go read through a hundred flags. Just give me the most common example. But again, so you know, things that would be useful for a human, I think are also very useful for a model.Swyx [00:31:03]: Yeah. I mean, there's one thing that you cannot give to code agents that is useful for human is this access to the internet. I wonder how to design that in, because one of the issues that I also had with just the idea of a suite bench is that you can't do follow up questions. You can't like look around for similar implementations. These are all things that I do when I try to fix code and we don't do that. It's not, it wouldn't be fair, like it'd be too easy to cheat, but then also it's kind of not being fair to these agents because they're not operating in a real world situation. Like if I had a real world agent, of course I'm giving it access to the internet because I'm not trying to pass a benchmark. I don't have a question in there more, more just like, I feel like the most obvious tool access to the internet is not being used.Erik [00:31:47]: I think that that's really important for humans, but honestly the models have so much general knowledge from pre-training that it's, it's like less important for them. I feel like versioning, you know, if you're working on a newer thing that was like, they came after the knowledge cutoff, then yes, I think that's very important. I think actually this, this is like a broader problem that there is a divergence between Sweebench and like what customers will actually care about who are working on a coding agent for real use. And I think one of those there is like internet access and being able to like, how do you pull in outside information? I think another one is like, if you have a real coding agent, you don't want to have it start on a task and like spin its wheels for hours because you gave it a bad prompt. You want it to come back immediately and ask follow up questions and like really make sure it has a very detailed understanding of what to do, then go off for a few hours and do work. So I think that like real tasks are going to be much more interactive with the agent rather than this kind of like one shot system. And right now there's no benchmark that, that measures that. And maybe I think it'd be interesting to have some benchmark that is more interactive. I don't know if you're familiar with TauBench, but it's a, it's a customer service benchmark where there's basically one LLM that's playing the user or the customer that's getting support and another LLM that's playing the support agent and they interact and try to resolve the issue.Swyx [00:33:08]: Yeah. We talked to the LMSIS guys. Awesome. And they also did MTBench for people listening along. So maybe we need MTSWE-Bench. Sure. Yeah.Erik [00:33:16]: So maybe, you know, you could have something where like before the SWE-Bench task starts, you have like a few back and forths with kind of like the, the author who can answer follow up questions about what they want the task to do. And of course you'd need to do that where it doesn't cheat and like just get the exact, the exact thing out of the human or out of the sort of user. But I think that would be a really interesting thing to see. If you look at sort of existing agent work, like a Repl.it's coding agent, I think one of the really great UX things they do is like first having the agent create a plan and then having the human approve that plan or give feedback. I think for agents in general, like having a planning step at the beginning, one, just having that plan will improve performance on the downstream task just because it's kind of like a bigger chain of thought, but also it's just such a better UX. It's way easier for a human to iterate on a plan with a model rather than iterating on the full task that sort of has a much slower time through each loop. If the human has approved this implementation plan, I think it makes the end result a lot more sort of auditable and trustable. So I think there's a lot of things sort of outside of SweetBench that will be very important for real agent usage in the world. Yeah.Swyx [00:34:27]: I will say also, there's a couple of comments on names that you dropped. Copilot also does the plan stage before it writes code. I feel like those approaches have generally been less Twitter successful because it's not prompt to code, it's prompt plan code. You know, so there's a little bit of friction in there, but it's not much. Like it's, it actually, it's, it, you get a lot for what it's worth. I also like the way that Devin does it, where you can sort of edit the plan as it goes along. And then the other thing with Repl.it, we had a, we hosted a sort of dev day pregame with Repl.it and they also commented about multi-agents. So like having two agents kind of bounce off of each other. I think it's a similar approach to what you're talking about with kind of the few shot example, just as in the prompts of clarifying what the agent wants. But typically I think this would be implemented as a tool calling another agent, like a sub-agent I don't know if you explored that, do you like that idea?Erik [00:35:20]: I haven't explored this enough, but I've definitely heard of people having good success with this. Of almost like basically having a few different sort of personas of agents, even if they're all the same LLM. I think this is one thing with multi-agent that a lot of people will kind of get confused by is they think it has to be different models behind each thing. But really it's sort of usually the same, the same model with different prompts. And yet having one, having them have different personas to kind of bring different sort of thoughts and priorities to the table. I've seen that work very well and sort of create a much more thorough and thought outSwyx [00:35:53]: response.Erik [00:35:53]: I think the downside is just that it adds a lot of complexity and it adds a lot of extra tokens. So I think it depends what you care about. If you want a plan that's very thorough and detailed, I think it's great. If you want a really quick, just like write this function, you know, you probably don't want to do that and have like a bunch of different calls before it does this.Alessio [00:36:11]: And just talking about the prompt, why are XML tags so good in Cloud? I think initially people were like, oh, maybe you're just getting lucky with XML. But I saw obviously you use them in your own agent prompts, so they must work. And why is it so model specific to your family?Erik [00:36:26]: Yeah, I think that there's, again, I'm not sure how much I can say, but I think there's historical reasons that internally we've preferred XML. I think also the one broader thing I'll say is that if you look at certain kinds of outputs, there is overhead to outputting in JSON. If you're trying to output code in JSON, there's a lot of extra escaping that needs to be done, and that actually hurts model performance across the board. Versus if you're in just a single XML tag, there's none of that sort of escaping thatSwyx [00:36:58]: needs to happen.Erik [00:36:58]: That being said, I haven't tried having it write HTML and XML, which maybe then you start running into weird escaping things there. I'm not sure. But yeah, I'd say that's some historical reasons, and there's less overhead of escaping.Swyx [00:37:12]: I use XML in other models as well, and it's just a really nice way to make sure that the thing that ends is tied to the thing that starts. That's the only way to do code fences where you're pretty sure example one start, example one end, that is one cohesive unit.Alessio [00:37:30]: Because the braces are nondescriptive. Yeah, exactly.Swyx [00:37:33]: That would be my simple reason. XML is good for everyone, not just Cloud. Cloud was just the first one to popularize it, I think.Erik [00:37:39]: I do definitely prefer to read XML than read JSON.Alessio [00:37:43]: Any other details that are maybe underappreciated? I know, for example, you had the absolute paths versus relative. Any other fun nuggets?Erik [00:37:52]: I think that's a good sort of anecdote to mention about iterating on tools. Like I said, spend time prompt engineering your tools, and don't just write the prompt, but write the tool, and then actually give it to the model and read a bunch of transcripts about how the model tries to use the tool. I think by doing that, you will find areas where the model misunderstands a tool or makes mistakes, and then basically change the tool to make it foolproof. There's this Japanese term, pokayoke, about making tools mistake-proof. You know, the classic idea is you can have a plug that can fit either way, and that's dangerous, or you can make it asymmetric so that it can't fit this way, it has to go like this, and that's a better tool because you can't use it the wrong way. So for this example of absolute paths, one of the things that we saw while testing these tools is, oh, if the model has done CD and moved to a different directory, it would often get confused when trying to use the tool because it's now in a different directory, and so the paths aren't lining up. So we said, oh, well, let's just force the tool to always require an absolute path, and then that's easy for the model to understand. It knows sort of where it is. It knows where the files are. And then once we have it always giving absolute paths, it never messes up even, like, no matter where it is because it just, if you're using an absolute path, it doesn't matter whereSwyx [00:39:13]: you are.Erik [00:39:13]: So iterations like that, you know, let us make the tool foolproof for the model. I'd say there's other categories of things where we see, oh, if the model, you know, opens vim, like, you know, it's never going to return. And so the tool is stuck.Swyx [00:39:28]: Did it get stuck? Yeah. Get out of vim. What?Erik [00:39:31]: Well, because the tool is, like, it just text in, text out. It's not interactive. So it's not like the model doesn't know how to get out of vim. It's that the way that the tool is, like, hooked up to the computer is not interactive. Yes, I mean, there is the meme of no one knows how to get out of vim. You know, basically, we just added instructions in the tool of, like, hey, don't launch commands that don't return.Swyx [00:39:54]: Yeah, like, don't launch vim.Erik [00:39:55]: Don't launch whatever. If you do need to do something, you know, put an ampersand after it to launch it in the background. And so, like, just, you know, putting kind of instructions like that just right in the description for the tool really helps the model. And I think, like, that's an underutilized space of prompt engineering, where, like, people might try to do that in the overall prompt, but just put that in the tool itself so the model knows that it's, like, for this tool, this is what's relevant.Swyx [00:40:20]: You said you worked on the function calling and tool use before you actually started this vBench work, right? Was there any surprises? Because you basically went from creator of that API to user of that API. Any surprises or changes you would make now that you have extensively dog-fooded in a state-of-the-art agent?Erik [00:40:39]: I want us to make, like, maybe, like, a little bit less verbose SDK. I think some way, like, right now, it just takes, I think we sort of force people to do the best practices of writing out sort of these full JSON schemas, but it would be really nice if you could just pass in a Python function as a tool. I think that could be something nice.Swyx [00:40:58]: I think that there's a lot of, like, Python- There's helper libraries. ... structure, you know. I don't know if there's anyone else that is specializing for Anthropic. Maybe Jeremy Howard's and Simon Willis's stuff. They all have Cloud-specific stuff that they are working on. Cloudette. Cloudette, exactly. I also wanted to spend a little bit of time with SuiteAgent. It seems like a very general framework. Like, is there a reason you picked it apart from it's the same authors as vBench, or?Erik [00:41:21]: The main thing we wanted to go with was the same authors as vBench, so it just felt sort of like the safest, most neutral option. And it was, you know, very high quality. It was very easy to modify, to work with. I would say it also actually, their underlying framework is sort of this, it's like, youSwyx [00:41:39]: know, think, act, observe.Erik [00:41:40]: That they kind of go through this loop, which is like a little bit more hard-coded than what we wanted to do, but it's still very close. That's still very general. So it felt like a good match as sort of the starting point for our agent. And we had already sort of worked with and talked with the SWE-Bench people directly, so it felt nice to just have, you know, we already know the authors. This will be easy to work with.Swyx [00:42:00]: I'll share a little bit of like, this all seems disconnected, but once you figure out the people and where they go to school, it all makes sense. So it's all Princeton. Yeah, the SWE-Bench and SuiteAgent.Erik [00:42:11]: It's a group out of Princeton.Swyx [00:42:12]: Yeah, and we had Shun Yu on the pod, and he came up with the React paradigm, and that's think, act, observe. That's all React. So they're all friends. Yep, yeah, exactly.Erik [00:42:22]: And you know, if you actually read our traces of our submission, you can actually see like think, act, observe in our logs. And we just didn't even change the printing code. So it's like doing still function calls under the hood, and the model can do sort of multiple function calls in a row without thinking in between if it wants to. But yeah, so a lot of similarities and a lot of things we inherited from SuiteAgent just as a starting point for the framework.Alessio [00:42:47]: Any thoughts about other agent frameworks? I think there's, you know, the whole gamut from very simple to like very complex.Swyx [00:42:53]: Autogen, CooEI, LandGraph. Yeah, yeah.Erik [00:42:56]: I think I haven't explored a lot of them in detail. I would say with agent frameworks in general, they can certainly save you some like boilerplate. But I think there's actually this like downside of making agents too easy, where you end up very quickly like building a much more complex system than you need. And suddenly, you know, instead of having one prompt, you have five agents that are talking to each other and doing a dialogue. And it's like, because the framework made that 10 lines to do, you end up building something that's way too complex. So I think I would actually caution people to like try to start without these frameworks if you can, because you'll be closer to the raw prompts and be able to sort of directly understand what's going on. I think a lot of times these frameworks also, by trying to make everything feel really magical, you end up sort of really hiding what the actual prompt and output of the model is, and that can make it much harder to debug. So certainly these things have a place, and I think they do really help at getting rid of boilerplate, but they come with this cost of obfuscating what's really happening and making it too easy to very quickly add a lot of complexity. So yeah, I would recommend people to like try it from scratch, and it's like not that bad.Alessio [00:44:08]: Would you rather have like a framework of tools? Do you almost see like, hey, it's maybe easier to get tools that are already well curated, like the ones that you build, if I had an easy way to get the best tool from you, andSwyx [00:44:21]: like you maintain the definition?Alessio [00:44:22]: Or yeah, any thoughts on how you want to formalize tool sharing?Erik [00:44:26]: Yeah, I think that's something that we're certainly interested in exploring, and I think there is space for sort of these general tools that will be very broadly applicable. But at the same time, most people that are building on these, they do have much more specific things that they're trying to do. You know, I think that might be useful for hobbyists and demos, but the ultimate end applications are going to be bespoke. And so we just want to make sure that the model's great at any tool that it uses. But certainly something we're exploring.Alessio [00:44:52]: So everything bespoke, no frameworks, no anything.Swyx [00:44:55]: Just for now, for now.Erik [00:44:56]: Yeah, I would say that like the best thing I've seen is people building up from like, build some good util functions, and then you can use those as building blocks. Yeah, yeah.Alessio [00:45:05]: I have a utils folder, or like all these scripts. My framework is like def, call, and tropic. And then I just put all the defaults.Swyx [00:45:12]: Yeah, exactly. There's a startup hidden in every utils folder, you know? No, totally not. Like, if you use it enough, like it's a startup, you know? At some point. I'm kind of curious, is there a maximum length of turns that it took? Like, what was the longest run? I actually don't.Erik [00:45:27]: I mean, it had basically infinite turns until it ran into a 200k context. I should have looked this up. I don't know. And so for some of those failed cases where it eventually ran out of context, I mean, it was over 100 turns. I'm trying to remember like the longest successful run, but I think it was definitely over 100 turns that some of the times.Swyx [00:45:48]: Which is not that much. It's a coffee break. Yeah.Erik [00:45:52]: But certainly, you know, these things can be a lot of turns. And I think that's because some of these things are really hard, where it's going to take, you know, many tries to do it. And if you think about like, think about a task that takes a human four hours to do. Think about how many different files you read, and like times you edit a file in four hours. That's a lot more than 100.Alessio [00:46:10]: How many times you open Twitter because you get distracted. But if you had a lot more compute, what's kind of like the return on the extra compute now? So like, you know, if you had thousands of turns or like whatever, like how much better would it get?Erik [00:46:23]: Yeah, this I don't know. And I think this is, I think sort of one of the open areas of research in general with agents is memory and sort of how do you have something that can do work beyond its context length where you're just purely appending. So you mentioned earlier things like pruning bad paths. I think there's a lot of interesting work around there. Can you just roll back but summarize, hey, don't go down this path? There be dragons. Yeah, I think that's very interesting that you could have something that that uses way more tokens without ever using at a time more than 200k. So I think that's very interesting. I think the biggest thing is like, can you make the model sort of losslessly summarize what it's learned from trying different approaches and bring things back? I think that's sort of the big challenge.Swyx [00:47:11]: What about different models?Alessio [00:47:12]: So you have Haiku, which is like, you know, cheaper. So you're like, well, what if I have a Haiku to do a lot of these smaller things and then put it back up?Erik [00:47:20]: I think Cursor might have said that they actually have a separate model for file editing.Swyx [00:47:25]: I'm trying to remember.Erik [00:47:25]: I think they were on maybe the Lex Fridman podcast where they said they have a bigger model, like write what the code should be and then a different model, like apply it. So I think there's a lot of interesting room for stuff like that. Yeah, fast supply.Swyx [00:47:37]: We actually did a pod with Fireworks that they worked with on. It's speculative decoding.Erik [00:47:41]: But I think there's also really interesting things about like, you know, paring down input tokens as well, especially sometimes the models trying to read like a 10,000 line file. That's a lot of tokens. And most of it is actually not going to be relevant. I think it'd be really interesting to like delegate that to Haiku. Haiku read this file and just pull out the most relevant functions. And then, you know, Sonnet reads just those and you save 90% on tokens. I think there's a lot of really interesting room for things like that. And again, we were just trying to do sort of the simplest, most minimal thing and show that it works. I'm really hoping that people, sort of the agent community builds things like that on top of our models. That's, again, why we released these tools. We're not going to go and do lots more submissions to SWE-Bench and try to prompt engineer this and build a bigger system. We want people to like the ecosystem to do that on top of our models. But yeah, so I think that's a really interesting one.Swyx [00:48:32]: It turns out, I think you did do 3.5 Haiku with your tools and it scored a 40.6. Yes.Erik [00:48:38]: So it did very well. It itself is actually very smart, which is great. But we haven't done any experiments with this combination of the two models. But yeah, I think that's one of the exciting things is that how well Haiku 3.5 did on SWE-Bench shows that sort of even our smallest, fastest model is very good at sort of thinking agentically and working on hard problems. Like it's not just sort of for writing simple text anymore.Alessio [00:49:02]: And I know you're not going to talk about it, but like Sonnet is not even supposed to be the best model, you know? Like Opus, it's kind of like we left it at three back in the corner intro. At some point, I'm sure the new Opus will come out. And if you had Opus Plus on it, that sounds very, very good.Swyx [00:49:19]: There's a run with SuiteAgent plus Opus, but that's the official SWE-Bench guys doing it.Erik [00:49:24]: That was the older, you know, 3.0.Swyx [00:49:25]: You didn't do yours. Yeah. Okay. Did you want to? I mean, you could just change the model name.Erik [00:49:31]: I think we didn't submit it, but I think we included it in our model card.Swyx [00:49:35]: Okay.Erik [00:49:35]: We included the score as a comparison. Yeah.Swyx [00:49:38]: Yeah.Erik [00:49:38]: And Sonnet and Haiku, actually, I think the new ones, they both outperformed the original Opus. Yeah. I did see that.Swyx [00:49:44]: Yeah. It's a little bit hard to find. Yeah.Erik [00:49:47]: It's not an exciting score, so we didn't feel like they need to submit it to the benchmark.Swyx [00:49:52]: We can cut over to computer use if we're okay with moving on to topics on this, if anything else. I think we're good.Erik [00:49:58]: I'm trying to think if there's anything else SWE-Bench related.Swyx [00:50:02]: It doesn't have to be also just specifically SWE-Bench, but just your thoughts on building agents, because you are one of the few people that have reached this leaderboard on building a coding agent. This is the state of the art. It's surprisingly not that hard to reach with some good principles. Right. There's obviously a ton of low-hanging fruit that we covered. Your thoughts on if you were to build a coding agent startup, what next?Erik [00:50:24]: I think the really interesting question for me, for all the startups out there, is this kind of divergence between the benchmarks and what real customers will want. So I'm curious, maybe the next time you have a coding agent startup on the podcast, you should ask them that. What are the differences that they're starting to make? Tomorrow.Swyx [00:50:40]: Oh, perfect, perfect. Yeah.Erik [00:50:41]: I'm actually very curious what they will see, because I also have seen, I feel like it's slowed down a little bit if I don't see the startups submitting to SWE-Bench that much anymore.Swyx [00:50:52]: Because of the traces, the trace. So we had Cosign on, they had a 50-something on full, on SWE-Bench full, which is the hardest one, and they were rejected because they didn't want to submit their traces. Yep. IP, you know? Yeah, that makes sense, that makes sense. Actually, tomorrow we're talking to Bolt, which is a cloud customer. You guys actually published a case study with them. I assume you weren't involved with that, but they were very happy with Cloud. Cool. One of the biggest launches of the year. Yeah, totally. We actually happened to b
Die sozialen Manipulationsstrategien sind allgegenwärtig, aber sie sind nur den wenigsten Menschen bewusst. Wir glauben, das muss sich ändern. Deswegen besprechen wir die wichtigsten Techniken der Herrschaft.
Guest: Sander Schulhoff, CEO and Co-Founder, Learn Prompting [@learnprompting]On LinkedIn | https://www.linkedin.com/in/sander-schulhoff/____________________________Host: Sean Martin, Co-Founder at ITSPmagazine [@ITSPmagazine] and Host of Redefining CyberSecurity Podcast [@RedefiningCyber]On ITSPmagazine | https://www.itspmagazine.com/sean-martinView This Show's Sponsors___________________________Episode NotesIn this episode of Redefining CyberSecurity, host Sean Martin engages with Sander Schulhoff, CEO and Co-Founder of Learn Prompting and a researcher at the University of Maryland. The discussion focuses on the critical intersection of artificial intelligence (AI) and cybersecurity, particularly the role of prompt engineering in the evolving AI landscape. Schulhoff's extensive work in natural language processing (NLP) and deep reinforcement learning provides a robust foundation for this insightful conversation.Prompt engineering, a vital part of AI research and development, involves creating effective input prompts that guide AI models to produce desired outputs. Schulhoff explains that the diversity of prompt techniques is vast and includes methods like the chain of thought, which helps AI articulate its reasoning steps to solve complex problems. However, the conversation highlights that there are significant security concerns that accompany these techniques.One such concern is the vulnerability of systems when they integrate user-generated prompts with AI models, especially those prompts that can execute code or interact with external databases. Security flaws can arise when these systems are not adequately sandboxed or otherwise protected, as demonstrated by Schulhoff through real-world examples like MathGPT, a tool that was exploited to run arbitrary code by injecting malicious prompts into the AI's input.Schulhoff's insights into the AI Village at DEF CON underline the community's nascent but growing focus on AI security. He notes an intriguing pattern: many participants in AI-specific red teaming events were beginners, which suggests a gap in traditional red teamer familiarity with AI systems. This gap necessitates targeted education and training, something Schulhoff is actively pursuing through initiatives at Learn Prompting.The discussion also covers the importance of studying and understanding the potential risks posed by AI models in business applications. With AI increasingly integrated into various sectors, including security, the stakes for anticipating and mitigating risks are high. Schulhoff mentions that his team is working on Hack A Prompt, a global prompt injection competition aimed at crowdsourcing diverse attack strategies. This initiative not only helps model developers understand potential vulnerabilities but also furthers the collective knowledge base necessary for building more secure AI systems.As AI continues to intersect with various business processes and applications, the role of security becomes paramount. This episode underscores the need for collaboration between prompt engineers, security professionals, and organizations at large to ensure that AI advancements are accompanied by robust, proactive security measures. By fostering awareness and education, and through collaborative competitions like Hack A Prompt, the community can better prepare for the multifaceted challenges that AI security presents.Top Questions AddressedWhat are the key security concerns associated with prompt engineering?How can organizations ensure the security of AI systems that integrate user-generated prompts?What steps can be taken to bridge the knowledge gap in AI security among traditional security professionals?___________________________SponsorsImperva: https://itspm.ag/imperva277117988LevelBlue: https://itspm.ag/attcybersecurity-3jdk3___________________________Watch this and other videos on ITSPmagazine's YouTube ChannelRedefining CyberSecurity Podcast with Sean Martin, CISSP playlist:
Guest: Sander Schulhoff, CEO and Co-Founder, Learn Prompting [@learnprompting]On LinkedIn | https://www.linkedin.com/in/sander-schulhoff/____________________________Host: Sean Martin, Co-Founder at ITSPmagazine [@ITSPmagazine] and Host of Redefining CyberSecurity Podcast [@RedefiningCyber]On ITSPmagazine | https://www.itspmagazine.com/sean-martinView This Show's Sponsors___________________________Episode NotesIn this episode of Redefining CyberSecurity, host Sean Martin engages with Sander Schulhoff, CEO and Co-Founder of Learn Prompting and a researcher at the University of Maryland. The discussion focuses on the critical intersection of artificial intelligence (AI) and cybersecurity, particularly the role of prompt engineering in the evolving AI landscape. Schulhoff's extensive work in natural language processing (NLP) and deep reinforcement learning provides a robust foundation for this insightful conversation.Prompt engineering, a vital part of AI research and development, involves creating effective input prompts that guide AI models to produce desired outputs. Schulhoff explains that the diversity of prompt techniques is vast and includes methods like the chain of thought, which helps AI articulate its reasoning steps to solve complex problems. However, the conversation highlights that there are significant security concerns that accompany these techniques.One such concern is the vulnerability of systems when they integrate user-generated prompts with AI models, especially those prompts that can execute code or interact with external databases. Security flaws can arise when these systems are not adequately sandboxed or otherwise protected, as demonstrated by Schulhoff through real-world examples like MathGPT, a tool that was exploited to run arbitrary code by injecting malicious prompts into the AI's input.Schulhoff's insights into the AI Village at DEF CON underline the community's nascent but growing focus on AI security. He notes an intriguing pattern: many participants in AI-specific red teaming events were beginners, which suggests a gap in traditional red teamer familiarity with AI systems. This gap necessitates targeted education and training, something Schulhoff is actively pursuing through initiatives at Learn Prompting.The discussion also covers the importance of studying and understanding the potential risks posed by AI models in business applications. With AI increasingly integrated into various sectors, including security, the stakes for anticipating and mitigating risks are high. Schulhoff mentions that his team is working on Hack A Prompt, a global prompt injection competition aimed at crowdsourcing diverse attack strategies. This initiative not only helps model developers understand potential vulnerabilities but also furthers the collective knowledge base necessary for building more secure AI systems.As AI continues to intersect with various business processes and applications, the role of security becomes paramount. This episode underscores the need for collaboration between prompt engineers, security professionals, and organizations at large to ensure that AI advancements are accompanied by robust, proactive security measures. By fostering awareness and education, and through collaborative competitions like Hack A Prompt, the community can better prepare for the multifaceted challenges that AI security presents.Top Questions AddressedWhat are the key security concerns associated with prompt engineering?How can organizations ensure the security of AI systems that integrate user-generated prompts?What steps can be taken to bridge the knowledge gap in AI security among traditional security professionals?___________________________SponsorsImperva: https://itspm.ag/imperva277117988LevelBlue: https://itspm.ag/attcybersecurity-3jdk3___________________________Watch this and other videos on ITSPmagazine's YouTube ChannelRedefining CyberSecurity Podcast with Sean Martin, CISSP playlist:
Get your FREE Selling Confidence™ ACTION PLAN: ▶️ www.sellingconfidencescore.com --- Thanks for listening! I appreciate you taking the time to listen and subscribe to The Daily Sales Message. James =================== Connect with me: LinkedIn Instagram YouTube =================== Want to work together? There are 4 ways I can help: (FREE) Are you selling the right thing? Find out here. (FREE) Subscribe to my newsletter The Weekly Sales Message Solve a specific selling issue with a Practical Sales Training™ course. Work with me 1:1 or in my group program - Click here for details.
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Demis Hassabis - Google DeepMind: The Podcast, published by Zach Stein-Perlman on August 16, 2024 on LessWrong. The YouTube "chapters" are mixed up, e.g. the question about regulation comes 5 minutes after the regulation chapter ends. Ignore them. Noteworthy parts: 8:40: Near-term AI is hyped too much (think current startups, VCs, exaggerated claims about what AI can do, crazy ideas that aren't ready) but AGI is under-hyped and under-appreciated. 16:45: "Gemini is a project that has only existed for a year . . . our trajectory is very good; when we talk next time we should hopefully be right at the forefront." 17:20-18:50: Current AI doesn't work as a digital assistant. The next era/generation is agents. DeepMind is well-positioned to work on agents: "combining AlphaGo with Gemini." 24:00: Staged deployment is nice: red-teaming then closed beta then public deployment. 28:37 Openness (at Google: e.g. publishing transformers, AlphaCode, AlphaFold) is almost always a universal good. But dual-use technology - including AGI - is an exception. With dual-use technology, you want good scientists to still use the technology and advance as quickly as possible, but also restrict access for bad actors. Openness is fine today but in 2-4 years or when systems are more agentic it'll be dangerous. Maybe labs should only open-source models that are lagging a year behind the frontier (and DeepMind will probably take this approach, and indeed is currently doing ~this by releasing Gemma weights). 31:20 "The problem with open source is if something goes wrong you can't recall it. With a proprietary model if your bad actor starts using it in a bad way you can close the tap off . . . but once you open-source something there's no pulling it back. It's a one-way door, so you should be very sure when you do that." 31:42: Can an AGI be contained? We don't know how to do that [this suggests a misalignment/escape threat model but it's not explicit]. Sandboxing and normal security is good for intermediate systems but won't be good enough to contain an AGI smarter than us. We'll have to design protocols for AGI in the future: "when that time comes we'll have better ideas for how to contain that, potentially also using AI systems and tools to monitor the next versions of the AI system." 33:00: Regulation? It's good that people in government are starting to understand AI and AISIs are being set up before the stakes get really high. International cooperation on safety and deployment norms will be needed since AI is digital and if e.g. China deploys an AI it won't be contained to China. Also: Because the technology is changing so fast, we've got to be very nimble and light-footed with regulation so that it's easy to adapt it to where the latest technology's going. If you'd regulated AI five years ago, you'd have regulated something completely different to what we see today, which is generative AI. And it might be different again in five years; it might be these agent-based systems that [] carry the highest risks. So right now I would [] beef up existing regulations in domains that already have them - health, transport, and so on - I think you can update them for AI just like they were updated for mobile and internet. That's probably the first thing I'd do, while . . . making sure you understand and test the frontier systems. And then as things become [clearer] start regulating around that, maybe in a couple years time would make sense. One of the things we're missing is [benchmarks and tests for dangerous capabilities]. My #1 emerging dangerous capability to test for is deception because if the AI can be deceptive then you can't trust other tests [deceptive alignment threat model but not explicit]. Also agency and self-replication. 37:10: We don't know how to design a system that could come up with th...
Link to original articleWelcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Demis Hassabis - Google DeepMind: The Podcast, published by Zach Stein-Perlman on August 16, 2024 on LessWrong. The YouTube "chapters" are mixed up, e.g. the question about regulation comes 5 minutes after the regulation chapter ends. Ignore them. Noteworthy parts: 8:40: Near-term AI is hyped too much (think current startups, VCs, exaggerated claims about what AI can do, crazy ideas that aren't ready) but AGI is under-hyped and under-appreciated. 16:45: "Gemini is a project that has only existed for a year . . . our trajectory is very good; when we talk next time we should hopefully be right at the forefront." 17:20-18:50: Current AI doesn't work as a digital assistant. The next era/generation is agents. DeepMind is well-positioned to work on agents: "combining AlphaGo with Gemini." 24:00: Staged deployment is nice: red-teaming then closed beta then public deployment. 28:37 Openness (at Google: e.g. publishing transformers, AlphaCode, AlphaFold) is almost always a universal good. But dual-use technology - including AGI - is an exception. With dual-use technology, you want good scientists to still use the technology and advance as quickly as possible, but also restrict access for bad actors. Openness is fine today but in 2-4 years or when systems are more agentic it'll be dangerous. Maybe labs should only open-source models that are lagging a year behind the frontier (and DeepMind will probably take this approach, and indeed is currently doing ~this by releasing Gemma weights). 31:20 "The problem with open source is if something goes wrong you can't recall it. With a proprietary model if your bad actor starts using it in a bad way you can close the tap off . . . but once you open-source something there's no pulling it back. It's a one-way door, so you should be very sure when you do that." 31:42: Can an AGI be contained? We don't know how to do that [this suggests a misalignment/escape threat model but it's not explicit]. Sandboxing and normal security is good for intermediate systems but won't be good enough to contain an AGI smarter than us. We'll have to design protocols for AGI in the future: "when that time comes we'll have better ideas for how to contain that, potentially also using AI systems and tools to monitor the next versions of the AI system." 33:00: Regulation? It's good that people in government are starting to understand AI and AISIs are being set up before the stakes get really high. International cooperation on safety and deployment norms will be needed since AI is digital and if e.g. China deploys an AI it won't be contained to China. Also: Because the technology is changing so fast, we've got to be very nimble and light-footed with regulation so that it's easy to adapt it to where the latest technology's going. If you'd regulated AI five years ago, you'd have regulated something completely different to what we see today, which is generative AI. And it might be different again in five years; it might be these agent-based systems that [] carry the highest risks. So right now I would [] beef up existing regulations in domains that already have them - health, transport, and so on - I think you can update them for AI just like they were updated for mobile and internet. That's probably the first thing I'd do, while . . . making sure you understand and test the frontier systems. And then as things become [clearer] start regulating around that, maybe in a couple years time would make sense. One of the things we're missing is [benchmarks and tests for dangerous capabilities]. My #1 emerging dangerous capability to test for is deception because if the AI can be deceptive then you can't trust other tests [deceptive alignment threat model but not explicit]. Also agency and self-replication. 37:10: We don't know how to design a system that could come up with th...
Did you know that apps on your phone can see which other apps you have installed? And that apps can talk to each other and share information, without you realizing? In this video we show you how to stop apps communicating with each other, and look at the scoped storage and sandboxing of apps in GrapheneOS, and what it does and doesn't protect against. 00:00 Apps Can Talk to Each Other02:18 Sandboxing and Using Scored Storage on GrapheneOS04:58 Secondary Profiles10:24 Google Play Services: How To Stop Their Tentacles12:10 Setting Up 2nd Profile Tutorial13:40 ConclusionSecondary profiles are a great privacy tool to add to your toolbox. It's super opaque what the apps on our phones are doing, and using something like GrapheneOS really helps you reclaim your privacy by default.Brought to you by NBTV team members: Lee Rennie, Cube Boy, Sam Ettaro, Will Sandoval and Naomi BrockwellWatch all of the tutorials in our Phone Privacy Playlist HERE:https://www.youtube.com/playlist?list=PLt3zZ-N423gXV-0pdxnRn-nw0WzVKh4NYTo support NBTV, visit https://www.nbtv.media/support(tax-deductible in the US)NBTV's new eBook out now!Beginner's Introduction To Privacy - https://amzn.to/3WDSfkuBeware of scammers, I will never give you a phone number or reach out to you with investment advice. I do not give investment advice.Visit the NBTV website:https://nbtv.mediaSupport the Show.
Tune in to the latest episode of AVWeek with Tim Albright and industry experts as they delve into the latest news and updates from the commercial AV world. In this episode, they've uncovered the shocking revelation of a backdoor in XZ Utils for Linux systems, enabling remote code execution, and explore the implications for system security, including the importance of sandboxing to mitigate such risks. Additionally, they've delved into the conversation on the power of industry credentials like the CTS-D in propelling careers in AV and IT, with inspiring stories and valuable career advancement strategies. See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
On this episode of the BackTable OBGYN Podcast, host Dr. Mark Hoffman is joined by Dr. Arpit Davé, an assistant professor at Penn State Health Milton S. Hershey Medical Center in the Department of Obstetrics and Gynecology. Together, they discuss the importance of surgical education and best practices for teaching new generations of surgeons. Both Dr. Davé and Mark emphasize TATA, or tools, access, tissue handling, and anatomy, when practicing and teaching how to master surgery. They discuss the benefits of fostering a “sandbox-learning” environment, or a zone of safety where learners can practice techniques on patients. They also delve into systematic approaches for surgical training and the challenges in measuring the progress of trainees. Most importantly, Dr. Davé and Mark explore how to teach trainees not just surgery, but how to learn about surgery so that they feel competent doing new surgeries as their career in medicine progresses. --- SHOW NOTES 00:00 - Introduction 04:34 - The Role of Teaching in Medicine and Lifelong Learning in Surgery 07:15 - The Challenges of Surgical Training Volume 09:22 - The Journey of Learning and Teaching Surgery 17:59 - Understanding TATA: Surgical Tools, Access, Tissue Handling, and Anatomy 27:01 - The Importance of Practice in Surgical Training 30:04 - The Role of Tissue Handling in Surgical Training 31:20 - Creating Zones of Safety in Surgical Practice 33:31 - The Concept of “Sandboxing” in Surgical Training 34:27 - The Importance of Incremental Learning in Surgery 35:22 - The Importance of Breaking Down Surgical Procedures into Steps 42:32 - The Meaning of “Access” in Surgery 47:26 - How to Teach Trainees to Handle Tough Surgeries and the Unknown 50:05 - The Future of Surgical Training and Education
Schickt ihr eure Software auch mal in den Sandkasten? In Türchen Nummer 16 geht es um Sandboxing, eine Möglichkeit, das Schadenspotenzial von Software zu reduzieren.
OAuth implementation failures, the State of DevOps report, data poisoning generative AIs with Nightshade, implementing spectre attacks with JavaScript and WebAssembly against WebKit, sandboxing apps Show Notes: https://securityweekly.com/asw-261
OAuth implementation failures, the State of DevOps report, data poisoning generative AIs with Nightshade, implementing spectre attacks with JavaScript and WebAssembly against WebKit, sandboxing apps Show Notes: https://securityweekly.com/asw-261
This MacVoices Live! session continues the discussion of the new UK rules regarding Apple's App Store and their impact on app purchases and security. The group endorses Setapp for installing apps on Macs but question comfort in installing apps outside the App Store. Chuck Joiner, David Ginsburg, Eric Bolden, Jim Rea, Jeff Gamet, Web Bixby, and Brian Flanigan-Arthurs compare the difference between third-party apps on Mac and iOS, emphasizing the importance of sandboxing for security, and cover challenges for developers listing apps outside the App Store and speculate on upcoming changes in the EU. The conversation concludes with concerns over Apple's authority in deciding which apps can exist and the potential advantages for Apple's own App Store. (Part 2) This edition of MacVoices is supported by MacVoices After Dark. What happens before and after the shows is uncensored, on-topic, off-topic, and always off the wall. Sign up as a MacVoices Patron and get access! http://patreon.com/macvoices Show Notes: Chapters: 0:00:27 The Potential Impact of New UK Rules on App Reliability 0:02:45 Stepping Outside Apple's Walled Garden: Concerns and Risks 0:04:25 The Importance of App Notarization on macOS 0:05:29 Vetting in App Store Process: Limited Protection on iOS 0:06:59 Mac OS lacks compatibility, causing frustration among users. 0:07:51 Perception vs. Reality: Safety of Apps from App Store 0:09:00 Introduction and SetApp's' iOS apps 0:10:34 Potential Challenges for Developers Listing Apps Only in App Store 0:12:04 Notarization and the App Approval Process 0:15:02 Apple's control over app existence 0:18:19 Apple's plan to ensure revenue from app sales 0:18:57 Apple's Fight Against EU Regulations 0:21:05 Apple's Compliance Strategy and Potential Legal Battles 0:27:42 The Future of App Stores on the Moon Links: Epic loses bid to make Apple change its App Store payment rules right now https://www.engadget.com/epic-loses-bid-to-make-apple-change-its-app-store-payment-rules-right-now-174924222.html SetApp Setapp Planning to Launch Alternative App Store for iOS in Europe https://www.macrumors.com/2023/08/15/setapp-to-launch-alternative-app-store/ Apple is ‘working with EU' on third-party app stores for iPhone, iPad, etc. https://macdailynews.com/2023/06/14/apple-is-working-with-eu-on-third-party-app-stores-for-iphone-ipad-etc/ Guests: Web Bixby has been in the insurance business for 40 years and has been an Apple user for longer than that.You can catch up with him on Facebook, Twitter, and LinkedIn. Eric Bolden is into macOS, plants, sci-fi, food, and is a rural internet supporter. You can connect with him on Twitter, by email at embolden@mac.com, on Mastodon at @eabolden@techhub.social, and on his blog, Trending At Work. Brian Flanigan-Arthurs is an educator with a passion for providing results-driven, innovative learning strategies for all students, but particularly those who are at-risk. He is also a tech enthusiast who has a particular affinity for Apple since he first used the Apple IIGS as a student. You can contact Brian on twitter as @brian8944. He also recently opened a Mastodon account at @brian8944@mastodon.cloud. Jeff Gamet is a technology blogger, podcaster, author, and public speaker. Previously, he was The Mac Observer's Managing Editor, and the TextExpander Evangelist for Smile. He has presented at Macworld Expo, RSA Conference, several WordCamp events, along with many other conferences. You can find him on several podcasts such as The Mac Show, The Big Show, MacVoices, Mac OS Ken, This Week in iOS, and more. Jeff is easy to find on social media as @jgamet on Twitter and Instagram, jeffgamet on LinkedIn., @jgamet@mastodon.social on Mastodon, and on his YouTube Channel at YouTube.com/jgamet. David Ginsburg is the host of the weekly podcast In Touch With iOS where he discusses all things iOS, iPhone, iPad, Apple TV, Apple Watch, and related technologies. He is an IT professional supporting Mac, iOS and Windows users. Visit his YouTube channel at https://youtube.com/daveg65 and find and follow him on Twitter @daveg65 and on Mastodon at @daveg65@mastodon.cloud Jim Rea built his own computer from scratch in 1975, started programming in 1977, and has been an independent Mac developer continuously since 1984. He is the founder of ProVUE Development, and the author of Panorama X, ProVUE's ultra fast RAM based database software for the macOS platform. He's been a speaker at MacTech, MacWorld Expo and other industry conferences. Follow Jim at provue.com and via @provuejim@techhub.social on Mastodon. Support: Become a MacVoices Patron on Patreon http://patreon.com/macvoices Enjoy this episode? Make a one-time donation with PayPal Connect: Web: http://macvoices.com Twitter: http://www.twitter.com/chuckjoiner http://www.twitter.com/macvoices Mastodon: https://mastodon.cloud/@chuckjoiner Facebook: http://www.facebook.com/chuck.joiner MacVoices Page on Facebook: http://www.facebook.com/macvoices/ MacVoices Group on Facebook: http://www.facebook.com/groups/macvoice LinkedIn: https://www.linkedin.com/in/chuckjoiner/ Instagram: https://www.instagram.com/chuckjoiner/ Subscribe: Audio in iTunes Video in iTunes Subscribe manually via iTunes or any podcatcher: Audio: http://www.macvoices.com/rss/macvoicesrss Video: http://www.macvoices.com/rss/macvoicesvideorss 00:00:27 The Potential Impact of New UK Rules on App Reliability 00:02:44 Stepping Outside Apple's Walled Garden: Concerns and Risks 00:04:25 The Importance of App Notarization on macOS 00:05:28 Vetting in App Store Process: Limited Protection on iOS 00:06:59 Mac OS lacks compatibility, causing frustration among users. 00:07:51 App setup or direct downloads won't alter app siloing. 00:08:59 Introduction and SetApp's iOS apps 00:10:33 Potential Challenges for Developers Listing Apps Only in App Store 00:12:03 Notarization and the App Approval Process 00:15:01 Apple's control over app existence 00:18:18 Apple's plan to ensure revenue from app sales 00:18:57 Apple's Fight Against EU Regulations 00:21:04 Apple's Compliance Strategy and Potential Legal Battles 00:27:41 The Future of App Stores on the Moon
Reach and engage with your audience! Check out Moosend free for 30 days at https://lmg.gg/moo Reserve the next generation of Samsung Galaxy devices at https://lmg.gg/galaxyunpacked Looking for electronic components and equipment? Consult the specialists! Head over to https://lmg.gg/CircuitSpecialists and save 10% using code “LMG” Timestamps (Courtesy of NoKi1119) Note: Timing may be off due to sponsor change: 0:00 Chapters. 1:26 Intro. 1:53 Topic #1: LTT's video accidentally DDoSes Medicat USB. 2:43 Who has LTT done this to before? 3:53 Medicat DDoSed again, LTT's history of overloading pages. 8:34 Topic #2: NVIDIA won't make FE RTX 4060 Ti 16GB. 10:19 NVIDIA won't send review samples, "why not do something?" 12:44 Linus mentions the spot price of DRAM. 14:30 HU's quote on TPMs' response to RTX 4060 Ti. 15:37 Why wouldn't TPMs add more memory? Linus's scenario. 18:28 LTT's incoming video, Linus on controlled & soldered chips. 25:24 Discussing scrapped shows by networks. 28:19 Screen Actors Guilds' stupid guidelines. 29:12 Linus on playing games, Linus's "vision." 31:22 LMG's past of studios sponsoring. 32:39 Merch Messages #1. 33:28 Private V.S. public sector work. 39:24 Worst time you've accidentally violated an NDA? 43:11 Topic #3: Proposed Cooper Davis Act forces sites to report users to DEA. 46:46 Possible over-reporting users for "suspicious" activity. 50:12 Discussing prescriptions V.S. concrete proof. 52:22 LifeLock offers a free year after identity theft. 55:32 Linus on second-hand car sales tax, discussing tax. 59:18 Topic #4: Water Cooled PC Build of the Month. 1:04:22 Sponsors. 1:09:34 LTT Screwdriver Stubby ft. Funny camera. 1:11:31 Launch date, Luke showcases the Stubby. 1:17:23 LTX 2023 exclusive merch. 1:19:12 LTT backpack update. 1:19:47 LTT x iFixit screwdriver ft. "Work," funny camera. 1:26:04 Merch Messages #2. 1:26:09 Would Linus be on time for LTX WAN Show? Merch messages via LTX booths? 1:27:20 Would Linus make a deodorant? ft. MrBeast Burgers. 1:33:43 What would Labs want to take an X-ray first? 1:34:26 Inspections, products of choice, food discussion. 1:51:31 Topic #5: "Glorbo" returns to World of Warcraft. 1:52:43 "Bot-operated news website," funny Reddit post & article. 1:55:14 Topic #6: Refurbishing phone screens using LASERs. 1:56:40 Topic #7: Activision restores old COD servers. 1:58:21 BattleBit, graphics V.S. gameplay, nostalgic games. 2:11:52 Luke shows TARKOV's K/D rating leaderboard. 2:13:26 Nintendo's remake of Super Mario RPG. 2:14:52 Topic #8: Meta discontinues Quest Pro. 2:21:08 Topic #9: Corsair purchases Drop. 2:24:24 Topic #10: ASUS now manufacturers NUCs. 2:24:49 Topic #11: Military information leaked due to a mistype. 2:25:32 Merch Messages #3 ft. After Dark WAN Show. 2:26:46 Luke's possible United Launch Alliance Testing Lab tour. 2:31:10 How did last week's free shipping shake out? ft. Calling Savage Nick. 2:40:10 Is a single cable eGPU setup viable in 2023? 2:44:08 How did Linus develop his writing & editing style? 2:47:10 Most stress Dan had producing something live? How did you manage? 2:48:35 How do you think anti-cheat devs will react to Sandboxing? 2:50:48 Framework Laptop 16" AMD Ryzen DIY is live. 2:57:10 Linus's conversation with Terren, Linus & Luke working together. Cont. Merch Messages #3. 3:01:10 Clear purple screwdriver when? 3:07:55 When will LTTStore backpacks start shipping with new zipper pulls? 3:09:01 What's your favorite meal to make yourself? 3:11:40 What is the best and dumbest convention swag you purchased or been given? 3:13:56 Ever thought of getting a dead mall? 3:14:14 Donating my old motherboard & RAM to students? Would Labs sponsor tech students? 3:15:38 Do you expect Apple to avoid EU's removable batteries bill? 3:15:50 Chances of getting LTTStore jeans? 3:16:15 Luke's FP creative day. 3:18:28 Selling LMG items on Microcenter? 3:20:53 Would LTT exist if you never met Luke? 3:29:09 Outro
Guest: Jason Haddix, CISO and Hacker in Charge at BuddoBot Inc [@BuddoBot]On LinkedIn | https://www.linkedin.com/in/jhaddix/On Twitter | https://twitter.com/Jhaddix____________________________Host: Sean Martin, Co-Founder at ITSPmagazine [@ITSPmagazine] and Host of Redefining CyberSecurity Podcast [@RedefiningCyber]On ITSPmagazine | https://www.itspmagazine.com/itspmagazine-podcast-radio-hosts/sean-martin____________________________This Episode's SponsorsImperva | https://itspm.ag/imperva277117988Pentera | https://itspm.ag/penteri67a___________________________Episode NotesIn this Redefining CyberSecurity Podcast, we provide an in-depth exploration of the potential implications of large language models (LLMs) and artificial intelligence in the cybersecurity landscape. Jason Haddix, a renowned expert in offensive security, shares his perspective on the evolving risks and opportunities that these new technologies bring to businesses and individuals alike. Sean and Jason explore the potential risks of using LLMs:
Guest: Jason Haddix, CISO and Hacker in Charge at BuddoBot Inc [@BuddoBot]On LinkedIn | https://www.linkedin.com/in/jhaddix/On Twitter | https://twitter.com/Jhaddix____________________________Host: Sean Martin, Co-Founder at ITSPmagazine [@ITSPmagazine] and Host of Redefining CyberSecurity Podcast [@RedefiningCyber]On ITSPmagazine | https://www.itspmagazine.com/itspmagazine-podcast-radio-hosts/sean-martin____________________________This Episode's SponsorsImperva | https://itspm.ag/imperva277117988Pentera | https://itspm.ag/penteri67a___________________________Episode NotesIn this Redefining CyberSecurity Podcast, we provide an in-depth exploration of the potential implications of large language models (LLMs) and artificial intelligence in the cybersecurity landscape. Jason Haddix, a renowned expert in offensive security, shares his perspective on the evolving risks and opportunities that these new technologies bring to businesses and individuals alike. Sean and Jason explore the potential risks of using LLMs:
We're back for another episode with Umut Alemdar - Head of Security Lab here at Hornetsecurity. Today, we're discussing Advanced Threat Protection (ATP) and its crucial role in detecting, preventing, and responding to increasingly sophisticated cyber threats. Throughout the episode, Andy and Umut discuss common ATP techniques such as sandboxing, time of click protection, and spam filters, all of which are critical in fortifying defenses against malicious actors. Furthermore, they emphasize the vital function of the natural language understanding module in ATP in detecting sophisticated social engineering attacks. While this episode focuses on ATP in general, Andy and Umut draw concrete examples from our own ATP scanning methods here at Hornetsecurity. Timestamps: 2:05 – What is Advanced Threat Protection 5:50 – What are common scanning techniques used by ATP technologies 10:35 – How does Sandboxing work in ATP scanning techniques? 13:07 – What is the role of AI within ATP scanning? 18:09 – Concrete example of where ATP saves the day 20:11 – Scanning for malicious QR codes Episode Resources: Advanced Threat Protection We used ChatGPT to Create Ransomware Bit.ly QR Code Index Andy on LinkedIn, Twitter or Mastodon Umut on LinkedIn
Mein Gast heute: Daniel Buhmann arbeitet als Principal Systems Engineer und Expert für Operational Technology & IoT bei Fortinet. Mit mehr als 15 Jahren Erfahrung in der industriellen Cybersicherheit arbeitet er eng mit Kunden und Partnern zusammen, um Sicherheitslösungen entsprechend ihren Bedürfnissen zu entwerfen und dabei die Anforderungen und Einschränkungen industrieller Umgebungen zu berücksichtigen. Wir reden gemeinsam über die Endpoint Security bei Legacy Geräten, Whitelisting vs Sandboxing, die Herausforderungen bei der Umsetzung, häufige Fehler und die Zukunft EDR.Fortinet OT: https://www.fortinet.com/ot
Mein Gast heute: Daniel Buhmann arbeitet als Principal Systems Engineer und Expert für Operational Technology & IoT bei Fortinet. Mit mehr als 15 Jahren Erfahrung in der industriellen Cybersicherheit arbeitet er eng mit Kunden und Partnern zusammen, um Sicherheitslösungen entsprechend ihren Bedürfnissen zu entwerfen und dabei die Anforderungen und Einschränkungen industrieller Umgebungen zu berücksichtigen. Wir reden gemeinsam über die Endpoint Security bei Legacy Geräten, Whitelisting vs Sandboxing, die Herausforderungen bei der Umsetzung, häufige Fehler und die Zukunft EDR.Fortinet OT: https://www.fortinet.com/ot
Mein Gast heute: Daniel Buhmann arbeitet als Principal Systems Engineer und Expert für Operational Technology & IoT bei Fortinet. Mit mehr als 15 Jahren Erfahrung in der industriellen Cybersicherheit arbeitet er eng mit Kunden und Partnern zusammen, um Sicherheitslösungen entsprechend ihren Bedürfnissen zu entwerfen und dabei die Anforderungen und Einschränkungen industrieller Umgebungen zu berücksichtigen. Wir reden gemeinsam über die Endpoint Security bei Legacy Geräten, Whitelisting vs Sandboxing, die Herausforderungen bei der Umsetzung, häufige Fehler und die Zukunft EDR.Fortinet OT: https://www.fortinet.com/ot
Hello and welcome to Crash's Course, a short form podcast where I share my thoughts and advice on playing and running tabletop role playing games in roughly about 5 minutes. Last time we talked about railroading, so in this episode I want to cover the other extreme: Sandboxing. You've likely seen the old joke: a … Continue reading "Crash's Course Ep 04: Sandboxing"
Link to Blog Post This week's Cyber Security Headlines – Week in Review, October 31-November 4, is hosted by Rich Stroffolino with our guest, Marcos Marrero, CISO, H.I.G. Capital Thanks to today's episode sponsor, Votiro UFOs are everywhere. They're in your applications, cloud storage, endpoints, and emails. That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/UFOs All links and the video of this episode can be found on CISO Series.com
Cyber incident at Boeing subsidiary causes flight planning disruptions Stripe to lay off 14% of workforce Over 250 US news websites deliver malware via supply chain attack Thanks to today's episode sponsor, Votiro UFOs are everywhere. They're in your applications, cloud storage, endpoints, and emails. That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/ufos For the stories behind the headlines, head to CISOseries.com.
W4SP malware stings PyPI LastPass warns of security hubris Dropbox breached Thanks to today's episode sponsor, Votiro UFOs are everywhere.They're in your applications, cloud storage, endpoints, and emails.That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/
LockBit dominates ransomware CISA on voting integrity A call for more ransomware reporting Thanks to today's episode sponsor, Votiro UFOs are everywhere.They're in your applications, cloud storage, endpoints, and emails.That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/
Threat group rides antivirus software to install malware White House organizes ransomware summit Ed tech company exposed user data Thanks to today's episode sponsor, Votiro UFOs are everywhere.They're in your applications, cloud storage, endpoints, and emails.That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/
Thomson Reuters leaks 3TB of sensitive data Massive cyberattack hits Slovak and Polish Parliaments Twitter trolls bombard platform after Elon Musk takeover Thanks to today's episode sponsor, Votiro UFOs are everywhere. They're in your applications, cloud storage, endpoints, and emails. That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/ufos. For the stories behind the headlines, head to CISOseries.com.
Link to Blog Post This week's Cyber Security Headlines – Week in Review, October 24-28, is hosted by Rich Stroffolino with our guest, Will Gregorian, former Senior Director, Technology Operations and Security, Rhino Thanks to this week's episode sponsor, Votiro UFOs are everywhere. They're in your applications, cloud storage, endpoints, and emails. That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/UFOs. All links and the video of this episode can be found on CISO Series.com
Russia warns West: We can target your commercial satellites New York Post says its site was hacked after posting offensive tweets White House announces 100-day cyber sprint for chemical sector Thanks to this week's episode sponsor, Votiro UFOs are everywhere.They're in your applications, cloud storage, endpoints, and emails. That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/UFOs For the stories behind the headlines, head to CISOseries.com.
Sigstore opens free software signing service Australian health insurer hacked Researcher details 20-year old SQLite bug Thanks to this week's episode sponsor, Votiro UFOs are everywhere.They're in your applications, cloud storage, endpoints, and emails. That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/UFOs
See Tickets discloses 2.5 year-long credit card breach US charges Chinese agents in Huawei obstruction case Hive begins leaking Tata Power's data Thanks to this week's episode sponsor, Votiro UFOs are everywhere.They're in your applications, cloud storage, endpoints, and emails. That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/UFOs For the stories behind the headlines, visit CISOseries.com
CISA warns of Daixin Team Exploit POCs used to host malware Iranian nuclear agency hacked Thanks to this week's episode sponsor, Votiro UFOs are everywhere.They're in your applications, cloud storage, endpoints, and emails. That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/UFOs
Exploited Windows zero-day lets JavaScript files bypass Mark of the Web security warnings FBI warns of ‘hack-and-leak' operations from group based in Iran Wholesale giant METRO confirmed to have suffered a cyberattack Thanks to this week's episode sponsor, Votiro UFOs are everywhere. They're in your applications, cloud storage, endpoints, and emails. That's right – UFOs – Unidentified File Objects – are hiding in files across your organization. UFOs can contain malware that exfiltrates data or deploys ransomware. And 70% of UFOs can't be detected by traditional scanning solutions like Anti-Virus and Sandboxing. That's where Votiro comes in. Votiro prevents UFOs before they hitch a ride in on files – without detection, and without slowing down business. Do you believe? Learn more at Votiro.com/UFOs For the stories behind the headlines, head to CISOseries.com.
Guest Matt Massicotte - Twitter @mattieChime Youtube Video: https://youtu.be/dc7x04Ao2xURelated Episodes Episode 130 - macOS by Tutorials with Sarah Reichelt Episode 89 - Cryptography with Marcin Krzyżanowski Episode 45 - Developer Community (Part 1) with Dave Verwer Related Links AnyCodable by FlightSchool An Introduction to ExtensionKit by Matt STTextView by Marcin Krzyzanowski Tree-sitter Short story about OpenPGP for iOS and OS X — ObjectivePGP by Marcin Krzyzanowski ExtensionKit - Apple Docs Mac App Store and investing engineering time BY Kaleidoscope SponsorsBushel - the macOS virtual machine app for developersI'm looking for beta testers! For developers who want to be rigorous and uncompromising in their app testing. You can set up your virtual machine for almost any configuration, from a fresh, factory reset of the Ventura beta all the way back to Big Sur. Test, simulate, roll back and debug apps and scripts however you need to without worrying about destroying your machine. If you want to be invited to our first TestFlight, or even if you just want updates on Bushel, sign up at the website, and we will get in touch with you.swiftpackageindex.com is the place to find Swift packages. over 5,000 packages indexed now, you'll find a package that can help. help you make better decisions about your dependencies. host DocC-based documentation for package authors. You can see how well maintained every package is, what platforms and Swift versions it's compatible with based on real-world build data, how many other dependencies it will bring in and much more. Unlike an open-source library, running an open-source website requires ongoing time for maintenance and supporting package authors in addition to the time we spend on new features. Our work is primarily funded by you - the Swift communityIf the site has helped you find a package, or if you want to support a community-run open-source project, please go to swiftpackageindex.com, look for the pink heart, and join over a hundred other people who support our work through GitHub sponsors.Open Source and Mac App Fear of dependencies What's the benefit of doing it in open source What kind of open source licenses are there How can you avoid your code being copied outside the license For a larger company what benefits do they get by open sourcing part of their code base? How does open sourcing work with iOS/mac apps and the App Store ExtensionKit What is ExtensionKit and how are you using it? How is it related to XPC How does something like this get installed and distributed How does Sandboxing relate to this What are some good candidates for using ExtensionKit Have you looked at the work iOS, watchOS, or tvOS? Social MediaEmailleo@brightdigit.comGitHub - @brightdigitTwitter BrightDigit - @brightdigitLeo - @leogdionLinkedInBrightDigitLeoInstagram - @brightdigitPatreon - empowerappshowCreditsMusic from https://filmmusic.io"Blippy Trance" by Kevin MacLeod (https://incompetech.com)License: CC BY (http://creativecommons.org/licenses/by/4.0/) ★ Support this podcast on Patreon ★
We're back in Derry and Toms to talk gaming again, this time with CLARKY THE CRUEL. Dave gets shocked by the crap number of TV channels we had in the UK as kids, Clarky tells us about his sandboxing adventures beyond the ultraworld, and I get inspired. And we digress. Obviously. You can find Clarky's Dissecting Worlds podcast archive and all of his game logs on his blog (including entires by yours truly describing the action from the perspective of post-apocalypse fiction powerhouse Roy Saveloy. Our banner art and logo is by Simon Perrins. Follow him on Twitter and check out his store. Listen to BITR Breakfast in the Ruins Radio on Radio Garden. We also mention Jason's podcast, the Nerd's RPG Variety Cast.
Episode 76 features Toby Bloomberg, a digital marketer, co-founder of Diva Foodies, host of #FoodTVChat, and she's been digitally marketing for almost as long as I have been aware of digital marketing. We discuss how much of that started, being intentional, food and creativity, engagement, and much much more. Throughout the conversation, we discuss: Finding a niche "Be intentional" Money and decision making Marketing and Art/Creativity Foorpreneuer Putting yourself out there Living for the approval of your creations Food is universal What is a "Foodie?" Where do you eat? Cooking at home Food marketing Twitter Live-tweeting Twitter Chats Sandboxing Engagement Food Tourism Culture And much more Mentioned and Helpful Links from This Episode Diva Foodies divafoodies on Instagram AgentPalmer.com Tweets @TobyDiva @DivaFoodies @ThePalmerFiles @AgentPalmer Other Links Field of Red Tape: ‘Our Team' is a fantastic study of stadium ownership through the lens of the SWB Red Barons Master of Redemption: Eddie Munson, Metal Lords, and metalhead vindication You can also hear more Palmer occasionally on Our Liner Notes, a musical conversation podcast with host Chris Maier or as co-host of The Podcast Digest with Dan Lizette. Music created and provided by Henno Heitur of Monkey Tongue Productions. --End Show Notes Transmission--
The ol' Railroading GM gets a bad rap in tabletop gaming. And perhaps rightly so. But what if I told you that every game with a planned story arc has rails? Learn the sliding scale of 4 different ways to run your games, from Hard Rails to Soft Rails and Choose Your Own Adventure to Sandboxes. Pick up on the pitfalls and player perspectives on each. And discover how to turn each of them into versatile tools for your GM toolbox. All on today's episode! Like our TTRPG content? Leave a review! Tell a friend! And interact with us on Twitter! --- This episode is sponsored by · Anchor: The easiest way to make a podcast. https://anchor.fm/app --- Send in a voice message: https://anchor.fm/epicblunders/message
The ideas that made Unix, hints for writing Unix tools, cron best practices, three different sorts of filesystem errors, LibreSSL 3.5.1 released, taskwarrior to manage tasks, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Unix Philosophy: A Quick Look at the Ideas that Made Unix (https://klarasystems.com/articles/unix-philosophy-a-quick-look-at-the-ideas-that-made-unix/) Hints for writing Unix Tools (https://monkey.org/~marius/unix-tools-hints.html) News Roundup Cron best practices (https://blog.sanctum.geek.nz/cron-best-practices/) Filesystems can experience at least three different sorts of errors (https://utcc.utoronto.ca/~cks/space/blog/tech/FilesystemsThreeErrorTypes) LibreSSL 3.5.1 development branch as well as 3.4.3 (stable) and 3.3.6 released (https://undeadly.org/cgi?action=article;sid=20220318065203) Taskwarrior to manage tasks (https://adventurist.me/posts/0165) Beastie Bits Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Andrew - virtualization (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/450/feedback/Andrew%20-%20virtualization.md) Brad - jails applications and interoperability (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/450/feedback/brad%20-%20jails%20applications%20and%20interoperability.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
In this episode, Audrow Nash speaks to Christian Fritz, CEO and founder of Transitive Robotics. Transitive Robotics makes software for building full stack robotics applications. In this conversation, they talk about how Transitive Robotic’s software works, their business model, sandboxing for security, creating a marketplace for robotics applications, and web tools, in general.EPISODE LINKS:– Christian’s LinkedIn: https://www.linkedin.com/in/christianfritz/– Transitive Robotics’ Website: https://transitiverobotics.com/– Transitive Robotics on LinkedIn: https://www.linkedin.com/company/transitive–robotics/– Transitive Robotics’ Twitter: https://twitter.com/transitiverob– Sign up for Transitive Robotics’ Beta: https://docs.google.com/forms/d/e/1FAIpQLSctAX9NI6IlZ10e7n408NSxULLKbbSyx8rrQSsLVECtlpHbeg/viewform– Contact Transitive Robotics: mailto:support@transitiverobotics.comPODCAST INFO:– Podcast website: https://sensethinkact.com– Apple Podcasts: https://podcasts.apple.com/us/podcast/sense–think–act/id1582090036– Spotify: https://open.spotify.com/show/52wK4oMDvgijRk6E82tC5d– RSS: https://sensethinkact.com/itunes.xml– Full episodes: https://www.youtube.com/c/SenseThinkActPodcast– Clips: https://www.youtube.com/channel/UChfnCpNwZzYtZ32J–pZvNDgOUTLINE:– (0:00:00) Episode introduction– (0:01:09) Introducing Christian and Transitive Robotics– (0:06:52) Fleet management + networking challenges– (0:17:15) Problems in fleet management– (0:21:23) Business model– (0:26:21) Using MQQT for message passing– (0:30:28) Developing your own capabilities– (0:35:35) Testing capabilities– (0:36:44) Sending data: authentication and authorization– (0:43:32) Sandboxing for safety– (0:56:10) Discussing web technologies– (1:01:03) Open source business models– (1:05:38) Scaling, finding market fit, and taking investment– (1:15:57) Plans in starting a marketplace– (1:20:01) Plans to grow the team + remote work– (1:29:36) Web and robotics– (1:33:08) Web technologies on the backend– (1:34:59) Future of roboticsSOCIAL:– Twitter: https://twitter.com/sense_think_act– Discourse: https://discourse.ros.org/c/sensethinkact/71
Apple M1 Linux development reaches a key milestone and boots a usable desktop; Ubuntu reveals a new product, and the secret SUSE project that leaked this week. Plus, the essential RISC-V code landing in the Linux kernel.
Ray and Ariel discuss how to run 5th Edition Dungeons & Dragons. Should you use Sandboxing or Railroading as your foundational style? We offer some actionable tips and tricks to help Game Masters experiment with both styles. Though we use D&D as the vehicle to discuss these concepts they are applicable to other popular roleplaying games like Blades in The Dark, Star Wars Fantasy Flight, Fate Core, Call of Cthulhu, Monster of the Week, Cyberpunk Red, and Masks. If you liked the show today, please follow us on Instagram. It is the only means we have to spread the word and put our show in front of new listeners.
This week on the show, we've got some new info on the talks from EuroBSDCon, a look at sharing a single ZFS pool between Linux and BSD, Sandboxing and much more! Stay tuned for your place to B...SD! This episode was brought to you by Headlines EuroBSDcon 2016 Presentation Slides (https://2016.eurobsdcon.org/PresentationSlides/) Due to circumstances beyond the control of the organizers of EuroBSDCon, there were not recordings of the talks given at the event. However, they have collected the slide decks from each of the speakers and assembled them on this page for you Also, we have some stuff from MeetBSD already: Youtube Playlist (https://www.youtube.com/playlist?list=PLb87fdKUIo8TAMC2HJLZ7H54edD2BeGWv) Not all of the sessions are posted yet, but the rest should appear shortly MeetBSD 2016 Trip Report: Domagoj Stolfa (https://www.freebsdfoundation.org/blog/meetbsd-2016-trip-report-domagoj-stolfa/) *** Cohabiting FreeBSD and Gentoo Linux on a Common ZFS Volume (https://ericmccorkleblog.wordpress.com/2016/11/15/cohabiting-freebsd-and-gentoo-linux-on-a-common-zfs-volume/) Eric McCorkle, who has contributed ZFS support to the FreeBSD EFI boot-loader code has posted an in-depth look at how he's setup dual-boot with FreeBSD and Gentoo on the same ZFS volume. He starts by giving us some background on how the layout is done. First up, GRUB is used as the boot-loader, allowing boot of both Linux and BSD The next non-typical thing was using /etc/fstab to manage mount-points, instead of the typical ‘zfs mount' usage, (apart from /home datasets) data/home is mounted to /home, with all of its child datasets using the ZFS mountpoint system data/freebsd and its child datasets house the FreeBSD system, and all have their mountpoints set to legacy data/gentoo and its child datasets house the Gentoo system, and have their mountpoints set to legacy as well So, how did he set this up? He helpfully provides an overview of the steps: Use the FreeBSD installer to create the GPT and ZFS pool Install and configure FreeBSD, with the native FreeBSD boot loader Boot into FreeBSD, create the Gentoo Linux datasets, install GRUB Boot into the Gentoo Linux installer, install Gentoo Boot into Gentoo, finish any configuration tasks The rest of the article walks us through the individual commands that make up each of those steps, as well as how to craft a GRUB config file capable of booting both systems. Personally, since we are using EFI, I would have installed rEFInd, and chain-loaded each systems EFI boot code from there, allowing the use of the BSD loader, but to each their own! HardenedBSD introduces Safestack into base (https://hardenedbsd.org/article/shawn-webb/2016-11-27/introducing-safestack) HardenedBSD has integrated SafeStack into its base system and ports tree SafeStack (http://clang.llvm.org/docs/SafeStack.html) is part of the Code Pointer Integrity (CPI) project within clang. “SafeStack is an instrumentation pass that protects programs against attacks based on stack buffer overflows, without introducing any measurable performance overhead. It works by separating the program stack into two distinct regions: the safe stack and the unsafe stack. The safe stack stores return addresses, register spills, and local variables that are always accessed in a safe way, while the unsafe stack stores everything else. This separation ensures that buffer overflows on the unsafe stack cannot be used to overwrite anything on the safe stack.” “As of 28 November 2016, with clang 3.9.0, SafeStack only supports being applied to applications and not shared libraries. Multiple patches have been submitted to clang by third parties to add support for shared libraries.” SafeStack is only enabled on AMD64 *** pledge(2)… or, how I learned to love web application sandboxing (https://learnbchs.org/pledge.html) We've talked about OpenBSD's sandboxing mechanism pledge() in the past, but today we have a great article by Kristaps Dzonsons, about how he grew to love it for Web Sandboxing. +First up, he gives us his opening argument that should make most of you sit up and listen: I use application-level sandboxing a lot because I make mistakes a lot; and when writing web applications, the price of making mistakes is very dear. In the early 2000s, that meant using systrace(4) on OpenBSD and NetBSD. Then it was seccomp(2) (followed by libseccomp(3)) on Linux. Then there was capsicum(4) on FreeBSD and sandbox_init(3) on Mac OS X. All of these systems are invoked differently; and for the most part, whenever it came time to interface with one of them, I longed for sweet release from the nightmare. Please, try reading seccomp(2). To the end. Aligning web application logic and security policy would require an arduous (and usually trial-and-error or worse, copy-and-paste) process. If there was any process at all — if the burden of writing a policy didn't cause me to abandon sandboxing at the start. And then there was pledge(2). This document is about pledge(2) and why you should use it and love it. “ +Not convinced yet? Maybe you should take his challenge: Let's play a drinking game. The challenge is to stay out of the hospital. 1.Navigate to seccomp(2). 2. Read it to the end. 3. Drink every time you don't understand. For capsicum(4), the challenge is no less difficult. To see these in action, navigate no further than OpenSSH, which interfaces with these sandboxes: sandbox-seccomp-filter.c or sandbox-capsicum.c. (For a history lesson, you can even see sandbox-systrace.c.) Keep in mind that these do little more than restrict resources to open descriptors and the usual necessities of memory, signals, timing, etc. Keep that in mind and be horrified. “ Now Kristaps has his theory on why these are so difficult (NS..), but perhaps there is a better way. He makes the case that pledge() sits right in that sweet-spot, being powerful enough to be useful, but easy enough to implement that developers might actually use it. All in all, a nice read, check it out! Would love to hear other developer success stories using pledge() as well. *** News Roundup Unix history repository, now on GitHub (http://www.osnews.com/story/29513/Unix_history_repository_now_on_GitHub) OS News has an interesting tidbit on their site today, about the entire commit history of Unix now being available online, starting all the way back in 1970 and bringing us forward to today. From the README The history and evolution of the Unix operating system is made available as a revision management repository, covering the period from its inception in 1970 as a 2.5 thousand line kernel and 26 commands, to 2016 as a widely-used 27 million line system. The 1.1GB repository contains about half a million commits and more than two thousand merges. The repository employs Git system for its storage and is hosted on GitHub. It has been created by synthesizing with custom software 24 snapshots of systems developed at Bell Labs, the University of California at Berkeley, and the 386BSD team, two legacy repositories, and the modern repository of the open source FreeBSD system. In total, about one thousand individual contributors are identified, the early ones through primary research. The data set can be used for empirical research in software engineering, information systems, and software archaeology. This is a fascinating find, especially will be of value to students and historians who wish to look back in time to see how UNIX evolved, and in this repo ultimately turned into modern FreeBSD. *** Yandex commits improvements to FreeBSD network stack (https://reviews.freebsd.org/D8526) “Rework ip_tryforward() to use FIB4 KPI.” This commit brings some code from the experimental routing branch into head As you can see from the graphs, it offers some sizable improvements in forwarding and firewalled packets per second commit (https://svnweb.freebsd.org/base?view=revision&revision=309257) *** The brief history of Unix socket multiplexing – select(2) system call (https://idea.popcount.org/2016-11-01-a-brief-history-of-select2/) Ever wondered about the details of socket multiplexing, aka the history of select(2)? Well Marek today gives a treat, with a quick look back at the history that made today's modern multiplexing possible. First, his article starts the way all good ones do, presenting the problem in silent-movie form: In mid-1960's time sharing was still a recent invention. Compared to a previous paradigm - batch-processing - time sharing was truly revolutionary. It greatly reduced the time wasted between writing a program and getting its result. Batch-processing meant hours and hours of waiting often to only see a program error. See this film to better understand the problems of 1960's programmers: "The trials and tribulations of batch processing". Enter the wild world of the 1970's, and we've now reached the birth of UNIX which tried to solve the batch processing problem with time-sharing. These days when a program was executed, it could "stall" (block) only on a couple of things1: + wait for CPU + wait for disk I/O + wait for user input (waiting for a shell command) or console (printing data too fast)“ Jump forward another dozen years or so, and the world changes yet again: This all changed in 1983 with the release of 4.2BSD. This revision introduced an early implementation of a TCP/IP stack and most importantly - the BSD Sockets API.Although today we take the BSD sockets API for granted, it wasn't obvious it was the right API. STREAMS were a competing API design on System V Revision 3. Coming in along with the sockets API was the select(2) call, which our very own Kirk McKusick gives us some background on: Select was introduced to allow applications to multiplex their I/O. Consider a simple application like a remote login. It has descriptors for reading from and writing to the terminal and a descriptor for the (bidirectional) socket. It needs to read from the terminal keyboard and write those characters to the socket. It also needs to read from the socket and write to the terminal. Reading from a descriptor that has nothing queued causes the application to block until data arrives. The application does not know whether to read from the terminal or the socket and if it guesses wrong will incorrectly block. So select was added to let it find out which descriptor had data ready to read. If neither, select blocks until data arrives on one descriptor and then awakens telling which descriptor has data to read. [...] Non-blocking was added at the same time as select. But using non-blocking when reading descriptors does not work well. Do you go into an infinite loop trying to read each of your input descriptors? If not, do you pause after each pass and if so for how long to remain responsive to input? Select is just far more efficient. Select also lets you create a single inetd daemon rather than having to have a separate daemon for every service. The article then wraps up with an interesting conclusion: > CSP = Communicating sequential processes In this discussion I was afraid to phrase the core question. Were Unix processes intended to be CSP-style processes? Are file descriptors a CSP-derived "channels"? Is select equivalent to ALT statement? I think: no. Even if there are design similarities, they are accidental. The file-descriptor abstractions were developed well before the original CSP paper. It seems that an operating socket API's evolved totally disconnected from the userspace CSP-alike programming paradigms. It's a pity though. It would be interesting to see an operating system coherent with the programming paradigms of the user land programs. A long (but good) read, and worth your time if you are interested in the history how modern multiplexing came to be. *** How to start CLion on FreeBSD? (https://intellij-support.jetbrains.com/hc/en-us/articles/206525024-How-to-start-CLion-on-FreeBSD) CLion (pronounced "sea lion") is a cross-platform C and C++ IDE By default, the Linux version comes bundled with some binaries, which obviously won't work with the native FreeBSD build Rather than using Linux emulation, you can replace these components with native versions pkg install openjdk8 cmake gdb Edit clion-2016.3/bin/idea.properties and change run.processes.with.pty=false Start CLion and open Settings | Build, Execution, Deployment | Toolchains Specify CMake path: /usr/local/bin/cmake and GDB path: /usr/local/bin/gdb Without a replacement for fsnotifier, you will get a warning that the IDE may be slow to detect changes to files on disk But, someone has already written a version of fsnotifier that works on FreeBSD and OpenBSD fsnotifier for OpenBSD and FreeBSD (https://github.com/idea4bsd/fsnotifier) -- The fsnotifier is used by IntelliJ for detecting file changes. This version supports FreeBSD and OpenBSD via libinotify and is a replacement for the bundled Linux-only version coming with the IntelliJ IDEA Community Edition. *** Beastie Bits TrueOS Pico – FreeBSD ARM/RPi Thin Clients (https://www.trueos.org/trueos-pico/) A Puppet package provider for FreeBSD's PkgNG package manager. (https://github.com/xaque208/puppet-pkgng) Notes from November London *BSD meetup (http://mailman.uk.freebsd.org/pipermail/ukfreebsd/2016-November/014059.html) SemiBug meeting on Dec 20th (http://lists.nycbug.org/pipermail/semibug/2016-November/000131.html) Feedback/Questions Erno - SSH without password (http://pastebin.com/SMvxur9v) Jonathan - Magical ZFS (http://pastebin.com/5ETL7nmj) George - TrueOS (http://pastebin.com/tSVvaV9e) Mohammad - Jails IP (http://pastebin.com/T8nUexd1) Gibheer - BEs (http://pastebin.com/YssXXp70) ***
This week on the show, we'll be chatting with Marc Espie. He's recently added some additional security measures to dpb, OpenBSD's package building tool, and we'll find out why they're so important. We've also got all this week's news, answers to your emails and even a BSDCan wrap-up, coming up on BSD Now - the place to B.. SD. This episode was brought to you by Headlines BSDCan 2015 videos (https://www.bsdcan.org/2015/schedule/) BSDCan just ended last week, but some of the BSD-related presentation videos are already online Allan Jude, UCL for FreeBSD (https://www.youtube.com/watch?v=8l6bhKIDecg) Andrew Cagney, What happens when a dwarf and a daemon start dancing by the light of the silvery moon? (https://www.youtube.com/watch?v=XDIcD4LR5HE) Andy Tanenbaum, A reimplementation of NetBSD (https://www.youtube.com/watch?v=0pebP891V0c) using a MicroKernel (https://www.youtube.com/watch?v=Bu1JuwVfYTc) Brooks Davis, CheriBSD: A research fork of FreeBSD (https://www.youtube.com/watch?v=DwCg-51vFAs) Giuseppe Lettieri, Even faster VM networking with virtual passthrough (https://www.youtube.com/watch?v=Lo6wDCapo4k) Joseph Mingrone, Molecular Evolution, Genomic Analysis and FreeBSD (https://www.youtube.com/watch?v=K2pnf1YcMTY) Olivier Cochard-Labbe, Large-scale plug&play x86 network appliance deployment over Internet (https://www.youtube.com/watch?v=6jhSvdnu4k0) Peter Hessler, Using routing domains / routing tables in a production network (https://www.youtube.com/watch?v=BizrC8Zr-YY) Ryan Lortie, a stitch in time: jhbuild (https://www.youtube.com/watch?v=YSVFnM3_2Ik) Ted Unangst, signify: Securing OpenBSD From Us To You (https://www.youtube.com/watch?v=9R5s3l-0wh0) Many more still to come... *** Documenting my BSD experience (http://pid1.com/posts/post1.html) Increasingly common scenario: a long-time Linux user (since the mid-90s) decides it's finally time to give BSD a try "That night I came home, I had been trying to find out everything I could about BSD and I watched many videos, read forums, etc. One of the shows I found was BSD Now. I saw that they helped people and answered questions, so I decided to write in." In this ongoing series of blog posts, a user named Michael writes about his initial experiences with trying different BSDs for some different tasks The first post covers ZFS on FreeBSD, used to build a file server for his house (and of course he lists the hardware, if you're into that) You get a glimpse of a brand new user trying things out, learning how great ZFS-based RAID arrays are and even some of the initial hurdles someone could run into He's also looking to venture into the realm of replacing some of his VMs with jails and bhyve soon His second post (http://pid1.com/posts/post2.html) explores replacing the firewall on his self-described "over complicated home network" with an OpenBSD box After going from ipfwadmin to ipchains to iptables, not even making it to nftables, he found the simple PF syntax to be really refreshing All the tools for his networking needs, the majority of which are in the base system, worked quickly and were easy to understand Getting to hear experiences like this are very important - they show areas where all the BSD developers' hard work has paid off, but can also let us know where we need to improve *** PC-BSD tries HardenedBSD builds (https://github.com/pcbsd/hardenedBSD-stable) The PC-BSD team has created a new branch of their git repo with the HardenedBSD ASLR patches integrated They're not the first major FreeBSD-based project to offer an alternate build - OPNsense did that (https://hardenedbsd.org/article/shawn-webb/2015-05-08/hardenedbsd-teams-opnsense) a few weeks ago - but this might open the door for more projects to give it a try as well With Personacrypt, OpenNTPD, LibreSSL and recent Tor integration through the tools, these additional memory protections will offer PC-BSD users even more security that a default FreeBSD install won't have Time will tell if more projects and products like FreeNAS might be interested too *** C-states in OpenBSD (https://www.marc.info/?l=openbsd-cvs&m=143423172522625&w=2) People who run BSD on their notebooks, you'll want to pay attention to this one OpenBSD has recently committed some ACPI improvements for deep C-states (http://www.hardwaresecrets.com/article/Everything-You-Need-to-Know-About-the-CPU-C-States-Power-Saving-Modes/611), enabling the processor to enter a low-power mode According (https://twitter.com/StevenUniq/status/610586711358316545) to a (https://www.marc.info/?l=openbsd-misc&m=143430996602802&w=2) few users (https://www.marc.info/?l=openbsd-misc&m=143429914700826&w=2) so far (https://www.marc.info/?l=openbsd-misc&m=143425943026225&w=2), the change has resulted in dramatically lower CPU temperatures on their laptops, as well as much better battery life If you're running OpenBSD -current on a laptop, try out the latest snapshot and report back (https://www.marc.info/?l=openbsd-misc&m=143423391222952&w=2) with your findings *** NetBSD at Open Source Conference 2015 Hokkaido (https://mail-index.netbsd.org/netbsd-advocacy/2015/06/13/msg000687.html) The Japanese NetBSD users group never sleeps, and they've hit yet another open source conference As is usually the case, lots of strange machines on display were running none other than NetBSD (though it was mostly ARM this time) We'll be having one of these guys on the show next week to discuss some of the lesser-known NetBSD platforms *** Interview - Marc Espie - espie@openbsd.org (mailto:espie@openbsd.org) / @espie_openbsd (https://twitter.com/espie_openbsd) Recent (https://www.marc.info/?l=openbsd-ports&m=143051151521627&w=2) improvements (https://www.marc.info/?l=openbsd-ports&m=143151777209226&w=2) to OpenBSD's dpb (http://www.bsdnow.tv/tutorials/dpb) tool News Roundup Introducing xhyve, bhyve on OS X (https://github.com/mist64/xhyve/blob/master/README.md) We've talked about FreeBSD's "bhyve" hypervisor a lot on the show, and now it's been ported to another OS As the name "xhyve" might imply, it's a port of bhyve to Mac OS X Currently it only has support for virtualizing a few Linux distributions, but more guest systems can be added in the future It runs entirely in userspace, and has no extra requirements beyond OS X 10.10 or newer There are also a few examples (http://www.pagetable.com/?p=831) on how to use it *** 4K displays on DragonFlyBSD (http://www.dragonflybsd.org/docs/newhandbook/docs/newhandbook/4KDisplays/) If you've been using DragonFly as a desktop, maybe with those nice Broadwell graphics, you'll be pleased to know that 4K displays work just fine Matthew Dillon wrote up a wiki page about some of the specifics, including a couple gotchas Some GUI applications might look weird on such a huge resolution, HDMI ports are mostly limited to a 30Hz refresh rate, and there are slightly steeper hardware requirements for a smooth experience *** Sandboxing port daemons on OpenBSD (http://coderinaworldofcode.blogspot.com/2015/06/chrooting-mumble-server-on-openbsd.html) We talked about different containment methods last week, and mentioned that a lot of the daemons in OpenBSD's base as chrooted by default - things from ports or packages don't always get the same treatment This blog post uses a mumble server as an example, but you can apply it to any service from ports that doesn't chroot by default It goes through the process of manually building a sandbox with all the libraries you'll need to run the daemon, and this setup will even wipe and refresh the chroot every time you restart it With a few small changes, similar tricks could be done on the other BSDs as well - everybody has chroots *** SmallWall 1.8.2 released (http://smallwall.freeforums.net/thread/44/version-1-8-2-released) SmallWall is a relatively new BSD-based project that we've never covered before It's an attempt to keep the old m0n0wall codebase going, and appears to have started around the time m0n0wall called it quits They've just released the first official version (http://www.smallwall.org/download.html), so you can give it a try now If you're interested in learning more about SmallWall, the lead developer just might be on the show in a few weeks... *** Feedback/Questions David writes in (http://slexy.org/view/s21gRTNnk7) Brian writes in (http://slexy.org/view/s2DdiMvELg) Dan writes in (http://slexy.org/view/s2h4ZS6SMd) Joel writes in (http://slexy.org/view/s20kA1jeXY) Steve writes in (http://slexy.org/view/s2wJ9HP1bs) ***
This time on the show, we'll be chatting with Jed Reynolds about ZFS. He's been using it extensively on a certain other OS, and we can both learn a bit about the other side's implementation. Answers to your questions and all this week's news, coming up on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Playing with sandboxing (http://blog.conviso.com.br/2015/05/playing-with-sandbox-analysis-of_13.html) Sandboxing and privilege separation are popular topics these days - they're the goal of the new "shill" scripting language, they're used heavily throughout OpenBSD, and they're gaining traction with the capsicum framework This blog post explores capsicum in FreeBSD, some of its history and where it's used in the base system They also include some code samples so you can verify that capsicum is actually denying the program access to certain system calls Check our interview about capsicum (http://www.bsdnow.tv/episodes/2014_05_28-the_friendly_sandbox) from a while back if you haven't seen it already *** OpenNTPD on by default (https://www.marc.info/?l=openbsd-cvs&m=143195693612629&w=4) OpenBSD has enabled ntpd (http://www.bsdnow.tv/episodes/2015_02_11-time_for_a_change) by default in the installer, rather than prompting the user if they want to turn it on In nearly every case, you're going to want to have your clock synced via NTP With the HTTPS constraints feature also enabled by default, this should keep the time checked and accurate, even against spoofing attacks Lots of problems can be traced back to the time on one system or another being wrong, so this will also eliminate some of those cases For those who might be curious (http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/etc/ntpd.conf), they're using the "pool.ntp.org (http://www.pool.ntp.org/en/)" cluster of addresses and google for HTTPS constraints (but these can be easily changed (http://www.bsdnow.tv/tutorials/ntpd)) *** FreeBSD workshop in Landshut (https://www.banym.de/freebsd/review-first-freebsd-workshop-in-landshut-on-15-may-2015) We mentioned a BSD installfest happening in Germany a few weeks back, and the organizer wrote in with a review of the event The installfest instead became a "FreeBSD workshop" session, introducing curious new users to some of the flagship features of the OS They covered when to use UFS or ZFS, firewall options, the release/stable/current branches and finally how to automate installations with Ansible If you're in south Germany and want to give similar introduction talks or Q&A sessions about the other BSDs, get in touch We'll hear more from him about how it went in the feedback section today *** Swap encryption in DragonFly (http://lists.dragonflybsd.org/pipermail/users/2015-May/207690.html) Doing full disk encryption (http://www.bsdnow.tv/tutorials/fde) is very important, but something that people sometimes overlook is encrypting their swap This can actually be more important than the contents of your disks, especially if an unencrypted password or key hits your swap (as it can be recovered quite easily) DragonFlyBSD has added a new experimental option to automatically encrypt your swap partition in fstab There was another way (http://lists.dragonflybsd.org/pipermail/users/2015-May/207691.html) to do it previously, but this is a lot easier You can achieve similar results in FreeBSD by adding ".eli" to the end of the swap device in fstab, there are a few steps (https://www.netbsd.org/docs/misc/#cgd-swap) to do it in NetBSD and swap in OpenBSD is encrypted by default A one-time key will be created and then destroyed in each case, making recovery of the plaintext nearly impossible *** Interview - Jed Reynolds - jed@bitratchet.com (mailto:jed@bitratchet.com) / @jed_reynolds (https://twitter.com/jed_reynolds) Comparing ZFS on Linux and FreeBSD News Roundup USB thermometer on OpenBSD (http://www.cambus.net/rding-temper-gold-usb-thermometer-on-openbsd/) So maybe you've got BSD on your server or router, maybe NetBSD on a toaster, but have you ever used a thermometer with one? This blog post introduces the RDing TEMPer Gold USB thermometer, a small device that can tell the room temperature, and how to get it working on OpenBSD Wouldn't you know it, OpenBSD has a native "ugold (http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/ugold.4)" driver to support it with the sensors framework How useful such a device would be is another story though *** NAS4Free now on ARM (http://sourceforge.net/projects/nas4free/files/NAS4Free-ARM/10.1.0.2.1511/) We talk a lot about hardware for network-attached storage devices on the show, but ARM doesn't come up a lot That might be changing soon, as NAS4Free has just released some ARM builds These new (somewhat experimental) images are based on FreeBSD 11-CURRENT Included in the announcement is a list of fully-supported and partially-supported hardware that they've tested it with If anyone has experience with running a NAS on slightly exotic hardware, write in to us *** pkgsrcCon 2015 CFP and info (http://pkgsrc.pub/pkgsrcCon/2015/) This year's pkgsrcCon will be in Berlin, Germany on July 4th and 5th (https://mail-index.netbsd.org/pkgsrc-users/2015/05/16/msg021560.html) They're looking for talk proposals and ideas for things you'd like to see If you or your company uses pkgsrc, or if you're just interested in NetBSD in general, it would be a good event to check out *** BSDTalk episode 253 (http://bsdtalk.blogspot.com/2015/05/bsdtalk253-george-neville-neil.html) BSDTalk has released another new episode In it, he interviews George Neville-Neil about the 2nd edition of "The Design and Implementation of the FreeBSD Operating System" They discuss what's new since the last edition, who the book's target audience is and a lot more We're up to 90 episodes now, slowly catching up to Will... *** Feedback/Questions Dominik writes in (http://slexy.org/view/s2SWlyuOeb) Brad writes in (http://slexy.org/view/s216z44lDU) Corvin writes in (http://slexy.org/view/s2djtX0dSE) James writes in (http://slexy.org/view/s21XM4hPRh) ***
© 2020-2025 Ivy Podcast Discovery LLC
