POPULARITY
43 episodes with openzfs
9 episodes with openzfs
8 episodes with openzfs
6 episodes with openzfs
5 episodes with openzfs
5 episodes with openzfs
5 episodes with openzfs
6 episodes with openzfs
2 episodes with openzfs
Key Considerations for Benchmarking Network Storage Performance, OpenZFS 2.3.0 available, Updates on AsiaBSDcon, GhostBSD Desktop Conference, Recovering from external zroot, Create a new issue in a Github repository with Ansible, Stories I refuse to believe, date limit in UFS1 filesystem extended, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Key Considerations for Benchmarking Network Storage Performance (https://klarasystems.com/articles/considerations-benchmarking-network-storage-performance/) OpenZFS 2.3.0 available (https://github.com/openzfs/zfs/releases/tag/zfs-2.3.0) News Roundup Updates on AsiaBSDCon 2025 - Cancelled - (https://lists.asiabsdcon.org/pipermail/announce/2025-January/000046.html) GhostBSD Desktop Conference (https://www.phoronix.com/news/BSD-Desktop-Conference-GhostBSD) Recovering from external zroot (https://adventurist.me/posts/00350) Create a new issue in a Github repository with Ansible (https://jpmens.net/2025/01/25/create-a-new-issue-in-a-github-repository/) Stories I refuse to believe (https://flak.tedunangst.com/post/stories-i-refuse-to-believe) Defer the January 19, 2038 date limit in UFS1 filesystems to February 7, 2106 (https://cgit.freebsd.org/src/commit/?id=1111a44301da39d7b7459c784230e1405e8980f8) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Feedback - Nelson - Ada/GCC (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/598/feedback/Nelson%20Feedback.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
Bambu Labs teaches us how to lose friends and alienate people. Then, Alex Tran from Immich joins us for a project update, and we shared some dreams for a community RSS project. Special Guest: Alex Tran.
This week we dig into your questions, and talk about Nvidia's new AI rig. -- During The Show -- 00:56 George From NYC Used computers for a church Lenovo laptops and Thinkcenters Dell Optiplex Ebay Made in the last 4 years Make sure they have TPM 2.0 04:50 Noah's new toy Flipper Zero (https://flipperzero.one/) Electronic multi-tool Had to legitimately bypass access control Read and emulate RFID and NFC Lots of Apps 08:58 HVAC - Ziggy Zigbee timer system? Wouldn't put the timer on device Steve's solution Why timers? 14:20 Battery Pack - Erik Anderson Power Pole Deep Cycle SLA INIU 100w Type C Dewalt Battery Adapter (https://www.amazon.com/Converter-Battery-Adapter-Regulator-Terminal/dp/B0CQJDGQDB) Offical Dewalt USB C Adapter (https://www.dewalt.com/product/dcb094k/20v-maxflexvolt-5-amp-usb-charging-kit?tid=577811) 83w 12v USB Outlet (https://www.amazon.com/dp/B0B1DHNLDS?ref=fed_asin_title) 21:40 NFS vs S3 for Home Lab - Brendan Recoverability Added complexity Would lean away from S3/Minio Hard to recover broken S3 file system It's ok to play with technology 28:21 Nextcloud Office - IK All-In-One master container Steve's attempt Not all Docker containers are official 33:52 News Wire Dillo 3.2 - github.io (https://dillo-browser.github.io/release/3.2.0/) OpenZFS 2.3 - github.com (https://github.com/openzfs/zfs/releases) Rsync 3.4 - samba.org (https://download.samba.org/pub/rsync/NEWS#3.4.0) Linux Mint 22.1 - linuxmint.com (https://www.linuxmint.com/rel_xia.php) Rhino Linux 2025.1 - rhinolinux.org (https://blog.rhinolinux.org/news-17) TuxCare Now Offering EOL MS Support - fossforce.com (https://fossforce.com/2025/01/tuxcare-stops-microsoft-from-killing-net-6-0/) Merit Systems Trying to Fund Open Source Devs - cnbc.com (https://www.cnbc.com/2025/01/16/merit-systems-raises-10-million-from-a16z-blockchain-capital.html) MiniMax Models - scmp.com (https://www.scmp.com/tech/big-tech/article/3294900/chinese-ai-start-minimax-releases-low-cost-open-source-models-rival-top-chatbots) 35:00 Ebook 2 Audio Book Ebook2Audiobook (https://github.com/DrewThomasson/ebook2audiobook/blob/main/README.md) Uses "local AI" Many types of text input Runs on CPU or GPU Surprised at the quality Steve's kid's apprenticeship Society has developed an aversion to anything "hard" Teaching the value of work and learning 42:00 Getting Started with AI What is AI? Math coprocessors CPUs are "generalist" processors GPUs are "specialized" processors CUDA Cores NVIDIA GB10 Grace Blackwell Superchip (https://www.nvidia.com/en-eu/project-digits/) 3 Classes of GPUs Design software story Project Digits (https://newsroom.arm.com/blog/arm-nvidia-project-digits-high-performance-ai) LMStudio.at (https://lmstudio.ai/) GPT4All (https://docs.gpt4all.io/) TecMint.com (https://www.tecmint.com/ai-for-linux-users/) -- The Extra Credit Section -- For links to the articles and material referenced in this week's episode check out this week's page from our podcast dashboard! This Episode's Podcast Dashboard (http://podcast.asknoahshow.com/424) Phone Systems for Ask Noah provided by Voxtelesys (http://www.voxtelesys.com/asknoah) Join us in our dedicated chatroom #GeekLab:linuxdelta.com on Matrix (https://element.linuxdelta.com/#/room/#geeklab:linuxdelta.com) -- Stay In Touch -- Find all the resources for this show on the Ask Noah Dashboard Ask Noah Dashboard (http://www.asknoahshow.com) Need more help than a radio show can offer? Altispeed provides commercial IT services and they're excited to offer you a great deal for listening to the Ask Noah Show. Call today and ask about the discount for listeners of the Ask Noah Show! Altispeed Technologies (http://www.altispeed.com/) Contact Noah live [at] asknoahshow.com -- Twitter -- Noah - Kernellinux (https://twitter.com/kernellinux) Ask Noah Show (https://twitter.com/asknoahshow) Altispeed Technologies (https://twitter.com/altispeed)
This week we get an update from the Ovens Data Center. Steve migrates a box from Ubuntu to Red Hat, and of course we answer your questions! -- During The Show -- 00:50 Intro Deadly cold weather Oven's data center update Spending "cloud money" on local media 05:55 Custom RHEL ISO ISO build system Red Hatters building all kinds of custom ISOs Why a custom ISO Red Hat Doc (https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html-single/composing_a_customized_rhel_system_image/index) Why switch to RHEL Professional Linux support Mixed environment? 20:20 CES POE powered TV ArsTechnica Atricle (https://arstechnica.com/gadgets/2025/01/the-8-most-interesting-pc-monitors-from-ces-2025/) Basecase (https://getbasecase.com/) Crowd funding Questions about interfaces Powered over barrel connector Very expensive 29:58 News Wire Firefox 134 - mozilla.org (https://www.mozilla.org/en-US/firefox/134.0/releasenotes/) Flatpak 1.16 - github.com (https://github.com/flatpak/flatpak/releases/tag/1.16.0) KDE Frameworks 6.10 - kde.org (https://kde.org/announcements/frameworks/6/6.10.0/) Tails 6.11 - blog.torproject.org (https://blog.torproject.org/new-release-tails-611/) Debian 12.9 - debian.org (https://www.debian.org/News/2025/20250111) 16GB Raspberry PI 5 - raspberrypi.com (https://www.raspberrypi.com/news/16gb-raspberry-pi-5-on-sale-now-at-120/) Nvidia Desktop AI "Super Computer" - theverge.com (https://www.theverge.com/2025/1/8/24339195/nvidia-digits-ai-supercomputer-in-person-photos-small) Phi-4 Fully Open-Source - venturebeat.com (https://venturebeat.com/ai/microsoft-makes-powerful-phi-4-model-fully-open-source-on-hugging-face/) Sky-T1-32B-Preview - techcrunch.com (https://techcrunch.com/2025/01/11/researchers-open-source-sky-t1-a-reasoning-ai-model-that-can-be-trained-for-less-than-450/) zdnet.com (https://www.zdnet.com/article/450-and-19-hours-is-all-it-takes-to-rival-openais-o1-preview/) 30:55 Social Media/Decentralized Networks Mastodon shifting to being owned by a non-profit Creator wants users in control of Mastodon People are choosing to embrace the open source decentralized option Network effect is starting to weaken in favor of interoperability ArsTechnica (https://arstechnica.com/tech-policy/2025/01/mastodon-becomes-nonprofit-to-make-sure-its-never-ruined-by-billionaire-ceo/) 36:15 OpenZFS 2.3 Support for Linux 4.18, up to Linux 6.12 LTS Adding new devices to existing RaidZ pool 9 to 5 Linux (https://9to5linux.com/openzfs-2-3-is-out-with-linux-6-12-support-raidz-expansion-fast-dedup-and-more) 38:30 Red Hat In-Vehical OS Standardized domains ISO 26262 Automotive Safety Integrity Level B (ASIL-B) thefastmode.com (https://www.thefastmode.com/technology-solutions/38950-red-hat-achieves-functional-safety-certification-for-in-vehicle-operating-system) 41:10 ZFS/QCOW2 - Carey Double writes ZFS doesn't support trim Options Snapshots Copying a QCOW2 file can be tricky 47:54 Daughters Internet Access - Vlad Shut off Internet access entirely Take the device Internet is different than technology 49:20 Politics in Software - Max Does the project throw it in your face? Be the bigger person Don't tolerate treating people badly because they don't agree with you -- The Extra Credit Section -- For links to the articles and material referenced in this week's episode check out this week's page from our podcast dashboard! This Episode's Podcast Dashboard (http://podcast.asknoahshow.com/423) Phone Systems for Ask Noah provided by Voxtelesys (http://www.voxtelesys.com/asknoah) Join us in our dedicated chatroom #GeekLab:linuxdelta.com on Matrix (https://element.linuxdelta.com/#/room/#geeklab:linuxdelta.com) -- Stay In Touch -- Find all the resources for this show on the Ask Noah Dashboard Ask Noah Dashboard (http://www.asknoahshow.com) Need more help than a radio show can offer? Altispeed provides commercial IT services and they're excited to offer you a great deal for listening to the Ask Noah Show. Call today and ask about the discount for listeners of the Ask Noah Show! Altispeed Technologies (http://www.altispeed.com/) Contact Noah live [at] asknoahshow.com -- Twitter -- Noah - Kernellinux (https://twitter.com/kernellinux) Ask Noah Show (https://twitter.com/asknoahshow) Altispeed Technologies (https://twitter.com/altispeed)
FreeBSD Quarterly Report, Welcome to the new category: filesystems, BSD Misconceptions, Notes on the compatibility of crypted passwords across Unixes in late 2024, Automating ZFS Snapshots for Peace of Mind, A few nice things in OpenZFS 2.3, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Quarterly Report (https://www.freebsd.org/status/report-2024-07-2024-09/) News Roundup Welcome to the new category: filesystems (https://news.freshports.org/2024/11/06/welcome-to-the-new-category-filesystems/) BSD Misconceptions (https://izder456.tumblr.com/post/759376596551483392/bsd-misconceptions) Notes on the compatibility of crypted passwords across Unixes in late 2024 (https://utcc.utoronto.ca/~cks/space/blog/unix/CryptedPasswordCompatibility2024) Automating ZFS Snapshots for Peace of Mind (https://it-notes.dragas.net/2024/08/21/automating-zfs-snapshots-for-peace-of-mind/) A few nice things in OpenZFS 2.3 (https://despairlabs.com/blog/posts/2024-10-05-nice-things-in-openzfs-23/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Izzy - Misconceptions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/587/feedback/izzy%20-%20misconceptions.md) John - UNIX Graphical Desktops (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/587/feedback/John-UNIXGraphicalDesktops.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
We start with Quantum Computing and RSA, Chat about Nvidia and the AI craze, and end with the the Wordpress drama. In between we cover updates to Ardour, Calibre, KDE Plasma, and Clonezilla. And finally there's some kernel news, like proxy execution for better performance, and OpenZFS coverage. For tips we have pathchk for filesystem portability checks, how-to for freezing a package version in Ubuntu, and an intro to the network tool netcat. The show notes are at https://bit.ly/40awxsQ and enjoy! Host: Jonathan Bennett Co-Hosts: Ken McDonald and Jeff Massie Want access to the video version and exclusive features? Become a member of Club TWiT today! https://twit.tv/clubtwit Club TWiT members can discuss this episode and leave feedback in the Club TWiT Discord.
What if you could control your lights, music, and other Home Assistant things from a panel on your wall? Do you run tech at your church and need the ability to deliver controls in a way, in a location, and with a budget that's approachable for everyone? This week, Thomas Kvamme from Orbit Panels joins Noah and Steve to discuss his new in wall PoE control panels based on the Raspberry Pi, Streamdeck, and Bitfocus Companion! -- During The Show -- 00:50 Intro Nice weather Little projects 01:45 Orbit Panels Thomas Kvamme - Owner and founder Why are you in ministry? Life before the stream deck StreamDeck (https://www.elgato.com/us/en/s/welcome-to-stream-deck) Bitfocus Companion (https://bitfocus.io/companion) List of Bitfocus Companion Integrations (https://bitfocus.io/connections) Bitfocus Companion Satellite (https://bitfocus.io/companion-satellite) Proprietary alternatives Origin of Orbit Panels Orbit Panels (https://orbitpanels.com/) Drop in replacement Design of the Orbit Panel Building an Orbit Panel How people use the Orbit Panels Home Assistant Integration Feedback on the panel Whats next? Available for sale! Use promo code LAUNCH for 10% off 45:00 News Wire Firefox 129 - mozilla.org (https://www.mozilla.org/en-US/firefox/129.0/releasenotes/) Wine 9.15 - gitlab.winehq.org (https://gitlab.winehq.org/wine/wine/-/releases/wine-9.15) KDE Frameworks 6.5.0 - kde.org (https://kde.org/announcements/frameworks/6/6.5.0/) Gnome 46.4 - discourse.gnome.org (https://discourse.gnome.org/t/gnome-46-4-released/22718) PopOS! 24.04 - omgubuntu.co.uk (https://www.omgubuntu.co.uk/2024/07/pop_os-24-04-alpha-with-cosmic-drops-on-august-8) OpenZFS 2.2.5 - github.com (https://github.com/openzfs/zfs/releases) ZLUDA Take Down - wccftech.com (https://wccftech.com/zluda-open-source-library-nvidia-cuda-on-amd-gpus-taken-down-amid-legal-concerns/) DHS & White House $11M Partnership - therecord.media (https://therecord.media/open-source-software-security-white-house-dhs-11million-funding) TUXEDO Infinity Flex 14 - tuxedocomputers.com (https://www.tuxedocomputers.com/en/TUXEDO-InfinityFlex-14-Gen1) CA AI Bill - reason.com (https://reason.com/2024/08/13/californias-ai-bill-threatens-to-derail-open-source-innovation/) Kernel Patch Boots Linux Faster - tomshardware.com (https://www.tomshardware.com/software/linux/linux-boot-time-reduced-by-0035-seconds-thanks-to-a-one-line-kernel-patch) 46:32 SimpleHelp - Brandon Tailscale/Headscale WireGuard RDP + VPN Simple Help instant support Proprietary is a place holder Mesh Central (https://github.com/Ylianst/MeshCentral) 49:09 Linphone SMS & VPN Question - Kris Google requirement is to support RCS Steve holding on 2.22.31 JMP.chat (https://jmp.chat/) Matrix and SMS Route back to home OpenVPN TailScale WireGuard SurfShark Private Internet Access (https://www.privateinternetaccess.com/) Host your own VPN on a VPS -- The Extra Credit Section -- For links to the articles and material referenced in this week's episode check out this week's page from our podcast dashboard! This Episode's Podcast Dashboard (http://podcast.asknoahshow.com/403) Phone Systems for Ask Noah provided by Voxtelesys (http://www.voxtelesys.com/asknoah) Join us in our dedicated chatroom #GeekLab:linuxdelta.com on Matrix (https://element.linuxdelta.com/#/room/#geeklab:linuxdelta.com) -- Stay In Touch -- Find all the resources for this show on the Ask Noah Dashboard Ask Noah Dashboard (http://www.asknoahshow.com) Need more help than a radio show can offer? Altispeed provides commercial IT services and they're excited to offer you a great deal for listening to the Ask Noah Show. Call today and ask about the discount for listeners of the Ask Noah Show! Altispeed Technologies (http://www.altispeed.com/) Contact Noah live [at] asknoahshow.com -- Twitter -- Noah - Kernellinux (https://twitter.com/kernellinux) Ask Noah Show (https://twitter.com/asknoahshow) Altispeed Technologies (https://twitter.com/altispeed) Special Guest: Thomas Kvamme.
A backdoor has been found in xz-utils, OpenZFS improves ZVOL performance on Linux, Twitter devs fail at regex, and adding SATA ports to a home NAS. Plug Support us on patreon and get an ad-free RSS feed with early episodes sometimes Hybrid Cloud Show is a new show that's part of the Late Night […]
A backdoor has been found in xz-utils, OpenZFS improves ZVOL performance on Linux, Twitter devs fail at regex, and adding SATA ports to a home NAS. Plug Support us on patreon and get an ad-free RSS feed with early episodes sometimes Hybrid Cloud Show is a new show that's part of the Late Night... Read More
Using Git offline, Make your own E-mail server, quiz: a tool for rapid OpenZFS development, Configuring openzfs for nvme databases, Mirroring OmniOS: The Complete Guide part 1, Installing OpenBSD 7.4 on a VisionFive 2 rev, and more... NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Using Git offline (https://www.gibbard.me/using_git_offline/) Make your own E-Mail server - FreeBSD, OpenSMTPD, Rspamd and Dovecot included - Part 1 (https://it-notes.dragas.net/2024/03/08/make-your-own-email-server-freebsd-opensmptd-rspamd-dovecot-part1/) News Roundup quiz: a tool for rapid OpenZFS development (https://despairlabs.com/blog/posts/2024-03-04-quiz-rapid-openzfs-development/) Configuring openzfs for nvme databases (https://github.com/letsencrypt/openzfs-nvme-databases) Mirroring OmniOS: The Complete Guide; Part One (https://antranigv.am/posts/2024/02/omnios-mirror-one/) Installing OpenBSD 7.4 on a VisionFive 2 rev 1.2a (https://quozul.dev/riscv/2023/12/22/installing-openbsd-on-visionfive-2.html) Terminal Latency (https://beuke.org/terminal-latency/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
This week on the show, you're not too late to develop the future, netmap on czgbe, OpenZFS 2.2.3, SSH Brute Forcing, some unknown OpenBSD Features, Release notes for the latest Omni OS, and more... NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines When the Power Macintosh ran NetWare (featuring Wormhole and Cyberpunk) (https://oldvcr.blogspot.com/2023/12/when-power-macintosh-ran-netware.html) You are not too late (https://kk.org/thetechnium/you-are-not-late/) News Roundup netmap on cxgbe interfaces (https://adventurist.me/posts/00318) OpenZFS 2.2.3 (https://github.com/openzfs/zfs/releases/tag/zfs-2.2.3) A recent abrupt change in Internet SSH brute force attacks against us (https://utcc.utoronto.ca/~cks/space/blog/sysadmin/SSHBruteForceAttacksAbruptlyDown) Some OpenBSD features that aren't widely known (https://dataswamp.org/~solene/2024-02-20-rarely-known-openbsd-features.html) Release Notes for OmniOS v11 r151048 (https://github.com/omniosorg/omnios-build/blob/44731424e67c8aaafe5c4e500fe7c4544a22f0f6/doc/ReleaseNotes.md#r151048o-2024-02-15) The Making of RP2040 Doom (https://kilograham.github.io/rp2040-doom/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brendan - Log Files (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/550/feedback/Brendan%20-%20Log%20Files.md) Mischa - EuroBSDcon (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/550/feedback/Mischa%20-%20EuroBSDcon.md) Sebastiano - Sed (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/550/feedback/Sebastiano%20-%20Sed.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
This week we're joined by FreeBSD & OpenZFS developer, Allan Jude, to learn all about FreeBSD. Allan gives us a brief history of BSD, tells us why it's his operating system of choice, compares it to Linux, explains the various BSDs out there & answers every curious question we have about this powerful (yet underrepresented) Unix-based operating system.
This week we're joined by FreeBSD & OpenZFS developer, Allan Jude, to learn all about FreeBSD. Allan gives us a brief history of BSD, tells us why it's his operating system of choice, compares it to Linux, explains the various BSDs out there & answers every curious question we have about this powerful (yet underrepresented) Unix-based operating system.
This is a recap of the top 10 posts on Hacker News on December 26th, 2023.This podcast was generated by wondercraft.ai(00:42): Japan to crack down on Apple and Google app store monopoliesOriginal post: https://news.ycombinator.com/item?id=38773429&utm_source=wondercraft_ai(02:34): A list of Hacker News's undocumented features and behaviorsOriginal post: https://news.ycombinator.com/item?id=38773957&utm_source=wondercraft_ai(04:00): Moderna's mRNA cancer vaccine works better than thoughtOriginal post: https://news.ycombinator.com/item?id=38775439&utm_source=wondercraft_ai(05:48): Website search hurts my feelingsOriginal post: https://news.ycombinator.com/item?id=38771513&utm_source=wondercraft_ai(07:38): Amazon Prime Video Will Start Showing Ads on January 29Original post: https://news.ycombinator.com/item?id=38777516&utm_source=wondercraft_ai(09:20): A copy-and-patch JIT compiler for CPythonOriginal post: https://news.ycombinator.com/item?id=38769874&utm_source=wondercraft_ai(11:10): Black Triangles (2014)Original post: https://news.ycombinator.com/item?id=38769850&utm_source=wondercraft_ai(12:56): Quality of care declines after private equity takes over hospitalsOriginal post: https://news.ycombinator.com/item?id=38773426&utm_source=wondercraft_ai(15:09): A data corruption bug in OpenZFS?Original post: https://news.ycombinator.com/item?id=38770168&utm_source=wondercraft_ai(17:15): Text Editor: Data StructuresOriginal post: https://news.ycombinator.com/item?id=38772754&utm_source=wondercraft_aiThis is a third-party project, independent from HN and YC. Text and audio generated using AI, by wondercraft.ai. Create your own studio quality podcast with text as the only input in seconds at app.wondercraft.ai. Issues or feedback? We'd love to hear from you: team@wondercraft.ai
Jim and Allan break down the details of the recent ZFS data corruption bug, and give their tips for managing a fleet of 40+ servers. Plug Support us on patreon and get an ad-free RSS feed with early episodes sometimes News Two new versions of OpenZFS fix long-hidden corruption bug Free Consulting […]
Jim and Allan break down the details of the recent ZFS data corruption bug, and give their tips for managing a fleet of 40+ servers. Plug Support us on patreon and get an ad-free RSS feed with early episodes sometimes News Two new versions of OpenZFS fix long-hidden corruption bug Free Consulting... Read More
This week we take a deep dive into the Reptar vuln in Intel processors plus we look into some relic vulnerabilities in Squid and OpenZFS and finally we detail new hardening measures in tracker-miners to keep your desktop safer.
This week Steve goes through his data migration story at his house. What things should you consider before moving large datasets around, and what things need to be taken into account for a solid backup plan? -- During The Show -- 01:52 Home Automation Leak Detection - Jeremy You can't really Using cameras 08:06 mmWave sensor update/comparison Seedstudio mmWave Sensor (https://wiki.seeedstudio.com/mmwave_human_detection_kit/) Space for other sensors Way better than a PIR sensor Aqara Water Sensor (https://cloudfree.shop/product/aqara-water-sensor/) 11:19 Point of sale gear? - Charlie Odoo (https://github.com/odoo/odoo) Open Source POS (https://github.com/opensourcepos/opensourcepos) UniCenta (https://unicenta.com/) Squirrel Systems (https://www.squirrelsystems.com/squirrel-pos-for-hotels) 13:28 Succession Planning - David Password dump Bitwarden Network diagram with pictures Good documentation Techy friends Dave Ramsey - Legacy box Legacy Folder Data, external drives 23:23 Odoo for Accounting and Bookkeeping - Tiny Looks like a solid platform Expensive Self hosting not really an option Accounting solid but very basic no payroll Not fully open source 25:51 Backups? - Mike Copying the file MIGHT be ok if file system has bit rot protection works till it doesn't Better to use database tools External drives 3.5 StarTech Enclosure (https://www.amazon.com/StarTech-com-10Gbps-Enclosure-SATA-Drives/dp/B00XLAZEFC) Pelican 1120 Case 2.5 Cable Matters Enclosure (https://www.amazon.com/Cable-Matters-Aluminum-External-Enclosure/dp/B07CQD6M5B) Steve's M.2 Enclosure (https://www.amazon.com/gp/product/B09T97Z7DM) ASUS ROG M.2 Enclosure (https://www.amazon.com/ASUS-ROG-Arion-Aluminum-Enclosure/dp/B07ZKB4SLK) 37:57 News Wire OpenZFS 2.2.1 - Phoronix (https://www.phoronix.com/news/OpenZFS-2.2.1-Released) Weston 13.0 - Freedesktop.org (https://lists.freedesktop.org/archives/wayland-devel/2023-November/043326.html) OpenSSL 3.2 - GitHub (https://github.com/openssl/openssl/blob/openssl-3.2.0/NEWS.md) PipeWire 1.0 - Phoronix (https://www.phoronix.com/news/PipeWire-1.0-Released) LibreOffice 7.6.3 On Android - Document Foundation (https://blog.documentfoundation.org/blog/2023/11/23/libreoffice-763-and-android-viewer-app/) Wine 8.21 - Gaming On Linux (https://www.gamingonlinux.com/2023/11/wine-821-brings-high-dpi-scaling-and-initial-vulkan-support-for-wayland/) Studio One 6.5 - Presonus Software (https://www.presonussoftware.com/en_US/blog/studio-one-6-5-for-linux) PeerTube v6 - Frama Blog (https://framablog.org/2023/11/28/peertube-v6-is-out-and-powered-by-your-ideas/) Proxmox 8.1 - Proxmox (https://www.proxmox.com/en/about/press-releases/proxmox-virtual-environment-8-1) OpenMandriva - LX 5.0 - Beta News (https://betanews.com/2023/11/25/openmandriva-lx-50-linux-download/) Nitrix 3.2.0 - NXOS.org (https://nxos.org/changelog/release-announcement-nitrux-3-2-0/) Ultra Marine Linux 39 - Fyra Labs (https://blog.fyralabs.com/ultramarine-39-released/) Linux 6.6 tagged LTS - Security Boulevard (https://securityboulevard.com/2023/11/linux-6-6-is-now-officially-an-lts-release/) Linux Runs 20% Faster on Ryzen 7995WX - Toms Hardware (https://www.tomshardware.com/news/ubuntu-runs-20-faster-than-windows-11-on-amd-threadripper-pro-7995wx) MicroCloud - Infoq (https://www.infoq.com/news/2023/11/canonical-microcloud-open-source/) GIMP Team Targeting May 2024 - Librearts.org (https://librearts.org/2023/11/gimp-3-0-roadmap/) X11 Being Removed from RHEL 10 - Red Hat (https://www.redhat.com/en/blog/rhel-10-plans-wayland-and-xorg-server) Fuctional Source License - The Register (https://www.theregister.com/2023/11/24/opinion_column/) Kinsing Malware - Hack Read (https://www.hackread.com/kinsing-crypto-malware-linux-apache-activemq-flaw/) SysJoker Malware - Cyber Security News (https://cybersecuritynews.com/sysjoker-malware-attacking-windows-linux-and-mac-users-abusing-onedrive/) Looney Tunables - Security Affairs (https://securityaffairs.com/154573/security/cisa-known-exploited-vulnerabilities-catalog-looney-tunables.html) Open Source Tesla - The Verge (https://www.theverge.com/2023/11/23/23973701/tesla-roadster-is-now-fully-open-source) AMD GPU & RISC-V - Toms Hardware (https://www.tomshardware.com/pc-components/gpus/amds-fastest-gaming-gpu-now-works-with-risc-v-cpus-amd-radeon-rx-7900-xtx-open-source-linux-drivers-available) Real AI - Mark Tech Post (https://www.marktechpost.com/2023/11/23/real-ai-wins-project-to-build-europes-open-source-large-language-model/) Synthetic Machine Learning Data - SD Times (https://sdtimes.com/data/capital-one-open-sources-new-project-for-generating-synthetic-data/) Uploading Minds - Crypto Slate (https://cryptoslate.com/buterin-sees-benefit-of-uploading-minds-and-need-for-open-source-innovation-in-ai/) AI Linux Optimization - Toms Hardware (https://www.tomshardware.com/news/chinese-company-uses-ai-to-optimize-linux-kernel) 41:11 Nativefier Makes native Linux app out of web pages Saves credentials and session Mind Drip One (http://docs.minddripone.com/how-to/install-use-nativefier/) Nativefier GUI GitHub (https://github.com/mattruzzi/nativefier-gui) 45:44 Data Migration Good to rotate drives Disk burn in (bunch of rsync) Rsync 26 hours rsync will preserve hard links with the right flags software raid is more portable nuke & pave 2 vdevs, 3 drives per vdev can only loose one drive ZFS send/receive is much faster and better IDrive (https://www.idrive.com/) Kopia (https://kopia.io/) Spider Oak One Plan for your target rsync commands a: Archive mode, which preserves permissions, ownership, and timestamps. v: Verbose mode, which prints out detailed information about the transfer. H: Preserve hard links. P: Preserve permissions. Dumping a database is intensive Proxmox gets in the way doesn't gain Steve anything Special snowflake Custom UI Good for multi node No updates KVM works the same everywhere Cockpit GUI Will eventually replace virtmanager -- The Extra Credit Section -- For links to the articles and material referenced in this week's episode check out this week's page from our podcast dashboard! This Episode's Podcast Dashboard (http://podcast.asknoahshow.com/365) Phone Systems for Ask Noah provided by Voxtelesys (http://www.voxtelesys.com/asknoah) Join us in our dedicated chatroom #GeekLab:linuxdelta.com on Matrix (https://element.linuxdelta.com/#/room/#geeklab:linuxdelta.com) -- Stay In Touch -- Find all the resources for this show on the Ask Noah Dashboard Ask Noah Dashboard (http://www.asknoahshow.com) Need more help than a radio show can offer? Altispeed provides commercial IT services and they're excited to offer you a great deal for listening to the Ask Noah Show. Call today and ask about the discount for listeners of the Ask Noah Show! Altispeed Technologies (http://www.altispeed.com/) Contact Noah live [at] asknoahshow.com -- Twitter -- Noah - Kernellinux (https://twitter.com/kernellinux) Ask Noah Show (https://twitter.com/asknoahshow) Altispeed Technologies (https://twitter.com/altispeed)
First up in the news: LXD-Linux Containers Forks LXD Project as "Incus", Indian Defense Services are Switching to Linux, Mint's birthday coming up, Messaging Layer Security, State of Solus, Alarm raised over MozillaVPN, EFF launches the TOR University Challenge, news on OpenZFS and ZFSBoot, and Devuan 5 is here In security and privacy: SkidMap and Downfall Then in our Wanderings: Bill and Majid have cloudy days Download
This is a recap of the top 10 posts on Hacker News on August 19th, 2023.This podcast was generated by wondercraft.ai(00:42): Amsterdam to use “noise cameras” against too loud carsOriginal post: https://news.ycombinator.com/item?id=37187837&utm_source=wondercraft_ai(02:26): Windy.com: global weather website with live filtersOriginal post: https://news.ycombinator.com/item?id=37187760&utm_source=wondercraft_ai(03:52): Why do old books smell so good?Original post: https://news.ycombinator.com/item?id=37188015&utm_source=wondercraft_ai(05:29): Stable Diffusion in C/C++Original post: https://news.ycombinator.com/item?id=37187663&utm_source=wondercraft_ai(07:19): WD refused to answer our questions about its self-wiping SanDisk SSDsOriginal post: https://news.ycombinator.com/item?id=37188736&utm_source=wondercraft_ai(09:17): Cheems, the Shiba Inu meme dog, has diedOriginal post: https://news.ycombinator.com/item?id=37189067&utm_source=wondercraft_ai(10:41): 70 years ago, an Anglo-US coup condemned Iran to decades of oppressionOriginal post: https://news.ycombinator.com/item?id=37191220&utm_source=wondercraft_ai(12:52): Iraq Blocks Telegram, Leaks Blackhole BGP RoutesOriginal post: https://news.ycombinator.com/item?id=37191332&utm_source=wondercraft_ai(14:26): OpenZFS – add disks to existing RAIDZOriginal post: https://news.ycombinator.com/item?id=37190493&utm_source=wondercraft_ai(16:05): AI-generated art lacks copyright protection, D.C. court saysOriginal post: https://news.ycombinator.com/item?id=37188791&utm_source=wondercraft_aiThis is a third-party project, independent from HN and YC. Text and audio generated using AI, by wondercraft.ai. Create your own studio quality podcast with text as the only input in seconds at app.wondercraft.ai. Issues or feedback? We'd love to hear from you: team@wondercraft.ai
OpenZFS, Your Data and the Challenge of Ransomware, I Didn't Learn Unix By Reading All The Manpages, I try to answer "how to become a systems engineer", Writing shell scripts in Nushell, Sudo and signal propagation, infecting SSH Public Keys with backdoors, OpenBSD Thinkpad, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines OpenZFS, Your Data and the Challenge of Ransomware (https://klarasystems.com/articles/openzfs-openzfs-your-data-and-the-challenge-of-ransomware/) I Didn't Learn Unix By Reading All The Manpages (https://www.owlfolio.org/research/i-didnt-learn-unix-by-reading-all-the-manpages/) News Roundup Feedback: I try to answer "how to become a systems engineer" (https://rachelbythebay.com/w/2023/05/30/eng/) Writing shell scripts in Nushell (https://jpospisil.com/2023/05/25/writing-shell-scripts-in-nushell) Sudo and signal propagation (https://dxuuu.xyz/sudo.html) Infecting SSH Public Keys with backdoors (https://blog.thc.org/infecting-ssh-public-keys-with-backdoors) OpenBSD Thinkpad (https://douglasrumbaugh.com/post/openbsd-thinkpad-good/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Recent advances in embedded Linux, Canonical takes full control of LXD, ZFS gets a handy Btrfs feature, and updates on the show's production.
Recent advances in embedded Linux, Canonical takes full control of LXD, ZFS gets a handy Btrfs feature, and updates on the show's production.
Sun Ray laptops, MIPS and getting root on them, OpenZFS for HPC Clusters, Self-Hosted Bookmarks using DAV and httpd on OpenBSD, Terraform + Proxmox + OpenBSD =
We are unimpressed by Apple's new headset, a particularly bountiful watering hole attack, misdirection from the AI industry, and connecting hard disks via a PCIe card. Plugs Support us on patreon OpenZFS, Your Data and the Challenge of Ransomware News Apple Vision Pro Hololens dev Twitter thread (archived version) Some Curseforge accounts might […]
FreeBSD Foundation Welcomes New Team Members, OpenZFS the Ideal Storage Solution for University Environments, SCaLE20X Conference Report, 916 days of Emacs, XTerm: It's Better Than You Thought, NetBSD Annual General Meeting 2023, and more NOTES** This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Foundation Welcomes New Team Members (https://freebsdfoundation.org/blog/freebsd-foundation-welcomes-new-team-members/) What Makes OpenZFS the Ideal Storage Solution for University Environments (https://klarasystems.com/articles/openzfs-what-makes-openzfs-the-ideal-storage-solution-for-university-environments//) News Roundup SCaLE20X Conference Report (https://freebsdfoundation.org/blog/scale20x-conference-report/) 916 days of Emacs (https://sqrtminusone.xyz/posts/2023-04-13-emacs/) XTerm: It's Better Than You Thought (https://aduros.com/blog/xterm-its-better-than-you-thought/) NetBSD AGM2023: Annual General Meeting, May 13, 21:00 UTC (https://mail-index.netbsd.org/netbsd-announce/2023/05/05/msg000348.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Adrian - Tilde (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/508/feedback/Adrian%20-%20Tilde.md) Dan - Root Shell (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/508/feedback/Dan%20-%20Root%20Shell.md) Florian - Salt Extension (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/508/feedback/Florian%20-%20Salt%20Extension.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
OpenZFS auditing for storage Performance, Privilege drop; privilege separation; and restricted-service operating mode in OpenBSD, OPNsense 23.1.1 release, Cloning a System with Ansible, FOSDEM 2023, BSDCan 2023 Travel Grants NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines OpenZFS auditing for storage Performance (https://klarasystems.com/articles/openzfs-auditing-for-storage-performance/) Privilege drop, privilege separation, and restricted-service operating mode in OpenBSD (https://sha256.net/privsep.html) News Roundup OPNsense 23.1.1 released (https://forum.opnsense.org/index.php?topic=32484.0) Cloning a System with Ansible (https://kernelpanic.life/software/cloning-a-system-with-ansible.html) FOSDEM 2023 (http://blog.netbsd.org/tnf/entry/fosdem_2023) BSDCan 2023 Travel Grant Application Now Open (https://freebsdfoundation.org/blog/bsdcan-2023-travel-grant-application-now-open/) The Undeadly Bits Game of Trees milestone (http://undeadly.org/cgi?action=article;sid=20230120073530) Game of Trees Daemon - video and slides (May make the older game of trees obsolete) (http://undeadly.org/cgi?action=article;sid=20230210065830) amd64 execute-only committed to -current (http://undeadly.org/cgi?action=article;sid=20230121125423) Using /bin/eject with USB flash drives (http://undeadly.org/cgi?action=article;sid=20230214061952) Tunneling vxlan(4) over WireGuard wg(4) (http://undeadly.org/cgi?action=article;sid=20230214061330) Console screendumps (http://undeadly.org/cgi?action=article;sid=20230128183032) Execute-only status report (http://undeadly.org/cgi?action=article;sid=20230130061324) OpenBSD in Canada (http://undeadly.org/cgi?action=article;sid=20230226065006) Privilege drop, privilege separation, and restricted-service operating mode in OpenBSD (http://undeadly.org/cgi?action=article;sid=20230219234206) Theo de Raadt on pinsyscall(2) (http://undeadly.org/cgi?action=article;sid=20230222064027) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Kevin - PLUG (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/498/feedback/Kevin%20-%20PLUG.md) Luna - FOSDEM (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/498/feedback/Luna%20-%20FOSDEM.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
Ubuntu makes its anti-Flatpak stance official, while KDE and GNOME team up to turn Flathub into a universal Linux app store. Plus, we try the Intel Arc GPU. Could this new hardware make Linux bulletproof?
OpenZFS has performance gains inbound, the end of a Linux era, and the achievement unlocked by the open-source NVIDIA driver.
OpenZFS has performance gains inbound, the end of a Linux era, and the achievement unlocked by the open-source NVIDIA driver.
OpenZFS has performance gains inbound, the end of a Linux era, and the achievement unlocked by the open-source NVIDIA driver.
OpenZFS has performance gains inbound, the end of a Linux era, and the achievement unlocked by the open-source NVIDIA driver.
The Cloud Pod recaps all of the positives and negatives of Amazon ReInvent 2022, the annual conference in Las Vegas, bringing together 50,000 cloud computing professionals. This year's keynote speakers include Adam Selpisky, CEO of Amazon Web Services, Swami Sivasubramanian, Vice President of Data and Machine Learning at AWS and Werner Vogels, Amazon's CTO. Attendees and web viewers were treated to new features and products, such as AWS Lambda Snapstart for Java Functions, New Quicksight capabilities and quality-of-life improvements to hundreds of services. Justin, Jonathan, Ryan, Peter and Special guest Joe Daly from the Finops foundation talk about the show and the announcements. Thank you to our sponsor, Foghorn Consulting, which provides top notch cloud and DevOps engineers to the world's most innovative companies. Initiatives stalled because you're having trouble hiring? Foghorn can be burning down your DevOps and Cloud backlogs as soon as next week. Episode Highlights ⏰ AWS Pricing Calculator now supports modernization cost estimates for Microsoft workloads. ⏰ AWS Re:Invent 2022 announcements and keynote updates. Top Quote
FreeBSD Q3 2022 status report, Leveraging MinIO and OpenZFS to avoid vendor lock in, FreeBSD on Firecracker platform, How Much Faster Is Making A Tar Archive Without Gzip, Postgres from packages on OpenBSD, Upgrading an NVMe zpool from 222G to 1TB drives, Don't use Reddit for Linux or BSD related questions, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Quarterly Status Report Third Quarter 2022 (https://www.freebsd.org/status/report-2022-07-2022-09/) Avoid Infrastructure Vendor Lock-in by leveraging MinIO and OpenZFS (https://klarasystems.com/articles/avoid-vendor-lock-in-with-minio-and-openzfs/) Announcing the FreeBSD/Firecracker platform (https://www.daemonology.net/blog/2022-10-18-FreeBSD-Firecracker.html) News Roundup How Much Faster Is Making A Tar Archive Without Gzip? (https://lowendbox.com/blog/how-much-faster-is-making-a-tar-archive-without-gzip/) PostgreSQL from packages on OpenBSD (https://www.dbi-services.com/blog/postgresql-from-packages-on-openbsd/) Upgrading an NVMe zpool from 222G to 1TB drives (https://dan.langille.org/2022/10/18/upgrading-an-nvme-zpool-from-222g-to-1tb-drives/) PSA: Don't use Reddit for Linux or BSD related questions (https://unixsheikh.com/articles/dont-use-reddit-for-linux-or-bsd-related-questions.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Hinnerk - vnet jails (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/481/feedback/Hinnerk%20-%20vnet%20jails.md) Tom's response example: https://adventurist.me/posts/00304 Hugo - Apple M2 (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/481/feedback/Hugo%20-%20Apple%20M2.md) kevin - emacs backspace (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/481/feedback/kevin%20-%20emacs%20backspace.md) ) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Why this latest release of Fedora misses the mark, and Ubuntu's quiet backing away from ZFS.
This week, Noah and Steve tell you about the new Intel NUC that supports a TRIPPLE GPU Slot, a company that provides commercial support for Linux users, and the courts are divided on the scope of cell phone searches! -- During The Show -- 04:00 Rural 5G Internet - Chris User Hardware Netgear RAX45 ZTE MC8010CA Telus in Canada CGNAT Cradle Point (https://cradlepoint.com/) Peplink (https://www.peplink.com/) 07:40 Listener Responds to 305 - Greg White list "assets.adobedtm.com" 09:20 HDMI-CEC - Avri Wikipedia CEC (https://en.wikipedia.org/wiki/Consumer_Electronics_Control) HDMI 2.0a Ethernet over HDMI (https://thehometheaterdiy.com/hdmi-with-ethernet/) Cable companies are going to IP TV STI Cables STI HDMI Decimator (https://www.btx.com/decimator-md-lx-hdmi-sdi-converter) 15:00 Listener follows up on own hosting question - Gary A2 Hosting 16:15 ParrotSec ParrotSec (https://parrotsec.org) Multiple flavors/editions 18:30 Euro Linux Euro Linux (https://en.euro-linux.com/eurolinux/desktop/) Euro Linux Review (https://www.debugpoint.com/eurolinux-desktop-review/) Desktop Functionality - Server Grade Quality Compatible with RHEL Simple Update Management 22:30 NUC Desktop ARS Technica (https://arstechnica.com/gadgets/2022/10/intel-is-making-a-nuc-desktop-thats-big-enough-for-a-triple-slot-gpu/) Tripple GPU Takes out the guess work Raptor Canyon Box K-Series Core i9, i7, i5 750 Watt power supply 32:28 News Wire NVK Driver Gaming on Linux (https://www.gamingonlinux.com/2022/10/nvk-is-a-new-open-source-mesa-vulkan-driver-for-nvidia-gpus/) Collabora (https://www.collabora.com/news-and-blog/news-and-events/introducing-nvk.html) Meta AI Inference Engine Analytics India Mag (https://analyticsindiamag.com/meta-open-sources-an-ai-inference-engine-that-works-on-both-nvidia-amd-gpus/) Kaos Linux 22.10 Source Forge (https://sourceforge.net/projects/kaosx/files/ISO/KaOS-2022.10-x86_64.iso) Robo Linux 12.08 Source Forge (https://sourceforge.net/projects/robolinux/files/) EasyOS 4.4.1 BK Home (https://bkhome.org/news/202210/easyos-441-released.html) Fatdog64 813 Puppy Linux (https://forum.puppylinux.com/viewtopic.php?t=7003) SparkyLinux 22.10 Sparky LInux (https://sparkylinux.org/sparky-2022-10/) Tuxedo OS Live ISO 9 to 5 Linux (https://9to5linux.com/tuxedo-computers-releases-live-iso-of-its-ubuntu-based-tuxedo-os-linux-distro) LibreSSL 3.6.0 Open BSD Journal (https://www.undeadly.org/cgi?action=article;sid=20221006105921) OpenSSH 9.1 Open BSD Journal (https://www.undeadly.org/cgi?action=article;sid=20221006091946) OpenBGPD 7.7 Open BSD Journal (https://www.undeadly.org/cgi?action=article;sid=20221007085644) OpenZFS 2.1.6 -Github (https://github.com/openzfs/zfs/releases/tag/zfs-2.1.6) WireShark 4.0 WireShark (https://www.wireshark.org/docs/relnotes/wireshark-4.0.0.html) Blender 3.3.1 LTS Blender Nation (https://www.blendernation.com/2022/10/07/blender-3-3-1-lts-and-blender-2-93-11-lts-released/) SUSE ALP v0.01 The Register (https://www.theregister.com/2022/10/05/suse_alp_v001/) Endor Labs $25M Silicon Angle (https://siliconangle.com/2022/10/10/endor-labs-launches-25m-secure-open-source-code-dependencies/) Legitify Open Source For U (https://www.opensourceforu.com/2022/10/a-new-open-source-scaling-and-security-tool-is-now-available-to-download/) MS 'Farm of the Future' Microsoft (https://blogs.microsoft.com/ai/microsoft-open-sources-its-farm-of-the-future-toolkit/) 34:30 Cellphone Searches EFF Article (https://www.eff.org/deeplinks/2022/10/new-federal-and-state-court-rulings-show-courts-are-divided-scope-cell-phone) Law Justia (https://law.justia.com/cases/federal/appellate-courts/ca5/19-10842/19-10842-2021-01-05.html) Richardson v. State United States v. Morton EFF Filed amicus brief Good Faith Exception Password vs Bio-Metrics/Reboot the phone 51:00 Launch of "Critical Thought" Noah's New Talk Show (News Radio 1310 AM KNOX) Focused on local & regional issues. Airs Monday - Friday 09:00am - 12:00pm (Central) Streamed online at knoxradio.com (https://player.listenlive.co/64061) More time to invest in Ask Noah Show / creating content. -- The Extra Credit Section -- For links to the articles and material referenced in this week's episode check out this week's page from our podcast dashboard! This Episode's Podcast Dashboard (http://podcast.asknoahshow.com/307) Phone Systems for Ask Noah provided by Voxtelesys (http://www.voxtelesys.com/asknoah) Join us in our dedicated chatroom #GeekLab:linuxdelta.com on Matrix (https://element.linuxdelta.com/#/room/#geeklab:linuxdelta.com) -- Stay In Touch -- Find all the resources for this show on the Ask Noah Dashboard Ask Noah Dashboard (http://www.asknoahshow.com) Need more help than a radio show can offer? Altispeed provides commercial IT services and they're excited to offer you a great deal for listening to the Ask Noah Show. Call today and ask about the discount for listeners of the Ask Noah Show! Altispeed Technologies (http://www.altispeed.com/) Contact Noah live [at] asknoahshow.com -- Twitter -- Noah - Kernellinux (https://twitter.com/kernellinux) Ask Noah Show (https://twitter.com/asknoahshow) Altispeed Technologies (https://twitter.com/altispeed)
Full system backups with FFS snapshots, ZFS and dump(8), tuning recordsize in OpenZFS, Optimizing FreeBSD Power Consumption on Modern Intel Laptops, remember to check for ZFS filesystems being mounted, Use tcpdump to save wireless bridge, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Full system backups with FFS snapshots, ZFS and dump(8) (https://www.unitedbsd.com/d/705-full-system-backups-with-ffs-snapshots-zfs-and-dump8) Tuning Recordsize in OpenZFS (https://klarasystems.com/articles/tuning-recordsize-in-openzfs/) News Roundup Optimizing FreeBSD Power Consumption on Modern Intel Laptops (https://www.neelc.org/posts/optimize-freebsd-for-intel-tigerlake/) I need to remember to check for ZFS filesystems being mounted (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSCheckForMounted) Use tcpdump to save wireless bridge (https://adventurist.me/posts/0027) Beastie Bits • [FreeBSD on the Vortex86DX CPU](https://www.cambus.net/freebsd-on-the-vortex86dx-cpu/) • [HAMMER2 vs USB stick pulls](https://www.dragonflydigest.com/2022/03/22/26800.html) • [New US mirror for DragonFly](https://www.dragonflydigest.com/2022/03/09/26742.html) • [HelloSystem 13.1 RC1](https://github.com/helloSystem/ISO/releases/tag/experimental-13.1-RC1) • [Video introduction to OpenBSD 7.0](https://www.youtube.com/watch?v=KeUsE-3nSes) • [Losses in the community](https://minnie.tuhs.org/pipermail/tuhs/2022-April/025643.html) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Sam - BSD Laptops (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/451/feedback/Sam%20-%20BSD%20Laptops.md) Reese - Electric Groff (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/451/feedback/Reese%20-%20Electric%20Groff.md) Alexandra - New to BSD (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/451/feedback/Alexandra%20-%20New%20to%20BSD.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
An open source dev goes rogue, a big single sign-on provider is compromised, self-hosting DNS, and more. Plugs Should I upgrade to OpenZFS 2.1? Support us on patreon News BIG sabotage: Famous npm package deletes files to protest Ukraine war The Fragile Open Source Ecosystem Isn't Ready for ‘Protestware' Okta Hack Exposes A […]
This week we take your questions then dig into the releases that are hot this week. KDE, Gnome, Pine64, and System76 all have something for you this week! -- During The Show -- 02:10 User recommends Self Hosted Email - James Mailu (https://mailu.io/1.9/) 04:15 KDE Plasma Questions - Stephen GVfs (https://en.wikipedia.org/wiki/GVfs) SystemD FSTAB mount SMB4K (https://smb4k.sourceforge.io/) 10:30 Trouble Running BricCad - Ryan BricCad (https://www.bricsys.com/) Malformed URL in AUR package Wayland permissions? strace (https://strace.io/) 13:25 Caller Jon NAS Backup Use a file system with compression Focus on data size MediaSonic Enclosure (http://www.amazon.com/dp/B003X26VV4/?tag=minddripmedia-20) SpiderOak (https://spideroak.com/) Ice Drive (https://icedrive.net/) 19:40 Pick of The Week The Note Binder (https://drewtechs.net/projects/note-binder#Section1.1) Note Organizing Binder Multiple file types 22:55 News Wire [Open Source in Film Making] ProVideo Coalition (https://www.provideocoalition.com/how-open-source-technology-is-used-in-modern-filmmaking/) ASWF (http://report.aswf.io/) Apache NiFi & Pulsar (https://www.datanami.com/2022/03/09/code-for-pulsar-nifi-tie-up-now-open-source/) New Spectre Vulnerabilities (https://www.vusec.net/projects/bhi-spectre-bhb/) OpenZFS 2.1.3 (https://www.theregister.com/2022/03/12/openzfs_213/) Arch Linux 20th Birthday (https://archlinux.org/retro/2002/) WhatsApp Code Verification (https://engineering.fb.com/2022/03/10/security/code-verify/) LF Energy (https://thenewstack.io/the-linux-foundations-open-source-stack-for-ev-charging-infrastructure/) PyGears AI Chip Design (https://www.hpcwire.com/off-the-wire/ucla-adopts-pygears-an-open-source-framework-for-ai-chip-design/) 24:50 Pine64 - Introducing The QuartzPro64 New single board computer First Pro-Grade SBC since RockPro64 $300+ Pine64 Blog (https://www.pine64.org/2022/03/15/march-update-introducing-the-quartzpro64/) 27:00 Pipewire Bluetooth audio stack improvements Future Proof Collabora worked on WirePlumber Auto profile switching Collabora Blog (https://www.collabora.com/news-and-blog/blog/2022/03/08/pipewire-a-year-in-review-look-ahead/) 33:30 Gnome 42 RC Added abilities and defaults Fingerprint dialog makes a come back FIDO2 Standard 9 to 5 Linux Article (https://9to5linux.com/gnome-42-release-candidate-brings-back-fingerprint-dialog-in-control-center) 38:00 KDE 5.92 Available New features and improvements KDE Frameworks 5.92 Spectacle and OBS KDE Neon (https://neon.kde.org/) 9 to 5 Linux Article (https://9to5linux.com/kde-frameworks-5-92-improves-dolphin-and-krunner-plugs-memory-leaks-and-fixes-bugs) 40:50 SSHGuard on Ubuntu 20.04 Blocks brute forcing SSH Multiple log formats Vitux.com (https://vitux.com/how-to-install-and-use-sshguard-on-ubuntu-20-04/) 43:03 System76 Open Source Keyboard Hot-swappable mechanical switches Open Source Chassis, PCB, and firmware Split Spacebar USB Hub Layers ArsTechnica Article (https://arstechnica.com/gadgets/2022/03/system76-launch-review-linux-friendly-keyboard-with-a-usb-hub/) Red Dragon Keyboad (https://redragonadria.com/product/surara-k582rgb-gaming-keyboard/) 50:00 Synergy Alternative / KVM Sharing Barrier (https://github.com/debauchee/barrier/) Cross Platform, should "just work" Everything is done in the open -- The Extra Credit Section -- For links to the articles and material referenced in this week's episode check out this week's page from our podcast dashboard! This Episode's Podcast Dashboard (http://podcast.asknoahshow.com/277) Phone Systems for Ask Noah provided by Voxtelesys (http://www.voxtelesys.com/asknoah) Join us in our dedicated chatroom #GeekLab:linuxdelta.com on Matrix (https://element.linuxdelta.com/#/room/#geeklab:linuxdelta.com) -- Stay In Touch -- Find all the resources for this show on the Ask Noah Dashboard Ask Noah Dashboard (http://www.asknoahshow.com) Need more help than a radio show can offer? Altispeed provides commercial IT services and they're excited to offer you a great deal for listening to the Ask Noah Show. Call today and ask about the discount for listeners of the Ask Noah Show! Altispeed Technologies (http://www.altispeed.com/) Contact Noah live [at] asknoahshow.com -- Twitter -- Noah - Kernellinux (https://twitter.com/kernellinux) Ask Noah Show (https://twitter.com/asknoahshow) Altispeed Technologies (https://twitter.com/altispeed) Special Guest: Steve Ovens.
Amazon FSx provides fully managed file storage in the cloud and offers capabilities and performance of popular commercial and open-source file systems. Customers can choose between four file systems: NetApp ONTAP, OpenZFS, Windows File Server, and Lustre. In this episode, Simon is joined by Andrew Crudge, (Principal Product Manager, AWS) and Delwin Olivan, (Senior Product Manager, AWS) to dive into the Amazon FSx family, discussing the latest Amazon FSx innovations and use cases, as well as AWS programs to help you migrate and modernize your file-based applications faster. Choosing a file system - https://aws.amazon.com/fsx/when-to-choose-fsx/ AWS Storage blog - https://aws.amazon.com/blogs/storage/ Learn more - https://aws.amazon.com/fsx/
The Linux secret behind the new TrueNAS release, Intel acquires a major Kernel contributor and our thoughts on Podman 4.0. Plus why the Simula One VR Linux computer could be worth a serious look.
Linux's résumé got a nice boost this week; why Google is paying for more kernel development, and how CloudLinux might be pulling ahead of the CentOS pack. Plus, our thoughts on Steam possibly coming to ChromeOS and the game-changing feature coming to ZFS.
Why we don't think Red Hat's expanded developer program is enough, our reaction to Ubuntu sticking with an older Gnome release, and a tiny delightful surprise.
On this episode of This Week in Linux, SUSE's acquisition of Rancher Labs is now complete and I'll tell you why this matters. System76 announced their first entry into the world of AMD powered Linux Laptops with the new Pangolin laptop. A developer seems to want to prove me wrong about Linux Support for Apple's M1 Mac so we'll talk about that. Pine64 announced that the KDE PinePhone is available for pre-order. Plus we've got a lot of desktop environment news this week with new releases of both Linux Mint's Cinnamon and Solus' Budgie desktop and the dreaded single-threaded issue for GNOME may be coming to an end. All that and much more coming up right now on This Week in Linux! SPONSORED BY: Digital Ocean ►► https://do.co/dln Bitwarden ►► https://bitwarden.com/dln TWITTER ►► https://twitter.com/michaeltunnell MASTODON ►► https://mastodon.social/@MichaelTunnell DLN COMMUNITY ►► https://destinationlinux.network/contact FRONT PAGE LINUX ►► https://frontpagelinux.com MERCH ►► https://dlnstore.com BECOME A PATRON ►► https://tuxdigital.com/contribute This Week in Linux is a Proud Member of the Destination Linux Network! https://destinationlinux.network SHOW NOTES ►► https://tuxdigital.com/twinl128 00:00 = Coming up on TWIL 128 01:06 = SUSE Acquisition Of Rancher Labs 04:11 = System76 Pangolin AMD Linux Laptop 06:11 = Digital Ocean - Cloud Hosting Kubernetes ( https://do.co/dln ) 07:00 = Developer Working on Linux for Apple Silicon 11:13 = PinePhone KDE Edition Pre-Orders 14:44 = GNOME 40 Moves Input Work To Separate Thread 17:44 = Bitwarden: Password Manager ( https://bitwarden.com/dln ) 18:56 = Budgie Desktop 10.5.2 Released 21:38 = Cinnamon Desktop 4.8 Released 23:40 = Open Source Security Summit 25:43 = Housekeeping: Live Streams Game Sphere Chativerse Oh My! 26:49 = NetMarketShare Wnds Browser War Reports 29:22 = OpenZFS 2.0 Released Other Videos: 6 Cool Things You Didn't Know About Linux's History: https://youtu.be/u9ZY41mNB9I How To Use Firefox's Best Feature, Multi-Account Containers: https://youtu.be/FfN5L5zAJUo Linux Explained - How Some Distros Are Based On Other Distros: https://youtu.be/OWk3D6x64tk 7 Reasons Why Firefox Is My Favorite Web Browser: https://youtu.be/bGTBH9yr8uw Thanks For Watching! Linux #OpenSource #TechNews
We talk about our recent trip to FOSDEM, we discuss the pros and cons of permissive licensing, cover the installation of OpenBSD on a dedibox with full-disk encryption, the new Lumina guide repository, and we explain ZFS vs. OpenZFS. This episode was brought to you by Headlines [FOSDEM Trip report] Your BSDNow hosts were both at FOSDEM in Brussels, Belgium over the weekend. On the friday before FOSDEM, we held a FreeBSD devsummit (3rd consecutive year), sponsored by the FreeBSD Foundation and organized by Benedict (with the help from Kristof Provost, who did it in previous years but could not make it this year). We had 21 people attend, a good mixture of FreeBSD committers (mostly ports) and guests. After introductions, we collected topics and discussed various topics, including a new plan for a future FreeBSD release roadmap (more frequent releases, so that features from HEAD can be tried out earlier in RELEASES). The devsummit concluded with a nice dinner in a nearby restaurant. On Saturday, first day of FOSDEM, we set up the FreeBSD Foundation table with flyers, stickers, FreeBSD Journal print editions, and a small RPI 3 demo system that Deb Goodkin brought. Our table was located next to the Illumos table like last year. This allowed us to continue the good relationship that we have with the Illumos people and Allan helped a little bit getting bhyve to run on Illumos with UEFI. Meanwhile, our table was visited by a lot of people who would ask questions about FreeBSD, take info material, or talk about their use cases. We were busy refilling the table throughout the day and luckily, we had many helpers at the table. Some items we had ran out in the early afternoon, an indicator of how popular they were. Saturday also featured a BSD devroom (https://twitter.com/fosdembsd), organized by Rodrigo Osorio. You can find the list of talks and the recordings on the BSD Devroom schedule (https://fosdem.org/2018/schedule/track/bsd/). The room was very crowded and popular. Deb Goodkin gave the opening talk with an overview of what the Foundation is doing to change the world. Other speakers from various BSD projects presented their talks after that with a range of topics. Among them, Allan gave his talk about ZFS: Advanced Integration (https://fosdem.org/2018/schedule/event/zfs_advanced_integration/), while Benedict presented his Reflections on Teaching a Unix Class With FreeBSD (https://fosdem.org/2018/schedule/event/reflections_on_reaching_unix_class_with_freebsd/). Sunday was just as busy on the FreeBSD table as Saturday and we finally ran out of stickers and some other goodies. We were happy with the results of the two days. Some very interesting conversations at the table about FreeBSD took place, some of which we're going to follow up afterwards. Check out the FOSDEM schedule as many talk recordings are already available, and especially the ones from the BSD devroom if you could not attend the conference. We would like to thank everyone who attended the FreeBSD devsummit, who helped out at the FreeBSD table and organized the BSD devroom. Also, thanks to all the speakers, organizers, and helping hands making FOSDEM another success this year. *** NetBSD kernel wscons IOCTL vulnerable bug class (http://blog.infosectcbr.com.au/2018/01/netbsd-kernel-wscons-ioctl-vulnerable.html) I discovered this bug class during the InfoSect public code review session we ran looking specifically at the NetBSD kernel. I found a couple of these bugs and then after the session was complete, I went back and realised the same bug was scattered in other drivers. In total, 17 instances of this vulnerability and its variants were discovered. In all fairness, I came across this bug class during my kernel audits in 2002 and most instances were patched. It just seems there are more bugs now in NetBSD while OpenBSD and FreeBSD have practically eliminated them. See slide 41 in http://www.blackhat.com/presentations/bh-usa-03/bh-us-03-cesare.pdf (http://www.blackhat.com/presentations/bh-usa-03/bh-us-03-cesare.pdf) for exactly the same bug (class) 16 years ago. The format of the this blog post is as follows: Introduction Example of the Bug Class How to Fix How to Detect Automatically with Coccinelle More Bugs Conclusion These source files had bugs ./dev/tc/tfb.c ./dev/ic/bt485.c ./dev/pci/radeonfb.c ./dev/ic/sti.c ./dev/sbus/tcx.c ./dev/tc/mfb.c ./dev/tc/sfb.c ./dev/tc/stic.c ./dev/tc/cfb.c ./dev/tc/xcfb.c ./dev/tc/sfbplus.c ./arch/arm/allwinner/awin_debe.c ./arch/arm/iomd/vidcvideo.c ./arch/pmax/ibus/pm.c ./dev/ic/igfsb.c ./dev/ic/bt463.c ./arch/luna68k/dev/lunafb.c Reporting of the bugs was easy. In less than a week from reporting the specific instances of each bug, patches were committed into the mainline kernel. Thanks to Luke Mewburn from NetBSD for coming to the code review session at InfoSect and coordinating with the NetBSD security team. The patches to fix these issues are in NetBSD: https://mail-index.netbsd.org/source-changes/2018/01/24/msg091428.html (https://mail-index.netbsd.org/source-changes/2018/01/24/msg091428.html) "Permissive licensing is wrong!” – Is it? (https://eerielinux.wordpress.com/2017/11/25/permissive-licensing-is-wrong-is-it-1-2/) A few weeks ago I've been attacked by some GNU zealots on a German tech site after speaking in favor of permissive licenses. Unfortunately a discussion was not possible there because that would require the will to actually communicate instead of simply accusing the other side of vile motives. Since I actually do care about this topic and a reader asked for a post about it in comments a while ago, here we go. This first part tries to sum up the most important things around the topic. I deliberately aim for an objective overview that tries not to be one-sided. The second part will then contain my points in defence of permissive licensing. Why license software at all? Licenses exist for reasons of protection. If you're the author/inventor of some software, a story or whatever product, you get to decide what to do with it. You can keep it for yourself or you can give it away. If you decide for the latter, you have to decide who may use it and in which way(s). In case you intend to give it to a (potentially) large group of people, you may not want to be asked for permission to xyz by everybody. That's when you decide to write a license which states what you are allowing and explicitly disallowing. Most of the well-known commercial licenses focus on what you're not allowed to do (usually things like copying, disassembling, etc.). Open source licenses on the other hand are meant to grant the user rights (e.g. the right to distribute) while reserving some rights or only giving permission under certain conditions – and they usually make you claim responsibility for using the software. For these reasons licenses can actually be a good thing! If you got an unlicensed piece of code, you're not legally allowed to do anything with it without getting the author's permission first. And even if you got that permission, your project would be risky, since the author can withdraw it later. A proper license protects both parties. The author doesn't get his mail account full of email asking for permission, he's save from legal trouble if his code breaks anything for you and at the same time you have legal certainty when you decide to put the code to long-term use. Permissive vs. Copyleft (in a nutshell) In short terms, permissive licensing usually goes like this: “Here you are, have fun. Oh, and don't sue me if it does something else than what you expect!” Yes, it's that easy and there's little to dispute over. Copyleft on the other side sounds like this (if you ask somebody in favor of Copyleft): “Sure, you can use it, it's free. Just keep it free, ok?”. Also quite simple. And not too bad, eh? Other people however read the same thing like this: “Yes, you're free to use it. Just read these ten pages of legalese and be dead certain that you comply. If you got something wrong, we will absolutely make you regret it.” The GNU Public license (GPL) The most popular copyleft license in use is the GPL (in various versions) (https://www.gnu.org/licenses/gpl.html). It got more and more complex with each version – and to be fair, it had to, because it was necessary to react to new threats and loop holes that were found later. The GNU project states that they are committed to protect what they call the four freedoms of free software: the freedom to use the software for any purpose the freedom to change the software to suit your needs the freedom to share the software with your friends and neighbors the freedom to share the changes you make These are freedoms that every supporter of open source software should be able to agree with. So what's the deal with all the hostility and fighting between the two camps? Let's take a look at a permissive license, too. The BSD license Unlike the GPL, the BSD family of licenses begun with a rather simple license that span four rules (“original BSD license”). It was later revised and reduced to three (“modified BSD license”). And the modern BSD license that e.g. FreeBSD uses is even just two (“simplified BSD license”). Did you read the GPLv3 that I linked to above? If you are using GPL'd code you really should. In case you don't feel like reading all of it, at least take a look and grasp how long that text is. Now compare it to the complete modern BSD license (https://opensource.org/licenses/bsd-license.php). What's the problem? There are essentially two problems that cause all the trouble. The first one is the question of what should be subject to the freedom that we're talking about. And closely related, the second one is where that freedom needs to end. Ironically both camps claim that freedom is the one important thing and it must not be restricted. The GPL is meant to protect the freedom of the software and enforces the availability of the source code, hence limiting the freedom of actual persons. BSD on the other hand is meant to protect the freedom of human beings who should be able to use the software as they see fit – even if that means closing down former open source code! The GNU camp taunts permissive licenses as being “lax” for not providing the protection that they want. The other camp points out that the GPL is a complex monster and that it is virulent in nature: Since it's very strict in a lot of areas, it's incompatible with many other licenses. This makes it complicated to mix GPL and non-GPL code and in the cases where it's legally possible, the GPL's terms will take precedence and necessarily be in effect for the whole combined work. Who's right? That totally depends on what you want to achieve. There are pros and cons to both – and in fact we're only looking at the big picture here. There's also e.g. the Apache license which is often deemed as kind of middle ground. Then you may want to consider the difference between weak (e.g. LGPL) as well as strong copyleft (GPL). Licensing is a potentially huge topic. But let's keep it simple here because the exact details are actually not necessary to understand the essence of our topic. In the next post I'll present my stance on why permissive licensing is a good thing and copyleft is more problematic than many people may think. “Permissive licensing is wrong?” – No it's not! (https://eerielinux.wordpress.com/2018/01/25/permissive-licensing-is-wrong-no-its-not-2-2/) The previous post gave a short introduction into the topic of software licenses, focusing on the GPL vs. BSD discussion. This one is basically my response to some typical arguments I've seen from people who seem to loathe permissive licensing. I'll write this in dialog style, hoping that this makes it a little lighter to read. Roundup Install OpenBSD on dedibox with full-disk encryption (https://poolp.org/posts/2018-01-29/install-openbsd-on-dedibox-with-full-disk-encryption/) TL;DR: I run several "dedibox" servers at online.net, all powered by OpenBSD. OpenBSD is not officially supported so you have to work-around. Running full-disk encrypted OpenBSD there is a piece of cake. As a bonus, my first steps within a brand new booted machine ;-) Step #0: choosing your server OpenBSD is not officially supported, I can't guarantee that this will work for you on any kind of server online.net provides, however I've been running https://poolp.org on OpenBSD there since 2008, only switching machines as they were getting a bit old and new offers came up. Currently, I'm running two SC 2016 (SATA) and one XC 2016 (SSD) boxes, all three running OpenBSD reliably ever since I installed them. Recently I've been willing to reinstall the XC one after I did some experiments that turned it into a FrankenBSD, so this was the right occasion to document how I do it for future references. I wrote an article similar to this a few years ago relying on qemu to install to the disk, since then online.net provided access to a virtual serial console accessed within the browser, making it much more convenient to install without the qemu indirection which hid the NIC devices and disks duid and required tricks. The method I currently use is a mix and adaptation from the techniques described in https://www.2f30.org/guides/openbsd-dedibox.html to boot the installer, and the technique described in https://geekyschmidt.com/2011/01/19/configuring-openbsd-softraid-fo-encryption.html to setup the crypto slice. Step #1: boot to rescue mode Step #2: boot to the installer Step #3: prepare softraid Step #4: reboot to encrypted OpenBSD system Bonus: further tightening your system enable doas disable the root account update system with syspatch add my ssh public key to my ~/.ssh/authorized_keys disable password authentication within ssh reboot so you boot on a brand new up-to-date system with latest stable kernel VOILA ! January 2018 Development Projects Update (https://www.freebsdfoundation.org/blog/january-2018-development-projects-update/) Spectre and Meltdown in FreeBSD Issues affecting most CPUs used in servers, desktops, laptops, and mobile devices are in the news. These hardware vulnerabilities, known by the code-names “Meltdown” and “Spectre”, allow malicious programs to read data to which they should not have access. This potentially includes credentials, cryptographic material, or other secrets. They were originally identified by a researcher from Google's Project Zero, and were also independently discovered by researchers and academics from Cyberus Technology, Graz University of Technology, the University of Pennsylvania, the University of Maryland, Rambus, the University of Adelaide and Data61. These vulnerabilities affect many CPU architectures supported by FreeBSD, but the 64-bit x86 family of processors from Intel and AMD are the most widely used, and are a high priority for software changes to mitigate the effects of Meltdown and Spectre. In particular, the Meltdown issue affects Intel CPUs and may be used to extract secret data from the running kernel, and therefore, is the most important issue to address. The FreeBSD Foundation collaborates with Intel, and under this relationship participated in a briefing to understand the details of these issues and plan the mitigations to be applied to the x86 architectures supported by FreeBSD. We also made arrangements to have FreeBSD's security officer join me in the briefing. It is through the generous support of the Foundation's donors that we are able to dedicate resources to focus on these issues on demand as they arise. Foundation staff member Konstantin (Kostik) Belousov is an expert on FreeBSD's Virtual Memory (VM) system as well as low-level x86 details, and is developing the x86 kernel mitigations for FreeBSD. The mitigation for Meltdown is known as Page Table Isolation (PTI). Kostik created a PTI implementation which was initially committed in mid-January and is available in the FreeBSD-CURRENT development repository. This is the same approach used by the Linux kernel to mitigate Meltdown. One of the drawbacks of the PTI mitigation is that it incurs a performance regression. Kostik recently reworked FreeBSD's use of Process-Context Identifiers (PCID) in order to regain some of the performance loss incurred by PTI. This change is also now available in FreeBSD-CURRENT. The issue known as Spectre comes in two variants, and variant 2 is the more troubling and pressing one. It may be mitigated in one of two ways: by using a technique called “retpoline” in the compiler, or by making use of a CPU feature introduced in a processor microcode update. Both options are under active development. Kostik's change to implement the CPU-based mitigation is currently in review. Unfortunately, it introduces a significant performance penalty and alternatives are preferred, if available. For most cases, the compiler-based retpoline mitigation is likely to be the chosen mitigation. Having switched to the Clang compiler for the base system and most of the ports collection some years ago, FreeBSD is well-positioned to deploy Clang-based mitigations. FreeBSD developer Dimitry Andric is spearheading the update of Clang/LLVM in FreeBSD to version 6.0 in anticipation of its official release; FreeBSD-CURRENT now includes an interim snapshot. I have been assisting with the import, particularly with respect to LLVM's lld linker, and will support the integration of retpoline. This support is expected to be merged into FreeBSD in the coming weeks. The Foundation's co-op students have also participated in the response to these vulnerabilities. Mitchell Horne developed the patch to control the PTI mitigation default setting, while Arshan Khanifar benchmarked the performance impact of the in-progress mitigation patches. In addition, Arshan and Mitchell each developed changes to FreeBSD's tool chain to support the full set of mitigations that will be applied. These mitigations will continue be tested, benchmarked, and refined in FreeBSD-CURRENT before being merged into stable branches and then being made available as updates to FreeBSD releases. Details on the timing of these merges and releases will be shared as they become available. I would like to acknowledge all of those in the FreeBSD community who have participated in FreeBSD's response to Meltdown and Spectre, for testing, reviewing, and coordinating x86 mitigations, for developing mitigations for other processor architectures and for the Bhyve hypervisor, and for working on the toolchain-based mitigations. Guides: Getting Started & Lumina Theme Submissions (https://lumina-desktop.org/guides-getting-started-lumina-themes/) I am pleased to announce the beginning of a new sub-series of blog posts for the Lumina project: Guides! The TrueOS/Lumina projects want to support our users as they use Lumina or experiment with TrueOS. To that end, we've recently set up a central repository for our users to share instructions or other “how-to” guides with each other! Project developers and contributors will also submit guides to the repository on occasion, but the overall goal is to provide a simple hub for instructions written by any Lumina or TrueOS user. This will make it easier for users to not only find a “how-to” for some procedure, but also a very easy way to “give back” to the community by writing simple instructions or more detailed guides. Guides Repository Our first guide to get the whole thing started was created by the TrueOS Linebacker (https://discourse.trueos.org/t/introducing-the-trueos-linebacker/991) (with technical assistance from our own q5sys). In this guide, Terry Tate will walk you through the steps necessary to submit new wallpaper images to the Lumina Themes collection. This procedure is fully documented with screenshots every step of the way, walking you through a simple procedure that only requires a web browser and a Github account! Guide: Lumina Themes Submissions (https://github.com/trueos/guides/blob/master/lumina-themes-submissions/readme.md) The end result of this guide was that Terry Tate was able to submit this cool new “Lunar-4K” wallpaper to the “lumina-nature” collection. TrueOS Community Guides (https://github.com/trueos/guides/tree/master) ZFS vs. OpenZFS (by Michael Dexter) (https://www.ixsystems.com/blog/zfs-vs-openzfs/) You've probably heard us say a mix of “ZFS” and “OpenZFS” and an explanation is long-overdue. Our Senior Analyst clears up what ZFS and OpenZFS refer to and how they differ. I admit that we geeks tend to get caught up in the nuts and bolts of enterprise storage and overlook the more obvious questions that users might have. You've probably noticed that this blog and the FreeNAS blog refer to “ZFS” and “OpenZFS” seemingly at random when talking about the amazing file system at the heart of FreeNAS and every storage product that iXsystems sells. I will do my best to clarify what exactly these two terms refer to. From its inception, “ZFS” has referred to the “Zettabyte File System” developed at Sun Microsystems and published under the CDDL Open Source license in 2005 as part of the OpenSolaris operating system. ZFS was revolutionary for completely decoupling the file system from specialized storage hardware and even a specific computer platform. The portable nature and advanced features of ZFS led FreeBSD, Linux, and even Apple developers to start porting ZFS to their operating systems and by 2008, FreeBSD shipped with ZFS in the 7.0 release. For the first time, ZFS empowered users of any budget with enterprise-class scalability and data integrity and management features like checksumming, compression and snapshotting, and those features remain unrivaled at any price to this day. On any ZFS platform, administrators use the zpool and zfs utilities to configure and manage their storage devices and file systems respectively. Both commands employ a user-friendly syntax such as‘zfs create mypool/mydataset' and I welcome you to watch the appropriately-titled webinar “Why we love ZFS & you should too” or try a completely-graphical ZFS experience with FreeNAS. Yes, ZFS is really as good as people say it is. After enjoying nearly a decade of refinement by a growing group of developers around the world, ZFS became the property of database vendor Oracle, which ceased public development of both ZFS and OpenSolaris in 2010. Disappointed but undeterred, a group of OpenSolaris users and developers forked the last public release of OpenSolaris as the Illumos project. To this day, Illumos represents the official upstream home of the Open Source OpenSolaris technologies, including ZFS. The Illumos project enjoys healthy vendor and user participation but the portable nature and compelling features of ZFS soon produced far more ZFS users than Illumos users around the world. While most if not all users of Illumos and its derivatives are ZFS users, the majority of ZFS users are not Illumos users, thanks significantly in part to FreeNAS which uses the FreeBSD operating system. This imbalance plus several successful ZFS Day events led ZFS co-founder Matt Ahrens and a group of ZFS developers to announce the OpenZFS project, which would remain a part of the Illumos code base but would be free to coordinate development efforts and events around their favorite file system. ZFS Day has grown into the two-day OpenZFS Developer Summit and is stronger than ever, a testament to the passion and dedication of the OpenZFS community. Oracle has steadily continued to develop its own proprietary branch of ZFS and Matt Ahrens points out that over 50% of the original OpenSolaris ZFS code has been replaced in OpenZFS with community contributions. This means that there are, sadly, two politically and technologically-incompatible branches of “ZFS” but fortunately, OpenZFS is orders of magnitude more popular thanks to its open nature. The two projects should be referred to as “Oracle ZFS” and “OpenZFS” to distinguish them as development efforts, but the user still types the ‘zfs' command, which on FreeBSD relies on the ‘zfs.ko' kernel module. My impression is that the terms of the CDDL license under which the OpenZFS branch of ZFS is published protects its users from any patent and trademark risks. Hopefully, this all helps you distinguish the OpenZFS project from the ZFS technology. Beastie Bits Explaining Shell (https://explainshell.com/) OPNsense® 18.1 Released (https://opnsense.org/opnsense-18-1-released/) “SSH Mastery 2/e” copyedits back (https://blather.michaelwlucas.com/archives/3104) Sponsoring a Scam (https://blather.michaelwlucas.com/archives/3106) Thursday, February 8, 2018 - Come to Netflix to talk about FreeBSD (https://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/246623825/) BSD User Group meeting in Stockholm: March 22, 17:30 - 21:00 (https://www.meetup.com/BSD-Users-Stockholm/events/247552279/) FreeBSD Flavoured talks from Linux.conf.au: You can't unit test C, right? (https://www.youtube.com/watch?v=z-uWt5wVVkU) and A Brief History of I/O (https://www.youtube.com/watch?v=qAhZEI_6lbc) EuroBSDcon 2018 website is up (https://2018.eurobsdcon.org/) Full day bhyvecon Tokyo, Japan, March 9, 2018 (http://bhyvecon.org/) *** Feedback/Questions Thomas - freebsd installer improvements (http://dpaste.com/3G2F7RC#wrap) Mohammad - FreeBSD 11 installation from a read only rescue disk (http://dpaste.com/0HGK3FQ#wrap) Stan - Follow up on guide you covered (http://dpaste.com/2S169SH#wrap) Jalal - couple questions (http://dpaste.com/35N8QXP#wrap)
We cover an interview about Unix Architecture Evolution, another vBSDcon trip report, how to teach an old Unix about backspace, new NUMA support coming to FreeBSD, and stack pointer checking in OpenBSD. This episode was brought to you by Headlines Unix Architecture Evolution from the 1970 PDP-7 to the 2017 FreeBSD (https://fosdem.org/2018/interviews/diomidis-spinellis/) Q: Could you briefly introduce yourself? I'm a professor of software engineering, a programmer at heart, and a technology author. Currently I'm also the editor in chief of the IEEE Software magazine. I recently published the book Effective Debugging, where I detail 66 ways to debug software and systems. Q: What will your talk be about, exactly? I will describe how the architecture of the Unix operating system evolved over the past half century, starting from an unnamed system written in PDP-7 assembly language and ending with a modern FreeBSD system. My talk is based, first, on a GitHub repository where I tried to record the system's history from 1970 until today and, second, on the evolution of documented facilities (user commands, system calls, library functions) across revisions. I will thus present the early system's defining architectural features (layering, system calls, devices as files, an interpreter, and process management) and the important ones that followed in subsequent releases: the tree directory structure, user contributed code, I/O redirection, the shell as a user program, groups, pipes, scripting, and little languages. Q: Why this topic? Unix stands out as a major engineering breakthrough due to its exemplary design, its numerous technical contributions, its impact, its development model, and its widespread use. Furthermore, the design of the Unix programming environment has been characterized as one offering unusual simplicity, power, and elegance. Consequently, there are many lessons that we can learn by studying the evolution of the Unix architecture, which we can apply to the design of new systems. I often see modern systems that suffer from a bloat of architectural features and a lack of clear form on which functionality can be built. I believe that many of the modern Unix architecture defining features are excellent examples of what we should strive toward as system architects. Q: What do you hope to accomplish by giving this talk? What do you expect? I'd like FOSDEM attendees to leave the talk with their mind full with architectural features of timeless quality. I want them to realize that architectural elegance isn't derived by piling design patterns and does not need to be expensive in terms of resources. Rather, beautiful architecture can be achieved on an extremely modest scale. Furthermore, I want attendees to appreciate the importance of adopting flexible conventions rather than rigid enforcement mechanisms. Finally, I want to demonstrate through examples that the open source culture was part of Unix from its earliest days. Q: What are the most significant milestones in the development of Unix? The architectural development of Unix follows a path of continuous evolution, albeit at a slowing pace, so I don't see here the most important milestones. I would however define as significant milestones two key changes in the way Unix was developed. The first occurred in the late 1970s when significant activity shifted from a closely-knit team of researchers at the AT&T Bell Labs to the Computer Science Research Group in the University of California at Berkeley. This opened the system to academic contributions and growth through competitive research funding. The second took place in the late 1980s and the 1990s when Berkeley open-sourced the the code it had developed (by that time a large percentage of the system) and enthusiasts built on it to create complete open source operating system distributions: 386BSD, and then FreeBSD, NetBSD, OpenBSD, and others. Q: In which areas has the development of Unix stalled? The data I will show demonstrate that there were in the past some long periods where the number of C library functions and system calls remained mostly stable. Nowadays there is significant growth in the number of all documented facilities with the exception of file formats. I'm looking forward to a discussion regarding the meaning of these growth patterns in the Q&A session after the talk. Q: What are the core features that still link the 1970 PDP-7 system to the latest FreeBSD 11.1 release, almost half a century apart? Over the past half-century the Unix system has grown by four orders of magnitude from a few thousand lines of code to many millions. Nevertheless, looking at a 1970s architecture diagram and a current one reveals that the initial architectural blocks are still with us today. Furthermore, most system calls, user programs, and C library functions of that era have survived until today with essentially similar functionality. I've even found in modern FreeBSD some lines of code that have survived unchanged for 40 years. Q: Can we still add innovative changes to operating systems like FreeBSD without breaking the ‘Unix philosophy'? Will there be a moment where FreeBSD isn't recognizable anymore as a descendant of the 1970 PDP-7 system? There's a saying that “form liberates”. So having available a time-tested form for developing operating system functionality allows you to innovate in areas that matter rather than reinventing the wheel. Such concepts include having commands act as a filter, providing manual pages with a consistent structure, supplying build information in the form of a Makefile, installing files in a well-defined directory hierarchy, implementing filesystems with an standardized object-oriented interface, and packaging reusable functions as a library. Within this framework there's ample space for both incremental additions (think of jq, the JSON query command) and radical innovations (consider the Solaris-derived ZFS and dtrace functionality). For this reason I think that BSD and Linux systems will always be recognizable as direct or intellectual descendants of the 1970s Research Unix editions. Q: Have you enjoyed previous FOSDEM editions? Immensely! As an academic I need to attend many scientific conferences and meetings in order to present research results and interact with colleagues. This means too much time spent traveling and away from home, and a limited number of conferences I'm in the end able to attend. Nevertheless, attending FOSDEM is an easy decision due to the world-changing nature of its theme, the breadth of the topics presented, the participants' enthusiasm and energy, as well as the exemplary, very efficient conference organization. Another vBSDCon trip report we just found (https://www.weaponizedawesome.com/blog/?cat=53) We just got tipped about another trip report from vBSDCon, this time from one of the first time speakers: W. Dean Freeman Recently I had the honor of co-presenting on the internals of FreeBSD's Kernel RNG with John-Mark Gurney at the 3rd biennial vBSDCon, hosted in Reston, VA hosted by Verisign. I've been in and out of the FreeBSD community for about 20 years. As I've mentioned on here before, my first Unix encounter was FreeBSD 2.2.8 when I was in the 7th or 8th grade. However, for all that time I've never managed to get out to any of the cons. I've been to one or two BUG meetings and I've met some folks from IRC before, but nothing like this. A BSD conference is a very different experience than anything else out there. You have to try it, it is the only way to truly understand it. I'd also not had to do a stand-up presentation really since college before this. So, my first BSD con and my first time presenting rolled into one made for an interesting experience. See, he didn't say terrifying. It went very well. You should totally submit a talk for the next conference, even if it is your first. That said, it was amazing and invigorating experience. I got to meet a few big names in the FreeBSD community, discuss projects, ideas for FreeBSD, etc. I did seem to spend an unusual amount of time talking about FIPS and Common Criteria with folks, but to me that's a good sign and indicative that there is interest in working to close gaps between FreeBSD and the current requirements so that we can start getting FreeBSD and more BSD-based products into the government and start whittling away the domination of Linux (especially since Oracle has cut Solaris, SPARC and the ZFS storage appliance business units). There is nothing that can match the high bandwidth interchange of ideas in person. The internet has made all kinds of communication possible, and we use it all the time, but every once in a while, getting together in person is hugely valuable. Dean then went on to list some of the talks he found most valuable, including DTrace, Capsicum, bhyve, *BSD security tools, and Paul Vixie's talk about gets() I think the talk that really had the biggest impact on me, however, was Kyle Kneisl's talk on BSD community dynamics. One of the key points he asked was whether the things that drew us to the BSD community in the first place would be able to happen today. Obviously, I'm not a 12 or 13 year old kid anymore, but it really got me thinking. That, combined with getting face time with people I'd previously only known as screen names has recently drawn me back into participating in IRC and rejoining mailing lists (wdf on freenode. be on the lookout!) Then Dean covered some thoughts on his own talk: JMG and my talk seems to have been well received, with people paying lots of attention. I don't know what a typical number of questions is for one of these things, but on day one there weren't that many questions. We got about 5 during our question time and spent most of the rest of the day fielding questions from interested attendees. Getting a “great talk!” from GNN after coming down from the stage was probably one of the major highlights for me. I remember my first solo talk, and GNN asking the right question in the middle to get me to explain a part of it I had missed. It was very helpful. I think key to the interest in our presentation was that JMG did a good job framing a very complicated topic's importance in terms everyone could understand. It also helped that we got to drop some serious truth bombs. Final Thoughts: I met a lot of folks in person for the first time, and met some people I'd never known online before. It was a great community and I'm glad I got a chance to expand my network. Verisign were excellent hosts and they took good care of both speakers (covering airfare, rooms, etc.) and also conference attendees at large. The dinners that they hosted were quite good as well. I'm definitely interested in attending vBSDCon again and now that I've had a taste of meeting IRL with the community on scale of more than a handful, I have every intention of finally making it to BSDCan next year (I'd said it in 2017, but then moved to Texas for a new job and it wasn't going to be practical). This year for sure, though! Teaching an Almost 40-year Old UNIX about Backspace (https://virtuallyfun.com/2018/01/17/teaching_an_almost_40-year_old_unix_about_backspace/) Introduction I have been messing with the UNIX® operating system, Seventh Edition (commonly known as UNIX V7 or just V7) for a while now. V7 dates from 1979, so it's about 40 years old at this point. The last post was on V7/x86, but since I've run into various issues with it, I moved on to a proper installation of V7 on SIMH. The Internet has some really good resources on installing V7 in SIMH. Thus, I set out on my own journey on installing and using V7 a while ago, but that was remarkably uneventful. One convenience that I have been dearly missing since the switch from V7/x86 is a functioning backspace key. There seem to be multiple different definitions of backspace: BS, as in ASCII character 8 (010, 0x08, also represented as ^H), and DEL, as in ASCII character 127 (0177, 0x7F, also represented as ^?). V7 does not accept either for input by default. Instead, # is used as the erase character and @ is used as the kill character. These defaults have been there since UNIX V1. In fact, they have been “there” since Multics, where they got chosen seemingly arbitrarily. The erase character erases the character before it. The kill character kills (deletes) the whole line. For example, “ba##gooo#d” would be interpreted as “good” and “bad line@good line” would be interpreted as “good line”. There is some debate on whether BS or DEL is the correct character for terminals to send when the user presses the backspace key. However, most programs have settled on DEL today. tmux forces DEL, even if the terminal emulator sends BS, so simply changing my terminal to send BS was not an option. The change from the defaults outlined here to today's modern-day defaults occurred between 4.1BSD and 4.2BSD. enf on Hacker News has written a nice overview of the various conventions Getting the Diff For future generations as well as myself when I inevitably majorly break this installation of V7, I wanted to make a diff. However, my V7 is installed in SIMH. I am not a very intelligent man, I didn't keep backup copies of the files I'd changed. Getting data out of this emulated machine is an exercise in frustration. In the end, I printed everything on screen using cat(1) and copied that out. Then I performed a manual diff against the original source code tree because tabs got converted to spaces in the process. Then I applied the changes to clean copies that did have the tabs. And finally, I actually invoked diff(1). Closing Thoughts Figuring all this out took me a few days. Penetrating how the system is put together was surprisingly fairly hard at first, but then the difficulty curve eased up. It was an interesting exercise in some kind of “reverse engineering” and I definitely learned something about tty handling. I was, however, not pleased with using ed(1), even if I do know the basics. vi(1) is a blessing that I did not appreciate enough until recently. Had I also been unable to access recursive grep(1) on my host and scroll through the code, I would've probably given up. Writing UNIX under those kinds of editing conditions is an amazing feat. I have nothing but the greatest respect for software developers of those days. News Roundup New NUMA support coming to FreeBSD CURRENT (https://lists.freebsd.org/pipermail/freebsd-current/2018-January/068145.html) Hello folks, I am working on merging improved NUMA support with policy implemented by cpuset(2) over the next week. This work has been supported by Dell/EMC's Isilon product division and Netflix. You can see some discussion of these changes here: https://reviews.freebsd.org/D13403 https://reviews.freebsd.org/D13289 https://reviews.freebsd.org/D13545 The work has been done in user/jeff/numa if you want to look at svn history or experiment with the branch. It has been tested by Peter Holm on i386 and amd64 and it has been verified to work on arm at various points. We are working towards compatibility with libnuma and linux mbind. These commits will bring in improved support for NUMA in the kernel. There are new domain specific allocation functions available to kernel for UMA, malloc, kmem, and vmpage*. busdmamem consumers will automatically be placed in the correct domain, bringing automatic improvements to some device performance. cpuset will be able to constrains processes, groups of processes, jails, etc. to subsets of the system memory domains, just as it can with sets of cpus. It can set default policy for any of the above. Threads can use cpusets to set policy that specifies a subset of their visible domains. Available policies are first-touch (local in linux terms), round-robin (similar to linux interleave), and preferred. For now, the default is round-robin. You can achieve a fixed domain policy by using round-robin with a bitmask of a single domain. As the scheduler and VM become more sophisticated we may switch the default to first-touch as linux does. Currently these features are enabled with VMNUMAALLOC and MAXMEMDOM. It will eventually be NUMA/MAXMEMDOM to match SMP/MAXCPU. The current NUMA syscalls and VMNUMAALLOC code was 'experimental' and will be deprecated. numactl will continue to be supported although cpuset should be preferred going forward as it supports the full feature set of the new API. Thank you for your patience as I deal with the inevitable fallout of such sweeping changes. If you do have bugs, please file them in bugzilla, or reach out to me directly. I don't always have time to catch up on all of my mailing list mail and regretfully things slip through the cracks when they are not addressed directly to me. Thanks, Jeff Stack pointer checking – OpenBSD (https://marc.info/?l=openbsd-tech&m=151572838911297&w=2) Stefan (stefan@) and I have been working for a few months on this diff, with help from a few others. At every trap and system call, it checks if the stack-pointer is on a page that is marked MAPSTACK. execve() is changed to create such mappings for the process stack. Also, libpthread is taught the new MAPSTACK flag to use with mmap(). There is no corresponding system call which can set MAP_FLAG on an existing page, you can only set the flag by mapping new memory into place. That is a piece of the security model. The purpose of this change is to twart stack pivots, which apparently have gained some popularity in JIT ROP attacks. It makes it difficult to place the ROP stack in regular data memory, and then perform a system call from it. Workarounds are cumbersome, increasing the need for far more gadgetry. But also the trap case -- if any memory experiences a demand page fault, the same check will occur and potentially also kill the process. We have experimented a little with performing this check during device interrupts, but there are some locking concerns and performance may then become a concern. It'll be best to gain experience from handle of syncronous trap cases first. chrome and other applications I use run fine! I'm asking for some feedback to discover what ports this breaks, we'd like to know. Those would be ports which try to (unconventionally) create their stacks in malloc()'d memory or inside another Data structure. Most of them are probably easily fixed ... Qt 5.9 on FreeBSD (https://euroquis.nl/bobulate/?p=1768) Tobias and Raphael have spent the past month or so hammering on the Qt 5.9 branch, which has (finally!) landed in the official FreeBSD ports tree. This brings FreeBSD back up-to-date with current Qt releases and, more importantly, up-to-date with the Qt release KDE software is increasingly expecting. With Qt 5.9, the Elisa music player works, for instance (where it has run-time errors with Qt 5.7, even if it compiles). The KDE-FreeBSD CI system has had Qt 5.9 for some time already, but that was hand-compiled and jimmied into the system, rather than being a “proper” ports build. The new Qt version uses a new build system, which is one of the things that really slowed us down from a packaging perspective. Some modules have been reshuffled in the process. Some applications depending on Qt internal-private headers have been fixed along the way. The Telegram desktop client continues to be a pain in the butt that way. Following on from Qt 5.9 there has been some work in getting ready for Clang 6 support; in general the KDE and Qt stack is clean and modern C++, so it's more infrastructural tweaks than fixing code. Outside of our silo, I still see lots of wonky C++ code being fixed and plenty of confusion between pointers and integers and strings and chars and .. ugh. Speaking of ugh, I'm still planning to clean up Qt4 on ARM aarch64 for FreeBSD; this boils down to stealing suitable qatomic implementations from Arch Linux. For regular users of Qt applications on FreeBSD, there should be few to no changes required outside the regular upgrade cycle. For KDE Plasma users, note that development of the ports has changed branches; as we get closer to actually landing modern KDE bits, things have been renamed and reshuffled and mulled over so often that the old plasma5 branch wasn't really right anymore. The kde5-import branch is where it's at nowadays, and the instructions are the same: the x11/kde5 metaport will give you all the KDE Frameworks 5, KDE Plasma Desktop and modern KDE Applications you need. Adding IPv6 to an Nginx website on FreeBSD / FreshPorts (https://dan.langille.org/2018/01/13/adding-ipv6-to-an-nginx-website-on-freebsd-freshports/) FreshPorts recently moved to an IPv6-capable server but until today, that capability has not been utilized. There were a number of things I had to configure, but this will not necessarily be an exhaustive list for you to follow. Some steps might be missing, and it might not apply to your situation. All of this took about 3 hours. We are using: FreeBSD 11.1 Bind 9.9.11 nginx 1.12.2 Fallout I expect some monitoring fallout from this change. I suspect some of my monitoring assumes IP4 and now that IPv6 is available, I need to monitor both IP addresses. ZFS on TrueOS: Why We Love OpenZFS (https://www.trueos.org/blog/zfs-trueos-love-openzfs/) TrueOS was the first desktop operating system to fully implement the OpenZFS (Zettabyte File System or ZFS for short) enterprise file system in a stable production environment. To fully understand why we love ZFS, we will look back to the early days of TrueOS (formerly PC-BSD). The development team had been using the UFS file system in TrueOS because of its solid track record with FreeBSD-based computer systems and its ability to check file consistency with the built-in check utility fsck. However, as computing demands increased, problems began to surface. Slow fsck file verification on large file systems, slow replication speeds, and inconsistency in data integrity while using UFS logging / journaling began to hinder users. It quickly became apparent that TrueOS users would need a file system that scales with evolving enterprise storage needs, offers the best data protection, and works just as well on a hobbyist system or desktop computer. Kris Moore, the founder of the TrueOS project, first heard about OpenZFS in 2007 from chatter on the FreeBSD mailing lists. In 2008, the TrueOS development team was thrilled to learn that the FreeBSD Project had ported ZFS. At the time, ZFS was still unproven as a graphical desktop solution, but Kris saw a perfect opportunity to offer ZFS as a cutting-edge file system option in the TrueOS installer, allowing the TrueOS project to act as an indicator of how OpenZFS would fair in real-world production use. The team was blown away by the reception and quality of OpenZFS on FreeBSD-based systems. By its nature, ZFS is a copy-on-write (CoW) file system that won't move a block of data until it both writes the data and verifies its integrity. This is very different from most other file systems in use today. ZFS is able to assure that data stays consistent between writes by automatically comparing write checksums, which mitigates bit rot. ZFS also comes with native RaidZ functionality that allows for enterprise data management and redundancy without the need for expensive traditional RAID cards. ZFS snapshots allow for system configuration backups in a split-second. You read that right. TrueOS can backup or restore snapshots in less than a second using the ZFS file system. Given these advantages, the TrueOS team decided to use ZFS as its exclusive file system starting in 2013, and we haven't looked back since. ZFS offers TrueOS users the stable workstation experience they want, while simultaneously scaling to meet the increasing demands of the enterprise storage market. TrueOS users are frequently commenting on how easy it is to use ZFS snapshots with our built-in snapshot utility. This allows users the freedom to experiment with their system knowing they can restore it in seconds if anything goes wrong. If you haven't had a chance to try ZFS with TrueOS, browse to our download page and make sure to grab a copy of TrueOS. You'll be blown away by the ease of use, data protection functionality, and incredible flexibility of RaidZ. Beastie Bits Source Code Podcast Interview with Michael W Lucas (https://blather.michaelwlucas.com/archives/3099) Operating System of the Year 2017: NetBSD Third place (https://w3techs.com/blog/entry/web_technologies_of_the_year_2017) OPNsense 18.1-RC1 released (https://opnsense.org/opnsense-18-1-rc1-released/) Personal OpenBSD Wiki Notes (https://balu-wiki.readthedocs.io/en/latest/security/openbsd.html) BSD section can use some contribution (https://guide.freecodecamp.org/bsd-os/) The Third Research Edition Unix Programmer's Manual (now available in PDF) (https://github.com/dspinellis/unix-v3man) Feedback/Questions Alex - my first freebsd bug (http://dpaste.com/3DSV7BC#wrap) John - Suggested Speakers (http://dpaste.com/2QFR4MT#wrap) Todd - Two questions (http://dpaste.com/2FQ450Q#wrap) Matthew - CentOS to FreeBSD (http://dpaste.com/3KA29E0#wrap) Brian - Brian - openbsd 6.2 and enlightenment .17 (http://dpaste.com/24DYF1J#wrap) ***
We review the information about Spectre & Meltdown thus far, we look at NetBSD memory sanitizer progress, Postgres on ZFS & show you a bit about NomadBSD. This episode was brought to you by Headlines Meltdown Spectre Official Site (https://meltdownattack.com/) Kernel-memory-leaking Intel processor design flaw forces Linux, Windows redesign (https://www.theregister.co.uk/2018/01/02/intel_cpu_design_flaw/) Intel's official response (https://newsroom.intel.com/news/intel-responds-to-security-research-findings/) The Register mocks intels response with pithy annotations (https://www.theregister.co.uk/2018/01/04/intel_meltdown_spectre_bugs_the_registers_annotations/) Intel's Analysis PDF (https://newsroom.intel.com/wp-content/uploads/sites/11/2018/01/Intel-Analysis-of-Speculative-Execution-Side-Channels.pdf) XKCD (https://xkcd.com/1938/) Response from FreeBSD (https://lists.freebsd.org/pipermail/freebsd-security/2018-January/009719.html) FreeBSD's patch WIP (https://reviews.freebsd.org/D13797) Why Raspberry Pi isn't vulnerable to Spectre or Meltdown (https://www.raspberrypi.org/blog/why-raspberry-pi-isnt-vulnerable-to-spectre-or-meltdown/) Xen mitigation patches (https://lists.xenproject.org/archives/html/xen-devel/2018-01/msg00110.html) Overview of affected FreeBSD Platforms/Architectures (https://wiki.freebsd.org/SpeculativeExecutionVulnerabilities) Groff's response (https://twitter.com/GroffTheBSDGoat/status/949372300368867328) ##### We'll cover OpenBSD, NetBSD, and DragonflyBSD's responses in next weeks episode. *** ###The LLVM Memory Sanitizer support work in progress (https://blog.netbsd.org/tnf/entry/the_llvm_memory_sanitizer_support) > In the past 31 days, I've managed to get the core functionality of MSan to work. This is an uninitialized memory usage detector. MSan is a special sanitizer because it requires knowledge of every entry to the basesystem library and every entry to the kernel through public interfaces. This is mandatory in order to mark memory regions as initialized. Most of the work has been done directly for MSan. However, part of the work helped generic features in compiler-rt. Sanitizers > Changes in the sanitizer are listed below in chronological order. Almost all of the changes mentioned here landed upstream. A few small patches were reverted due to breaking non-NetBSD hosts and are rescheduled for further investigation. I maintain these patches locally and have moved on for now to work on the remaining features. NetBSD syscall hooks > I wrote a large patch (815kb!) adding support for NetBSD syscall hooks for use with sanitizers. NetBSD ioctl(2) hooks > Similar to the syscall hooks, there is need to handle every ioctl(2) call. I've created the needed patch, this time shorter - for less than 300kb. New patches still pending for upstream review > There are two corrections that I've created, and they are still pending upstream for review: Add MSan interceptor for fstat(2)](https://reviews.llvm.org/D41637) Correct the setitimer interceptor on NetBSD)](https://reviews.llvm.org/D41502) > I've got a few more local patches that require cleanup before submitting to review. NetBSD basesystem corrections Sanitizers in Go The MSan state as of today Solaris support in sanitizers > I've helped the Solaris team add basic support for Sanitizers (ASan, UBsan). This does not help NetBSD directly, however indirectly it improves the overall support for non-Linux hosts and helps to catch more Linuxisms in the code. Plan for the next milestone > I plan to continue the work on MSan and correct sanitizing of the NetBSD basesystem utilities. This mandates me to iterate over the basesystem libraries implementing the missing interceptors and correcting the current support of the existing ones. My milestone is to build all src/bin programs against Memory Sanitizer and when possible execute them cleanly. This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: http://netbsd.org/donations/#how-to-donate (http://netbsd.org/donations/#how-to-donate) *** ##News Roundup ###MWL's 2017 Wrap-Up (https://blather.michaelwlucas.com/archives/3078) > The obvious place to start is my 2016 wrap-up post](https://blather.michaelwlucas.com/archives/2822), where I listed goals for 2017. As usual, these goals were wildly delusional. > The short answer is, my iron was back up to normal. My writing speed wasn't, though. I'd lost too much general health, and needed hard exercise to recover it. Yes, writing requires physical endurance. Maintaining that level of concentration for several hours a day demands a certain level of blood flow to the brain. I could have faked it in a day job, but when self-employed as an artist? Not so much. > Then there's travel. I did my usual BSDCan trip, plus two educational trips to Lincoln City, Oregon. The current political mayhem convinced me that if I wanted to hit EuroBSDCon any time in the next few years, I should do it in the very near future. So I went to Paris, where I promptly got pickpocketed. (Thankfully, they didn't get my passport.) I was actively writing the third edition of Absolute FreeBSD, so I visited BSDCam in Cambridge to get the latest information and a sense of where FreeBSD was going. I also did weekends at Kansas LinuxFest (because they asked and paid for my trip) and Penguicon. > (Because people will ask: why EuroBSDCon and not AsiaBSDCon? A six-hour transatlantic flight requires that I take a substantial dose of heavy-grade tranquilizers. I'm incapable of making intelligent decisions while on those drugs, or for several hours afterward. They don't last long enough for twelve-hour flight to Japan, so I need to be accompanied by someone qualified to tell me when I need to take the next dose partway through the flight. This isn't a predetermined time that I can set an alarm for; it depends on how the clonazepam affects me at those altitudes. A drug overdose while flying over the North Pole would be bad. When I can arrange that qualified companion, I'll make the trip.) > I need most of the preceding week to prepare for long trips. I need the following week to recover from time shifts and general exhaustion. Additionally, I have to hoard people juice for a few weeks beforehand so I can deal with folks during these expeditions. Travel disrupts my dojo time as well, which impacts my health. > Taken as a whole: I didn't get nearly as much done as I hoped. I wrote more stories, but Kris Rusch bludgeoned me into submitting them to trad markets. (The woman is a brute, I tell you. Cross her at your peril.) Among my 2017 titles, my fiction outsold the tech books. No, not Prohibition Orcs–all four of the people who buy those love them, but the sales tell me I've done something wrong with those tales. My cozy mystery git commit murder outsold Relayd and Httpd Mastery. But what outdid them both, as well as most of my older books? What title utterly dominated my sales for the last quarter of the year? It was of course, my open source software political satire disguised as porn Savaged by Systemd: an Erotic Unix Encounter. (https://www.michaelwarrenlucas.com/index.php/romance#sbs) > I can't believe I just wrote that paragraph. The good news is, once I recovered from EuroBSDCon, my writing got better. I finished Absolute FreeBSD, 3rd edition and submitted it to the publisher. I wrote the second edition of SSH Mastery (no link, because you can't order it yet.) I'm plowing through git sync murder, the sequel to git commit murder. I don't get to see the new Star Wars movie until I finish GSM, so hopefully that'll be this month. All in all, I wrote 480,200 words in 2017. Most of that was after September. It's annoyingly close to breaking half a million, but after 2016's scandalous 195,700, I'll take it. *** ###PG Phriday: Postgres on ZFS (https://blog.2ndquadrant.com/pg-phriday-postgres-zfs/) > ZFS is a filesystem originally created by Sun Microsystems, and has been available for BSD over a decade. While Postgres will run just fine on BSD, most Postgres installations are historically Linux-based systems. ZFS on Linux has had much more of a rocky road to integration due to perceived license incompatibilities. > As a consequence, administrators were reluctant or outright refused to run ZFS on their Linux clusters. It wasn't until OpenZFS was introduced in 2013 that this slowly began to change. These days, ZFS and Linux are starting to become more integrated, and Canonical of Ubuntu fame even announced direct support for ZFS in their 16.04 LTS release. > So how can a relatively obscure filesystem designed by a now-defunct hardware and software company help Postgres? Let's find out! Eddie waited til he finished high school > Old server hardware is dirt cheap these days, and make for a perfect lab for testing suspicious configurations. This is the server we'll be using for these tests for those following along at home, or want some point of reference: Dell R710 x2 Intel X5660 CPUs, for up to 24 threads 64GB RAM x4 1TB 7200RPM SATA HDDs H200 RAID card configured for Host Bus Adapter (HBA) mode 250GB Samsung 850 EVO SSD > The H200 is particularly important, as ZFS acts as its own RAID system. It also has its own checksumming and other algorithms that don't like RAID cards getting in the way. As such, we put the card itself in a mode that facilitates this use case. > Due to that, we lose out on any battery-backed write cache the RAID card might offer. To make up for it, it's fairly common to use an SSD or other persistent fast storage to act both as a write cache, and a read cache. This also transforms our HDDs into hybrid storage automatically, which is a huge performance boost on a budget. She had a guitar and she taught him some chords > First things first: we need a filesystem. This hardware has four 1TB HDDs, and a 250GB SSD. To keep this article from being too long, we've already placed GPT partition tables on all the HDDs, and split the SSD into 50GB for the OS, 32GB for the write cache, and 150GB for the read cache. A more robust setup would probably use separate SSDs or a mirrored pair for these, but labs are fair game. They moved into a place they both could afford > Let's start by getting a performance baseline for the hardware. We might expect peak performance at 12 or 24 threads because the server has 12 real CPUs and 24 threads, but query throughput actually topped out at concurrent 32 processes. We can scratch our heads over this later, for now, we can consider it the maximum capabilities of this hardware. Here's a small sample: ``` $> pgbench -S -j 32 -c 32 -M prepared -T 20 pgbench ... tps = 264661.135288 (including connections establishing) tps = 264849.345595 (excluding connections establishing) ``` So far, this is pretty standard behavior. 260k prepared queries per second is great read performance, but this is supposed to be a filesystem demonstration. Let's get ZFS involved. + The papers said Ed always played from the heart Let's repeat that same test with writes enabled. Once that happens, filesystem syncs, dirty pages, WAL overhead, and other things should drastically reduce overall throughput. That's an expected result, but how much are we looking at, here? ``` $> pgbench -j 32 -c 32 -M prepared -T 10 pgbench ... tps = 6153.877658 (including connections establishing) tps = 6162.392166 (excluding connections establishing) ``` SSD cache or not, storage overhead is a painful reality. Still, 6000 TPS with writes enabled is a great result for this hardware. Or is it? Can we actually do better? Consider the Postgres fullpagewrites parameter. Tomas Vondra has written about it in the past as a necessity to prevent WAL corruption due to partial writes. The WAL is both streaming replication and crash recovery, so its integrity is of utmost importance. As a result, this is one parameter almost everyone should leave alone. ZFS is Copy on Write (CoW). As a result, it's not possible to have a torn page because a page can't be partially written without reverting to the previous copy. This means we can actually turn off fullpagewrites in the Postgres config. The results are some fairly startling performance gains: $> pgbench -j 32 -c 32 -M prepared -T 10 pgbench tps = 10325.200812 (including connections establishing) tps = 10336.807218 (excluding connections establishing) That's nearly a 70% improvement. Due to write amplification caused by full page writes, Postgres produced 1.2GB of WAL files during a 1-minute pgbench test, but only 160MB with full page writes disabled. To be fair, a 32-thread pgbench write test is extremely abusive and certainly not a typical usage scenario. However, ZFS just ensured our storage a much lower write load by altering one single parameter. That means the capabilities of the hardware have also been extended to higher write workloads as IO bandwidth is not being consumed by WAL traffic. + They both met movie stars, partied and mingled Astute readers may have noticed we didn't change the default ZFS block size from 128k to align with the Postgres default of 8kb. As it turns out, the 128kb blocks allow ZFS to better combine some of those 8kb Postgres pages to save space. That will allow our measly 2TB to go a lot further than is otherwise possible. Please note that this is not de-duplication, but simple lz4 compression, which is nearly real-time in terms of CPU overhead. De-duplication on ZFS is currently an uncertain bizzaro universe populated with misshapen horrors crawling along a broken landscape. It's a world of extreme memory overhead for de-duplication tables, and potential lost data due to inherent conflicts with the CoW underpinnings. Please don't use it, let anyone else use it, or even think about using it, ever. + They made a record and it went in the chart We're still not done. One important aspect of ZFS as a CoW filesystem, is that it has integrated snapshots. Consider the scenario where a dev is connected to the wrong system and drops what they think is a table in a QA environment. It turns out they were in the wrong terminal and just erased a critical production table, and now everyone is frantic. + The future was wide open It's difficult to discount an immediately observable reduction in write overhead. Snapshots have a multitude of accepted and potential use cases, as well. In addition to online low-overhead compression, and the hybrid cache layer, ZFS boasts a plethora of features we didn't explore. Built-in checksums with integrated self-healing suggest it isn't entirely necessary to re-initialize an existing Postgres instance to enable checksums. The filesystem itself ensures checksums are validated and correct, especially if we have more than one drive resource in our pool. It even goes the extra mile and actively corrects inconsistencies when encountered. I immediately discounted ZFS back in 2012 because the company I worked for at the time was a pure Linux shop. ZFS was only available using the FUSE driver back then, meaning ZFS only worked through userspace with no real kernel integration. It was fun to tinker with, but nobody sane would use that on a production server of any description. Things have changed quite drastically since then. I've stopped waiting for btrfs to become viable, and ZFS has probably taken the throne away from XFS as my filesystem of choice. Future editions of the Postgres High Availability Cookbook will reflect this as well. Postgres MVCC and ZFS CoW seem made for each other. I'm curious to see what will transpire over the next few years now that ZFS has reached mainstream acceptance in at least one major Linux distribution. NomadBSD (https://github.com/mrclksr/NomadBSD) About NomadBSD is a live system for flash drives, based on FreeBSD. Screenshots http://freeshell.de/~mk/download/nomadbsd-ss1.png http://freeshell.de/~mk/download/nomadbsd-ss2.png Requirements for building the image A recent FreeBSD system Requirements for running NomadBSD A 4GB (or more) flash drive A System capable running FreeBSD 11.1 (amd64) Building the image ~~ csh # make image ~~ Writing the image to an USB memory stick ~~ csh # dd if=nomadbsd.img of=/dev/da0 bs=10240 conv=sync ~~ Resize filesystem to use the entire USB memory Boot NomadBSD into single user mode, and execute: ~~ # gpart delete -i 2 da0s1 # gpart resize -i 1 da0 # gpart commit da0s1 ~~ Determine the partition size in megabytes using fdisk da0 and calculate the remaining size of da0s1a: = - . ~~ # gpart resize -i 1 -s M da0s1 # gpart add -t freebsd-swap -i 2 da0s1 # glabel label NomadBSDsw da0s1b # service growfs onestart # reboot ~~ FreeBSD forum thread (https://forums.freebsd.org/threads/63888/) A short screen capture video of the NomadBSD system running in VirtualBox (https://freeshell.de/~mk/download/nomad_capture.mp4) *** ##Beastie Bits Coolpkg, a package manager inspired by Nix for OpenBSD (https://github.com/andrewchambers/coolpkg) zrepl - ZFS replication (https://zrepl.github.io/) OpenBSD hotplugd automount script (https://bijanebrahimi.github.io/blog/openbsd-hotplugd-scripting.html) Ancient troff sources vs. modern-day groff (https://virtuallyfun.com/2017/12/22/learn-ancient-troff-sources-vs-modern-day-groff/) Paypal donation balance and status.. thanks everyone! (http://lists.dragonflybsd.org/pipermail/users/2017-December/313752.html) Supervised FreeBSD rc.d script for a Go daemon (updated in last few days) (https://redbyte.eu/en/blog/supervised-freebsd-init-script-for-go-deamon/) A Brief History of sed (https://blog.sourcerer.io/a-brief-history-of-sed-6eaf00302ed) Flamegraph: Why does my AWS instance boot so slow? (http://www.daemonology.net/timestamping/tslog-c5.4xlarge.svg) *** ##Feedback/Questions Jeremy - Replacing Drive in a Zpool (http://dpaste.com/319593M#wrap) Dan's Blog (https://dan.langille.org/2017/08/16/swapping-5tb-in-3tb-out/) Tim - Keeping GELI key through reboot (http://dpaste.com/11QTA06) Brian - Mixing 2.5 and 3.5 drives (http://dpaste.com/2JQVD10#wrap) Troels - zfs swap on FreeBSD (http://dpaste.com/147WAFR#wrap) ***
We have a first PS4 kernel exploit, the long awaited OpenZFS devsummit report by Allan, DragonflyBSD 5.0 is out, we show you vmadm to manage jails, and parallel processing with Unix tools. This episode was brought to you by Headlines The First PS4 Kernel Exploit: Adieu (https://fail0verflow.com/blog/2017/ps4-namedobj-exploit/) The First PS4 Kernel Exploit: Adieu Plenty of time has passed since we first demonstrated Linux running on the PS4. Now we will step back a bit and explain how we managed to jump from the browser process into the kernel such that ps4-kexec et al. are usable. Over time, ps4 firmware revisions have progressively added many mitigations and in general tried to lock down the system. This post will mainly touch on vulnerabilities and issues which are not present on the latest releases, but should still be useful for people wanting to investigate ps4 security. Vulnerability Discovery As previously explained, we were able to get a dump of the ps4 firmware 1.01 kernel via a PCIe man-in-the-middle attack. Like all FreeBSD kernels, this image included “export symbols” - symbols which are required to perform kernel and module initialization processes. However, the ps4 1.01 kernel also included full ELF symbols (obviously an oversight as they have been removed in later firmware versions). This oversight was beneficial to the reverse engineering process, although of course not a true prerequisite. Indeed, we began exploring the kernel by examining built-in metadata in the form of the syscall handler table - focusing on the ps4-specific entries. Each process object in the kernel contains its own “idt” (ID Table) object. As can be inferred from the snippet above, the hash table essentially just stores pointers to opaque data blobs, along with a given kind and name. Entries may be accessed (and thus “locked”) with either read or write intent. Note that IDTTYPE is not a bitfield consisting of only unique powers of 2. This means that if we can control the kind of an identry, we may be able to cause a type confusion to occur (it is assumed that we may control name). Exploitation To an exploiter without ps4 background, it might seem that the easiest way to exploit this bug would be to take advantage of the write off the end of the malloc'd namedobjusrt object. However, this turns out to be impossible (as far as I know) because of a side effect of the ps4 page size being changed to 0x4000 bytes (from the normal of 0x1000). It appears that in order to change the page size globally, the ps4 kernel developers opted to directly change the related macros. One of the many changes resulting from this is that the smallest actual amount of memory which malloc may give back to a caller becomes 0x40 bytes. While this also results in tons of memory being completely wasted, it does serve to nullify certain exploitation techniques (likely completely by accident…). Adieu The namedobj exploit was present and exploitable (albeit using a slightly different method than described here) until it was fixed in firmware version 4.06. This vulnerability was also found and exploited by (at least) Chaitin Tech, so props to them! Taking a quick look at the 4.07 kernel, we can see a straightforward fix (4.06 is assumed to be identical - only had 4.07 on hand while writing this post): int sys_namedobj_create(struct thread *td, void *args) { // ... rv = EINVAL; kind = *((_DWORD *)args + 4) if ( !(kind & 0x4000) && *(_QWORD *)args ) { // ... (unchanged) } return rv; } And so we say goodbye to a nice exploit. I hope you enjoyed this blast from the past :) Keep hacking! OpenZFS Developer Summit 2017 Recap (https://www.ixsystems.com/blog/openzfs-devsummit-2017/) The 5th annual OpenZFS Developer Summit was held in San Francisco on October 24-25. Hosted by Delphix at the Children's Creativity Museum in San Francisco, over a hundred OpenZFS contributors from a wide variety of companies attended and collaborated during the conference and developer summit. iXsystems was a Gold sponsor and several iXsystems employees attended the conference, including the entire Technical Documentation Team, the Director of Engineering, the Senior Analyst, a Tier 3 Support Engineer, and a Tier 2 QA Engineer. Day 1 of the conference had 9 highly detailed, informative, and interactive technical presentations from companies which use or contribute to OpenZFS. The presentations highlighted improvements to OpenZFS developed “in-house” at each of these companies, with most improvements looking to be made available to the entire OpenZFS community in the near to long term. There's a lot of exciting stuff happening in the OpenZFS community and this post provides an overview of the presented features and proof-of-concepts. The keynote was delivered by Mark Maybee who spoke about the past, present, and future of ZFS at Oracle. An original ZFS developer, he outlined the history of closed-source ZFS development after Oracle's acquisition of Sun. ZFS has a fascinating history, as the project has evolved over the last decade in both open and closed source forms, independent of one another. While Oracle's proprietary internal version of ZFS has diverged from OpenZFS, it has implemented many of the same features. Mark was very proud of the work his team had accomplished over the years, claiming Oracle's ZFS products have accounted for over a billion dollars in sales and are used in the vast majority of Fortune 100 companies. However, with Oracle aggressively moving into cloud storage, the future of closed source ZFS is uncertain. Mark presented a few ideas to transform ZFS into a mainstream and standard file system, including adding more robust support for Linux. Allan Jude from ScaleEngine talked about ZStandard, a new compression method he is developing in collaboration with Facebook. It offers compression comparable to gzip, but at speeds fast enough to keep up with hard drive bandwidth. According to early testing, it improves both the speed and compression efficiency over the current LZ4 compression algorithm. It also offers a new “dictionary” feature for improving image compression, which is of particular interest to Facebook. In addition, when using ZFS send and receive, it will adapt the compression ratio to make the most efficient use of the network bandwidth. Currently, deleting a clone on ZFS is a time-consuming process, especially when dealing with large datasets that have diverged over time. Sara Hartse from Delphix described how “clone fast delete” speeds up clone deletion. Rather than traversing the entire dataset during clone deletion, changes to the clone are tracked in a “live list” which the delete process uses to determine which blocks to free. In addition, rather than having to wait for the clone to finish, the delete process backgrounds the task so you can keep working without any interruptions. Sara shared the findings of a test they ran on a clone with 500MB of data, which took 45 minutes to delete with the old method, and under a minute using the live list. This behavior is an optional property as it may not be appropriate for long-lived clones where deletion times are not a concern. At this time, it does not support promoted clones. Olaf Faaland from Lawrence Livermore National Labs demonstrated the progress his team has made to improve ZFS pool imports with MMP (Multi-Modifier Protection), a watchdog system to make sure that ZFS pools in clustered High Availability environments are not imported by more than one host at a time. MMP uses uberblocks and other low-level ZFS features to monitor pool import status and otherwise safeguard the import process. MMP adds fields to on-disk metadata so it does not depend on hardware, such as SAS. It supports multi-node HA configs and does not affect non-HA systems. However, it does have issues with long I/O delays so existing HA software is recommended as an additional fallback. Jörgen Lundman of GMO Internet gave an entertaining talk on the trials and tribulations of porting ZFS to OS X. As a bonus, he talked about porting ZFS to Windows, and showed a working demo. While not yet in a usable state, it demonstrated a proof-of-concept of ZFS support for other platforms. Serapheim Dimitropoulos from Delphix discussed Faster Allocation with the Log Spacemap as a means of optimizing ZFS allocation performance. He began with an in-depth overview of metaslabs and how log spacemaps are used to track allocated and freed blocks. Since blocks are only allocated from loaded metaslabs but freed blocks may apply to any metaslab, over time logging the freed blocks to each appropriate metaslab with every txg becomes less efficient. Their solution is to create a pool-wide metaslab for unflushed entries. Shailendra Tripathi from Tegile presented iFlash: Dynamic Adaptive L2ARC Caching. This was an interesting talk on what is required to allow very different classes of resources to share the same flash device–in their case, ZIL, L2ARC, and metadata. To achieve this, they needed to address the following differences for each class: queue priority, metaslab load policy, allocation, and data protection (as cache has no redundancy). Isaac Huang of Intel introduced DRAID, or parity declustered RAID. Once available, this will provide the same levels of redundancy as traditional RAIDZ, providing the administrator doubles the amount of options for providing redundancy for their use case. The goals of DRAID are to address slow resilvering times and the write throughput of a single replacement drive being a bottleneck. This solution skips block pointer tree traversal when rebuilding the pool after drive failure, which is the cause of long resilver times. This means that redundancy is restored quickly, mitigating the risk of losing additional drives before the resilver completes, but it does require a scrub afterwards to confirm data integrity. This solution supports logical spares, which must be defined at vdev creation time, which are used to quickly restore the array. Prakash Surya of Delphix described how ZIL commits currently occur in batches, where waiting threads have to wait for the batch to complete. His proposed solution was to replace batch commits and to instead notify the waiting thread after its ZIL commit in order to greatly increase throughput. A new tunable for the log write block timeout can also be used to log write blocks more efficiently. Overall, the quality of the presentations at the 2017 OpenZFS conference was high. While quite technical, they clearly explained the scope of the problems being addressed and how the proposed solutions worked. We look forward to seeing the described features integrated into OpenZFS. The videos and slides for the presentations should be made available over the next month or so at the OpenZFS website. OpenZFS Photo Album (https://photos.google.com/share/AF1QipNxYQuOm5RDxRgRQ4P8BhtoLDpyCuORKWiLPT0WlvUmZYDdrX3334zu5lvY_sxRBA?key=MW5fR05MdUdPaXFKVDliQVJEb3N3Uy1uMVFFdVdR) DragonflyBSD 5.0 (https://www.dragonflybsd.org/release50/) DragonFly version 5.0 brings the first bootable release of HAMMER2, DragonFly's next generation file system. HAMMER2 Preliminary HAMMER2 support has been released into the wild as-of the 5.0 release. This support is considered EXPERIMENTAL and should generally not yet be used for production machines and important data. The boot loader will support both UFS and HAMMER2 /boot. The installer will still use a UFS /boot even for a HAMMER2 installation because the /boot partition is typically very small and HAMMER2, like HAMMER1, does not instantly free space when files are deleted or replaced. DragonFly 5.0 has single-image HAMMER2 support, with live dedup (for cp's), compression, fast recovery, snapshot, and boot support. HAMMER2 does not yet support multi-volume or clustering, though commands for it exist. Please use non-clustered single images for now. ipfw Updates IPFW has gone through a number of updates in DragonFly and now offers better performance. pf and ipfw3 are also still supported. Improved graphics support The i915 driver has been brought up to match what's in the Linux 4.7.10 kernel. Intel GPUs are supported up to the Kabylake generation. vga_switcheroo(4) module added, allowing the use of Intel GPUs on hybrid-graphics systems. The new apple_gmux driver enables switching to the Intel video chipset on dual Intel/NVIDIA and Intel/Radeon Macbook computers. Other user-affecting changes efisetup(8) added. DragonFly can now support over 900,000 processes on a single machine. Client-side SSH by default does not try password authentication, which is the default behavior in newer versions of OpenSSH. Pass an explicit '-o PasswordAuthentication=yes' or change /etc/ssh/ssh_config if you need the old behavior. Public key users are unaffected. Clang status A starting framework has been added for using clang as the alternate base compiler in DragonFly, to replace gcc 4.7. It's not yet complete. Clang can of course be added as a package. Package updates Many package updates but I think most notably we need to point to chrome60 finally getting into dports with accelerated video and graphics support. 64-bit status Note that DragonFly is a 64-bit-only operating system as of 4.6, and will not run on 32-bit hardware. AMD Ryzen is supported and DragonFly 5.0 has a workaround for a hardware bug (http://lists.dragonflybsd.org/pipermail/commits/2017-August/626190.html). DragonFly quickly released a v5.0.1 with a few patches Download link (https://www.dragonflybsd.org/download/) News Roundup (r)vmadm – managing FreeBSD jails (https://blog.project-fifo.net/rvmadm-managing-freebsd-jails/) We are releasing the first version (0.1.0) of our clone of vmadm for FreeBSD jails today. It is not done or feature complete, but it does provides basic functionality. At this point, we think it would be helpful to get it out there and get some feedback. As of today, it allows basic management of datasets, as well as creating, starting, stopping, and destroying jails. Why another tool to manage jails However, before we go into details let's talk why we build yet another jail manager? It is not the frequent NIH syndrome, actually quite the opposite. In FiFo 0.9.2 we experimented with iocage as a way to control jails. While iocage is a useful tool when used as a CLI utility it has some issues when used programmatically. When managing jails automatically and not via a CLI tool things like performance, or a machine parsable interface matter. While on a CLI it is acceptable if a call takes a second or two, for automatically consuming a tool this delay is problematic. Another reason for the decision was that vmadm is an excellent tool. It is very well designed. SmartOs uses vmadm for years now. Given all that, we opted for adopting a proven interface rather than trying to create a new one. Since we already interface with it on SmartOS, we can reuse a majority of our management code between SmartOS and FreeBSD. What can we do Today we can manage datasets, which are jail templates in the form of ZFS volumes. We can list and serve them from a dataset-server, and fetch those we like want. At this point, we provide datasets for FreeBSD 10.0 to 11.1, but it is very likely that the list will grow. As an idea here is a community-driven list of datasets (https://datasets.at/) that exist for SmartOS today. Moreover, while those datasets will not work, we hope to see the same for BSD jails. After fetching the dataset, we can define jails by using a JSON file. This file is compatible with the zone description used on SmartOS. It does not provide all the same features but a subset. Resources such as CPU and memory can be defined, networking configured, a dataset selected and necessary settings like hostname set. With the jail created, vmadm allows managing its lifetime, starting, stopping it, accessing the console and finally destroying it. Updates to jails are supported to however as of today they are only taken into account after restarting the jail. However, this is in large parts not a technical impossibility but rather wasn't high up on the TODO list. It is worth mentioning that vmadm will not pick up jails created in other tools or manually. Only using vmadm created jails was a conscious decision to prevent it interfering with existing setups or other utilities. While conventional tools can manage jails set up with vmadm just fine we use some special tricks like nested jails to allow for restrictions required for multi-tenancy that are hard or impossible to achieve otherwise. Whats next First and foremost we hope to get some feedback and perhaps community engagement. In the meantime, as announced earlier this year (https://blog.project-fifo.net/fifo-in-2017/), we are hard at work integrating FreeBSD hypervisors in FiFo, and as of writing this, the core actions work quite well. Right now only the barebone functions are supported, some of the output is not as clear as we would like. We hope to eventually add support for behyve to vmadm the same way that it supports KVM on SmartOS. Moreover, the groundwork for this already exists in the nested jail techniques we are using. Other than that we are exploring ways to allow for PCI pass through in jails, something not possible in SmartOS zones right now that would be beneficial for some users. In general, we want to improve compatibility with SmartOS as much as possible and features that we add over time should make the specifications invalid for SmartOS. You can get the tool from github (https://github.com/project-fifo/r-vmadm). *** Parallel processing with unix tools (http://www.pixelbeat.org/docs/unix-parallel-tools.html) There are various ways to use parallel processing in UNIX: piping An often under appreciated idea in the unix pipe model is that the components of the pipe run in parallel. This is a key advantage leveraged when combining simple commands that do "one thing well" split -n, xargs -P, parallel Note programs that are invoked in parallel by these, need to output atomically for each item processed, which the GNU coreutils are careful to do for factor and sha*sum, etc. Generally commands that use stdio for output can be wrapped with the stdbuf -oL command to avoid intermixing lines from parallel invocations make -j Most implementations of make(1) now support the -j option to process targets in parallel. make(1) is generally a higher level tool designed to process disparate tasks and avoid reprocessing already generated targets. For example it is used very effictively when testing coreutils where about 700 tests can be processed in 13 seconds on a 40 core machine. implicit threading This goes against the unix model somewhat and definitely adds internal complexity to those tools. The advantages can be less data copying overhead, and simpler usage, though its use needs to be carefully considered. A disadvantage is that one loses the ability to easily distribute commands to separate systems. Examples are GNU sort(1) and turbo-linecount The example provided counts lines in parallel: The examples below will compare the above methods for implementing multi-processing, for the function of counting lines in a file. First of all let's generate some test data. We use both long and short lines to compare the overhead of the various methods compared to the core cost of the function being performed: $ seq 100000000 > lines.txt # 100M lines $ yes $(yes longline | head -n9) | head -n10000000 > long-lines.txt # 10M lines We'll also define the add() { paste -d+ -s | bc; } helper function to add a list of numbers. Note the following runs were done against cached files, and thus not I/O bound. Therefore we limit the number of processes in parallel to $(nproc), though you would generally benefit to raising that if your jobs are waiting on network or disk etc. + We'll use this command to count lines for most methods, so here is the base non multi-processing performance for comparison: $ time wc -l lines.txt $ time wc -l long-lines.txt split -n Note using -n alone is not enough to parallelize. For example this will run serially with each chunk, because since --filter may write files, the -n pertains to the number of files to split into rather than the number to process in parallel. $ time split -n$(nproc) --filter='wc -l' lines.txt | add You can either run multiple invocations of split in parallel on separate portions of the file like: $ time for i in $(seq $(nproc)); do split -n$i/$(nproc) lines.txt | wc -l& done | add Or split can do parallel mode using round robin on each line, but that's huge overhead in this case. (Note also the -u option significant with -nr): $ time split -nr/$(nproc) --filter='wc -l' lines.txt | add Round robin would only be useful when the processing per item is significant. Parallel isn't well suited to processing a large single file, rather focusing on distributing multiple files to commands. It can't efficiently split to lightweight processing if reading sequentially from pipe: $ time parallel --will-cite --block=200M --pipe 'wc -l' < lines.txt | add Like parallel, xargs is designed to distribute separate files to commands, and with the -P option can do so in parallel. If you have a large file then it may be beneficial to presplit it, which could also help with I/O bottlenecks if the pieces were placed on separate devices: split -d -n l/$(nproc) lines.txt l. Those pieces can then be processed in parallel like: $ time find -maxdepth 1 -name 'l.*' | xargs -P$(nproc) -n1 wc -l | cut -f1 -d' ' | add If your file sizes are unrelated to the number of processors then you will probably want to adjust -n1 to batch together more files to reduce the number of processes run in total. Note you should always specify -n with -P to avoid xargs accumulating too many input items, thus impacting the parallelism of the processes it runs. make(1) is generally used to process disparate tasks, though can be leveraged to provide low level parallel processing on a bunch of files. Note also the make -O option which avoids the need for commands to output their data atomically, letting make do the synchronization. We'll process the presplit files as generated for the xargs example above, and to support that we'll use the following Makefile: %: FORCE # Always run the command @wc -l < $@ FORCE: ; Makefile: ; # Don't include Makefile itself One could generate this and pass to make(1) with the -f option, though we'll keep it as a separate Makefile here for simplicity. This performs very well and matches the performance of xargs. $ time find -name 'l.*' -exec make -j$(nproc) {} + | add Note we use the POSIX specified "find ... -exec ... {} +" construct, rather than conflating the example with xargs. This construct like xargs will pass as many files to make as possible, which make(1) will then process in parallel. OpenBSD gives a hint on forgetting unlock mutex (http://nanxiao.me/en/openbsd-gives-a-hint-on-forgetting-unlock-mutex/) OpenBSD gives a hint on forgetting unlock mutex Check following simple C++ program: > ``` #include int main(void) { std::mutex m; m.lock(); return 0; } ``` The mutex m forgot unlock itself before exiting main function: m.unlock(); Test it on GNU/Linux, and I chose ArchLinux as the testbed: $ uname -a Linux fujitsu-i 4.13.7-1-ARCH #1 SMP PREEMPT Sat Oct 14 20:13:26 CEST 2017 x86_64 GNU/Linux $ clang++ -g -pthread -std=c++11 test_mutex.cpp $ ./a.out $ The process exited normally, and no more words was given. Build and run it on OpenBSD 6.2: clang++ -g -pthread -std=c++11 test_mutex.cpp ./a.out pthread_mutex_destroy on mutex with waiters! The OpenBSD prompts “pthreadmutexdestroy on mutex with waiters!“. Interesting! *** Beastie Bits Updates to the NetBSD operating system since OSHUG #57 & #58 (http://mailman.uk.freebsd.org/pipermail/ukfreebsd/2017-October/014148.html) Creating a jail with FiFo and Digital Ocean (https://blog.project-fifo.net/fifo-jails-digital-ocean/) I'm thinking about OpenBSD again (http://stevenrosenberg.net/blog/bsd/openbsd/2017_0924_openbsd) Kernel ASLR on amd64 (https://blog.netbsd.org/tnf/entry/kernel_aslr_on_amd64) Call for Participation - BSD Devroom at FOSDEM (https://people.freebsd.org/~rodrigo/fosdem18/) BSD Stockholm Meetup (https://www.meetup.com/BSD-Users-Stockholm/) *** Feedback/Questions architect - vBSDCon (http://dpaste.com/15D5SM4#wrap) Brad - Packages and package dependencies (http://dpaste.com/3MENN0X#wrap) Lars - dpb (http://dpaste.com/2SVS18Y) Alex re: PS4 Network Throttling (http://dpaste.com/028BCFA#wrap) ***
EuroBSDcon trip report, how to secure OpenBSD's LDAP server, ZFS channel programs in FreeBSD HEAD and why software is storytelling. This episode was brought to you by Headlines EuroBSDcon Trip Report This is from Frank Moore, who has been supplying us with collections of links for the show and who we met at EuroBSDcon in Paris for the first time. Here is his trip report. My attendance at the EuroBSDCon 2017 conference in Paris was sprinkled with several 'firsts'. My first visit to Paris, my first time travelling on a EuroTunnel Shuttle train and my first time at any BSD conference. Hopefully, none of these will turn out to be 'lasts'. I arrived on the Wednesday afternoon before the conference started on Thursday morning. My hotel was conveniently located close to the conference centre in Paris' 3rd arrondissement. This area is well-known as a buzzy enclave of hip cafes, eateries, independent shops, markets, modern galleries and museums. It certainly lived up to its reputation. Even better, the weather held over the course of the conference, only raining once, with the rest of the time being both warm and sunny. The first two days were taken up with attending Dr Kirk McKusick's excellent tutorial 'An Introduction to the FreeBSD Open-Source Operating System'. This is training "straight from the horse's mouth". Kirk has worked extensively on The FreeBSD operating system since the 1980's, helping to design the original BSD filesystem (FFS) and later working on UFS as well. Not only is Kirk an engaging speaker, making what could be a dry topic very interesting, he also sprinkles liberal doses of history and war stories throughout his lectures. Want to know why a protocol was designed the way that it was? Or why a system flag has a particular value or position in a record? Kirk was there and has the first-hand answer. He reminisces about his meetings and work with other Unix and BSD luminaries and debunks and confirms common myths in equal measure. Kirk's teaching style and knowledge are impressive. Every section starts with an overview and a big picture diagram before drilling down into the nitty-gritty detail. Nothing feels superfluous, and everything fits together logically. It's easy to tell that the material and its delivery have been honed over many years, but without feeling stale. Topics covered included the kernel, processes, virtual memory, threads, I/O, devices, FFS, ZFS, and networking. The slides were just as impressive, with additional notes written by a previous student and every slide containing a reference back to the relevant page(s) in the 2nd edition of Kirk's operating system book. As well as a hard copy for those that requested it, Kirk also helpfully supplied soft copies of all the training materials. The breaks in between lectures were useful for meeting the students from the other tutorials and for recovering from the inevitable information overload. It's not often that you can get to hear someone as renowned as Dr McKusick give a lecture on something as important as the FreeBSD operating system. If you have any interest in FreeBSD, Unix history, or operating systems in general, I would urge you to grab the opportunity to attend one of his lectures. You won't be disappointed. The last two days of the conference consisted of various hour-long talks by members of each of the main BSD systems. All of them were fairly evenly represented except Dragonfly BSD which unfortunately only had one talk. With three talks going on at any one time, it was often difficult to pick which one to go to. At other times there might be nothing to pique the interest. Attendance at a talk is not mandatory, so for those times when no talks looked inviting, just hanging out in one of the lobby areas with other attendees was often just as interesting and informative. The conference centre itself was certainly memorable with the interior design of an Egyptian temple or pyramid. All the classrooms were more than adequate while the main auditorium was first-class and easily held the 300+ attendees comfortably. All in all, the facilities, catering and organisation were excellent. Kudos to the EuroBSDCon team, especially Bapt and Antoine for all their hard work and hospitality. As a long-time watcher and occasional contributor to the BSD Now podcast it was good to meet both Allan and Benedict in the flesh. And having done some proofreading for Michael Lucas previously, it was nice to finally meet him as well. My one suggestion to the organisers of the next conference would be to provide more hand-holding for newbies. As a first-time attendee at a BSD conference it would have been nice to have been formally introduced to various people within the projects as the goto people for their areas. I could do this myself, but it's not always easy finding the right person and wrangling an introduction. I also think it was a missed opportunity for each project to recruit new developers to their cause. Apparently, this is already in place at BSDCan, but should probably be rolled out across all BSD conferences. Having said all that, my aims for the conference were to take Dr McKusick's course, meet a few BSD people and make contacts within one of the BSD projects to start contributing. I was successful on all these fronts, so for me this was mission accomplished. Another first! autoconf/clang (No) Fun and Games (https://undeadly.org/cgi?action=article;sid=20170930133438) Robert Nagy (robert@) wrote in with a fascinating story of hunting down a recent problem with ports: You might have been noticing the amount of commits to ports regarding autoconf and nested functions and asking yourself… what the hell is this all about? I was hanging out at my friend Antoine (ajacoutot@)'s place just before EuroBSDCon 2017 started and we were having drinks and he told me that there is this weird bug where Gnome hangs completely after just a couple of seconds of usage and the gnome-shell process just sits in the fsleep state. This started to happen at the time when inteldrm(4) was updated, the default compiler was switched to clang(1) and futexes were turned on by default. The next day we started to have a look at the issue and since the process was hanging in fsleep, it seemed clear that the cause must be futexes, so we had to start bisecting the base system, which resulted in random success and failure. In the end we figured out that it is neither futex nor inteldrm(4) related, so the only thing that was left is the switch to clang. Now the problem is that we have to figure out what part of the system needs to be build with clang to trigger this issue, so we kept on going and systematically recompiled the base system with gcc until everything was ruled out … and it kept on hanging. We were drunk and angry that now we have to go and check hundreds of ports because gnome is not a small standalone port, so between two bottles of wine a build VM was fired up to do a package build with gcc, because manually building all the dependencies would just take too long and we had spent almost two days on this already. Next day ~200 packages were available to bisect and figure out what's going on. After a couple of tries it turned out that the hang is being caused by the gtk+3 package, which is bad since almost everything is using gtk+3. Now it was time to figure out what file the gtk+3 source being built by clang is causing the issue. (Compiler optimizations were ruled out already at this point.) So another set of bisecting happened, building each subdirectory of gtk+3 with clang and waiting for the hang to manifest … and it did not. What the $f? Okay so something else is going on and maybe the configure script of gtk+3 is doing something weird with different compilers, so I quickly did two configure runs with gcc and clang and simply diff'd the two directories. Snippets from the diff: -GDKHIDDENVISIBILITYCFLAGS = -fvisibility=hidden GDKHIDDENVISIBILITYCFLAGS = -ltcvprogcompilerrttiexceptions=no ltcvprogcompilerrttiexceptions=yes -#define GDKEXTERN attribute((visibility("default"))) extern -ltprogcompilernobuiltinflag=' -fno-builtin' +ltprogcompilernobuiltinflag=' -fno-builtin -fno-rtti -fno-exceptions' Okay, okay that's something, but wait … clang has symbol visibility support so what is going on again? Let's take a peek at config.log: configure:29137: checking for -fvisibility=hidden compiler flag configure:29150: cc -c -fvisibility=hidden -I/usr/local/include -I/usr/X11R6/include conftest.c >&5 conftest.c:82:17: error: function definition is not allowed here int main (void) { return 0; } ^ 1 error generated. Okay that's clearly an error but why exactly? autoconf basically generates a huge shell script that will check for whatever you throw at it by creating a file called conftest.c and putting chunks of code into it and then trying to compile it. In this case the relevant part of the code was: | int | main () | { | int main (void) { return 0; } | ; | return 0; | } That is a nested function declaration which is a GNU extension and it is not supported by clang, but that's okay, the question is why the hell would you use nested functions to check for simple compiler flags. The next step was to go and check what is going on in configure.ac to see how the configure script is generated. In the gtk+3 case the following snippet is used: AC_MSG_CHECKING([for -fvisibility=hidden compiler flag]) ACTRYCOMPILE([], [int main (void) { return 0; }], ACMSGRESULT(yes) enablefvisibilityhidden=yes, ACMSGRESULT(no) enablefvisibilityhidden=no) According to the autoconf manual the ACTRYCOMPILE macro accepts the following parameters: That clearly states that a function body has to be specified because the function definition is already provided automatically, so doing ACTRYCOMPILE([], [int main (void) { return 0;}], instead of ACTRYCOMPILE([],[] will result in a nested function declaration, which will work just fine with gcc, even though the autoconf usage is wrong. After fixing the autoconf macro in gtk+3 and rebuilding the complete port from scratch with clang, the hang completely went away as the proper CFLAGS and LDFLAGS were picked up by autoconf for the build. At this point we realized that most of the ports tree uses autoconf so this issue might be a lot bigger than we thought, so I asked sthen@ to do a grep on the ports object directory and just search for "function definition is not allowed here", which resulted in about ~60 additional ports affected. Out of the list of ports there were only two false positive matches. These were actually trying to test whether the compiler supports nested functions. The rest were a combination of several autoconf macros used in a wrong way, e.g: ACTRYCOMPILE, ACTRYLINK. Most of them were fixable by just removing the extra function declaration or by switching to other autoconf macros like ACLANGSOURCE where you can actually declare your own functions if need be. The conclusion is that this issue was a combination of people not reading documentation and just copy/pasting autoconf snippets, instead of reading their documentation and using the macros in the way they were intended, and the fact that switching to a new compiler is never easy and bugs or undefined behaviour are always lurking in the dark. Thanks to everyone who helped fixing all the ports up this quickly! Hopefully all of the changes can be merged upstream, so that others can benefit as well. Interview - David Carlier - @devnexen (https://twitter.com/devnexen) Software Engineer at Afilias *** News Roundup Setting up OpenBSD's LDAP Server (ldapd) with StartTLS and SASL (http://blog.databasepatterns.com/2017/08/setting-up-openbsds-ldap-server-ldapd.html) A tutorial on setting up OpenBSD's native LDAP server with TLS encryption and SASL authentication OpenBSD has its own LDAP server, ldapd. Here's how to configure it for use with StartTLS and SASL authentication Create a certificate (acme-client anyone?) Create a basic config file listen on em0 tls certificate ldapserver This will listen on the em0 interface with tls using the certificate called ldapserver.crt / ldapserver.key Validate the configuration: /usr/sbin/ldapd -n Enable and start the service: rcctl enable ldapd rcctl start ldapd On the client machine: pkg_add openldap-client Copy the certificate to /etc/ssl/trusted.crt Add this line to /etc/openldap/ldap.conf TLS_CACERT /etc/ssl/trusted.crt Enable and start the service rcctl enable saslauthd rcctl start saslauthd Connect to ldapd (-ZZ means force TLS, use -H to specify URI): ldapsearch -H ldap://ldapserver -ZZ FreeBSD Picks Up Support for ZFS Channel Programs in -current (https://svnweb.freebsd.org/base?view=revision&revision=324163) ZFS channel programs (ZCP) adds support for performing compound ZFS administrative actions via Lua scripts in a sandboxed environment (with time and memory limits). This initial commit includes both base support for running ZCP scripts, and a small initial library of API calls which support getting properties and listing, destroying, and promoting datasets. Testing: in addition to the included unit tests, channel programs have been in use at Delphix for several months for batch destroying filesystems. Take a simple task as an example: Create a snapshot, then set a property on that snapshot. In the traditional system for this, when you issue the snapshot command, that closes the currently open transaction group (say #100), and opens a new one, #101. While #100 is being written to disk, other writes are accumulated in #101. Once #100 is flushed to disk, the ‘zfs snapshot' command returns. You can then issue the ‘zfs set' command. This actually ends up going into transaction group #102. Each administrative action needs to wait for the transaction group to flush, which under heavy loads could take multiple seconds. Now if you want to create AND set, you need to wait for two or three transaction groups. Meanwhile, during transaction group #101, the snapshot existed without the property set, which could cause all kinds of side effects. ZFS Channel programs solves this by allowing you to perform a small scripted set of actions as a single atomic operation. In Delphix's appliance, they often needed to do as many as 15 operations together, which might take multiple minutes. Now with channel programs it is much faster, far safer, and has fewer chances of side effects BSDCan 2017 - Matt Ahrens: Building products based on OpenZFS, using channel programs -- Video Soon (http://www.bsdcan.org/2017/schedule/events/854.en.html) Software Is About Storytelling (http://bravenewgeek.com/software-is-about-storytelling/) Tyler Treat writes on the brave new geek blog: Software engineering is more a practice in archeology than it is in building. As an industry, we undervalue storytelling and focus too much on artifacts and tools and deliverables. How many times have you been left scratching your head while looking at a piece of code, system, or process? It's the story, the legacy left behind by that artifact, that is just as important—if not more—than the artifact itself. And I don't mean what's in the version control history—that's often useless. I mean the real, human story behind something. Artifacts, whether that's code or tools or something else entirely, are not just snapshots in time. They're the result of a series of decisions, discussions, mistakes, corrections, problems, constraints, and so on. They're the product of the engineering process, but the problem is they usually don't capture that process in its entirety. They rarely capture it at all. They commonly end up being nothing but a snapshot in time. It's often the sign of an inexperienced engineer when someone looks at something and says, “this is stupid” or “why are they using X instead of Y?” They're ignoring the context, the fact that circumstances may have been different. There is a story that led up to that point, a reason for why things are the way they are. If you're lucky, the people involved are still around. Unfortunately, this is not typically the case. And so it's not necessarily the poor engineer's fault for wondering these things. Their predecessors haven't done enough to make that story discoverable and share that context. I worked at a company that built a homegrown container PaaS on ECS. Doing that today would be insane with the plethora of container solutions available now. “Why aren't you using Kubernetes?” Well, four years ago when we started, Kubernetes didn't exist. Even Docker was just in its infancy. And it's not exactly a flick of a switch to move multiple production environments to a new container runtime, not to mention the politicking with leadership to convince them it's worth it to not ship any new code for the next quarter as we rearchitect our entire platform. Oh, and now the people behind the original solution are no longer with the company. Good luck! And this is on the timescale of about five years. That's maybe like one generation of engineers at the company at most—nothing compared to the decades or more software usually lives (an interesting observation is that timescale, I think, is proportional to the size of an organization). Don't underestimate momentum, but also don't underestimate changing circumstances, even on a small time horizon. The point is, stop looking at technology in a vacuum. There are many facets to consider. Likewise, decisions are not made in a vacuum. Part of this is just being an empathetic engineer. The corollary to this is you don't need to adopt every bleeding-edge tech that comes out to be successful, but the bigger point is software is about storytelling. The question you should be asking is how does your organization tell those stories? Are you deliberate or is it left to tribal knowledge and hearsay? Is it something you truly value and prioritize or simply a byproduct? Documentation is good, but the trouble with documentation is it's usually haphazard and stagnant. It's also usually documentation of how and not why. Documenting intent can go a long way, and understanding the why is a good way to develop empathy. Code survives us. There's a fantastic talk by Bryan Cantrill on oral tradition in software engineering (https://youtu.be/4PaWFYm0kEw) where he talks about this. People care about intent. Specifically, when you write software, people care what you think. As Bryan puts it, future generations of programmers want to understand your intent so they can abide by it, so we need to tell them what our intent was. We need to broadcast it. Good code comments are an example of this. They give you a narrative of not only what's going on, but why. When we write software, we write it for future generations, and that's the most underestimated thing in all of software. Documenting intent also allows you to document your values, and that allows the people who come after you to continue to uphold them. Storytelling in software is important. Without it, software archeology is simply the study of puzzles created by time and neglect. When an organization doesn't record its history, it's bound to repeat the same mistakes. A company's memory is comprised of its people, but the fact is people churn. Knowing how you got here often helps you with getting to where you want to be. Storytelling is how we transcend generational gaps and the inevitable changing of the old guard to the new guard in a maturing engineering organization. The same is true when we expand that to the entire industry. We're too memoryless—shipping code and not looking back, discovering everything old that is new again, and simply not appreciating our lineage. Beastie Bits 1st BSD Users Stockholm Meetup (https://www.meetup.com/en-US/BSD-Users-Stockholm/) Absolute FreeBSD, 3rd Edition draft completed (https://blather.michaelwlucas.com/archives/3020) Absolute FreeBSD, 3rd Edition Table of Contents (https://blather.michaelwlucas.com/archives/2995) t2k17 Hackathon Report: My first time (Aaron Bieber) (https://undeadly.org/cgi?action=article;sid=20170824193521) The release of pfSense 2.4.0 will be slightly delayed to apply patches for vulnerabilities in 3rd party packages that are part of pfSense (https://www.netgate.com/blog/no-plan-survives-contact-with-the-internet.html) Feedback/Questions Ben writes in that zrepl is in ports now (http://dpaste.com/1XMJYMH#wrap) Peter asks us about Netflix on BSD (http://dpaste.com/334WY4T#wrap) meka writes in about dhclient exiting (http://dpaste.com/3GSGKD3#wrap) ***
We recap EuroBSDcon in Paris, tell the story behind a pf PR, and show you how to do screencasting with OpenBSD. This episode was brought to you by Headlines Recap of EuroBSDcon 2017 in Paris, France (https://2017.eurobsdcon.org) EuroBSDcon was held in Paris, France this year, which drew record numbers this year. With over 300 attendees, it was the largest BSD event I have ever attended, and I was encouraged by the higher than expected number of first time attendees. The FreeBSD Foundation held a board meeting on Wednesday afternoon with the members who were in Paris. Topics included future conferences (including a conference kit we can mail to people who want to represent FreeBSD) and planning for next year. The FreeBSD Devsummit started on Thursday at the beautiful Mozilla Office in Paris. After registering and picking up our conference bag, everyone gathered for a morning coffee with lots of handshaking and greeting. We then gathered in the next room which had a podium with microphone, screens as well as tables and chairs. After developers sat down, Benedict opened the devsummit with a small quiz about France for developers to win a Mogics Power Bagel (https://www.mogics.com/?page_id=3824). 45 developers participated and DES won the item in the end. After introductions and collecting topics of interest from everyone, we started with the Work in Progress (WIP) session. The WIP session had different people present a topic they are working on in 7 minute timeslots. Topics ranged from FreeBSD Forwarding Performance, fast booting options, and a GELI patch under review to attach multiple providers. See their slides on the FreeBSD wiki (https://wiki.freebsd.org/DevSummit/201709). After lunch, the FreeBSD Foundation gave a general update on staff and funding, as well as a more focused presentation about our partnership with Intel. People were interested to hear what was done so far and asked a few questions to the Intel representative Glenn Weinberg. After lunch, developers worked quietly on their own projects. The mic remained open and occasionally, people would step forward and gave a short talk without slides or motivated a discussion of common interest. The day concluded with a dinner at a nice restaurant in Paris, which allowed to continue the discussions of the day. The second day of the devsummit began with a talk about the CAM-based SDIO stack by Ilya Bakulin. His work would allow access to wifi cards/modules on embedded boards like the Raspberry Pi Zero W and similar devices as many of these are using SDIO for data transfers. Next up was a discussion and Q&A session with the FreeBSD core team members who were there (missing only Benno Rice, Kris Moore, John Baldwin, and Baptiste Daroussin, the latter being busy with conference preparations). The new FCP (FreeBSD community proposals) were introduced for those who were not at BSDCan this year and the hows and whys about it. Allan and I were asked to describe our experiences as new members of core and we encouraged people to run for core when the next election happens. After a short break, Scott Long gave an overview of the work that's been started on NUMA (Non-Uniform Memory Architecture), what the goals of the project are and who is working on it. Before lunch, Christian Schwarz presented his work on zrepl, a new ZFS replication solution he developed using Go. This sparked interest in developers, a port was started (https://reviews.freebsd.org/D12462) and people suggested to Christian that he should submit his talk to AsiaBSDcon and BSDCan next year. Benedict had to leave before lunch was done to teach his Ansible tutorial (which was well attended) at the conference venue. There were organized dinners, for those two nights, quite a feat of organization to fit over 100 people into a restaurant and serve them quickly. On Saturday, there was a social event, a river cruise down the Seine. This took the form of a ‘standing' dinner, with a wide selection of appetizer type dishes, designed to get people to walk around and converse with many different people, rather than sit at a table with the same 6-8 people. I talked to a much larger group of people than I had managed to at the other dinners. I like having both dinner formats. We would also like to thank all of the BSDNow viewers who attended the conference and made the point of introducing themselves to us. It was nice to meet you all. The recordings of the live video stream from the conference are available immediately, so you can watch the raw versions of the talks now: Auditorium Keynote 1: Software Development in the Age of Heroes (https://youtu.be/4iR8g9-39LM?t=179) by Thomas Pornin (https://twitter.com/BearSSLnews) Tuning FreeBSD for routing and firewalling (https://youtu.be/4iR8g9-39LM?t=1660) by Olivier Cochard-Labbé (https://twitter.com/ocochardlabbe) My BSD sucks less than yours, Act I (https://youtu.be/4iR8g9-39LM?t=7040) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_) My BSD sucks less than yours, Act II (https://youtu.be/4iR8g9-39LM?t=14254) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_) Reproducible builds on NetBSD (https://youtu.be/4iR8g9-39LM?t=23351) by Christos Zoulas Your scheduler is not the problem (https://youtu.be/4iR8g9-39LM?t=26845) by Martin Pieuchot Keynote 2: A French story on cybercrime (https://youtu.be/4iR8g9-39LM?t=30540) by Éric Freyssinet (https://twitter.com/ericfreyss) Case studies of sandboxing base system with Capsicum (https://youtu.be/jqdHYEH_BQY?t=731) by Mariusz Zaborski (https://twitter.com/oshogbovx) OpenBSD's small steps towards DTrace (a tale about DDB and CTF) (https://youtu.be/jqdHYEH_BQY?t=6030) by Jasper Lievisse Adriaanse The Realities of DTrace on FreeBSD (https://youtu.be/jqdHYEH_BQY?t=13096) by George Neville-Neil (https://twitter.com/gvnn3) OpenSMTPD, current state of affairs (https://youtu.be/jqdHYEH_BQY?t=16818) by Gilles Chehade (https://twitter.com/PoolpOrg) Hoisting: lessons learned integrating pledge into 500 programs (https://youtu.be/jqdHYEH_BQY?t=21764) by Theo de Raadt Keynote 3: System Performance Analysis Methodologies (https://youtu.be/jqdHYEH_BQY?t=25463) by Brendan Gregg (https://twitter.com/brendangregg) Closing Session (https://youtu.be/jqdHYEH_BQY?t=29355) Karnak “Is it done yet ?” The never ending story of pkg tools (https://youtu.be/1hjzleqGRYk?t=71) by Marc Espie (https://twitter.com/espie_openbsd) A Tale of six motherboards, three BSDs and coreboot (https://youtu.be/1hjzleqGRYk?t=7498) by Piotr Kubaj and Katarzyna Kubaj State of the DragonFly's graphics stack (https://youtu.be/1hjzleqGRYk?t=11475) by François Tigeot From NanoBSD to ZFS and Jails – FreeBSD as a Hosting Platform, Revisited (https://youtu.be/1hjzleqGRYk?t=16227) by Patrick M. Hausen Bacula – nobody ever regretted making a backup (https://youtu.be/1hjzleqGRYk?t=20069) by Dan Langille (https://twitter.com/DLangille) Never Lose a Syslog Message (https://youtu.be/qX0BS4P65cQ?t=325) by Alexander Bluhm Running CloudABI applications on a FreeBSD-based Kubernetes cluster (https://youtu.be/qX0BS4P65cQ?t=5647) by Ed Schouten (https://twitter.com/EdSchouten) The OpenBSD web stack (https://youtu.be/qX0BS4P65cQ?t=13255) by Michael W. Lucas (https://twitter.com/mwlauthor) The LLDB Debugger on NetBSD (https://youtu.be/qX0BS4P65cQ?t=16835) by Kamil Rytarowski What's in store for NetBSD 8.0? (https://youtu.be/qX0BS4P65cQ?t=21583) by Alistair Crooks Louxor A Modern Replacement for BSD spell(1) (https://youtu.be/6Nen6a1Xl7I?t=156) by Abhinav Upadhyay (https://twitter.com/abhi9u) Portable Hotplugging: NetBSD's uvm_hotplug(9) API development (https://youtu.be/6Nen6a1Xl7I?t=5874) by Cherry G. Mathew Hardening pkgsrc (https://youtu.be/6Nen6a1Xl7I?t=9343) by Pierre Pronchery (https://twitter.com/khorben) Discovering OpenBSD on AWS (https://youtu.be/6Nen6a1Xl7I?t=14874) by Laurent Bernaille (https://twitter.com/lbernail) OpenBSD Testing Infrastructure Behind bluhm.genua.de (https://youtu.be/6Nen6a1Xl7I?t=18639) by Jan Klemkow The school of hard knocks – PT1 (https://youtu.be/8wuW8lfsVGc?t=276) by Sevan Janiyan (https://twitter.com/sevanjaniyan) 7 years of maintaining firefox, and still looking ahead (https://youtu.be/8wuW8lfsVGc?t=5321) by Landry Breuil Branch VPN solution based on OpenBSD, OSPF, RDomains and Ansible (https://youtu.be/8wuW8lfsVGc?t=12385) by Remi Locherer Running BSD on AWS (https://youtu.be/8wuW8lfsVGc?t=15983) by Julien Simon and Nicolas David Getting started with OpenBSD device driver development (https://youtu.be/8wuW8lfsVGc?t=21491) by Stefan Sperling A huge thanks to the organizers, program committee, and sponsors of EuroBSDCon. Next year, EuroBSDcon will be in Bucharest, Romania. *** The story of PR 219251 (https://www.sigsegv.be//blog/freebsd/PR219251) The actual story I wanted Kristof to tell, the pf bug he fixed at the Essen Hackathon earlier this summer. As I threatened to do in my previous post, I'm going to talk about PR 219251 for a bit. The bug report dates from only a few months ago, but the first report (that I can remeber) actually came from Shawn Webb on Twitter, of all places Despite there being a stacktrace it took quite a while (nearly 6 months in fact) before I figured this one out. It took Reshad Patuck managing to distill the problem down to a small-ish test script to make real progress on this. His testcase meant that I could get core dumps and experiment. It also provided valuable clues because it could be tweaked to see what elements were required to trigger the panic. This test script starts a (vnet) jail, adds an epair interface to it, sets up pf in the jail, and then reloads the pf rules on the host. Interestingly the panic does not seem to occur if that last step is not included. Obviously not the desired behaviour, but it seems strange. The instances of pf in the jails are supposed to be separate. We try to fetch a counter value here, but instead we dereference a bad pointer. There's two here, so already we need more information. Inspection of the core dump reveals that the state pointer is valid, and contains sane information. The rule pointer (rule.ptr) points to a sensible location, but the data is mostly 0xdeadc0de. This is the memory allocator being helpful (in debug mode) and writing garbage over freed memory, to make use-after-free bugs like this one easier to find. In other words: the rule has been free()d while there was still a state pointing to it. Somehow we have a state (describing a connection pf knows about) which points to a rule which no longer exists. The core dump also shows that the problem always occurs with states and rules in the default vnet (i.e. the host pf instance), not one of the pf instances in one of the vnet jails. That matches with the observation that the test script does not trigger the panic unless we also reload the rules on the host. Great, we know what's wrong, but now we need to work out how we can get into this state. At this point we're going to have to learn something about how rules and states get cleaned up in pf. Don't worry if you had no idea, because before this bug I didn't either. The states keep a pointer to the rule they match, so when rules are changed (or removed) we can't just delete them. States get cleaned up when connections are closed or they time out. This means we have to keep old rules around until the states that use them expire. When rules are removed pfunlinkrule() adds then to the Vpfunlinkedrules list (more on that funny V prefix later). From time to time the pf purge thread will run over all states and mark the rules that are used by a state. Once that's done for all states we know that all rules that are not marked as in-use can be removed (because none of the states use it). That can be a lot of work if we've got a lot of states, so pfpurgethread() breaks that up into smaller chuncks, iterating only part of the state table on every run. We iterate over all of our virtual pf instances (VNETFOREACH()), check if it's active (for FreeBSD-EN-17.08, where we've seen this code before) and then check the expired states with pfpurgeexpiredstates(). We start at state 'idx' and only process a certain number (determined by the PFTMINTERVAL setting) states. The pfpurgeexpiredstates() function returns a new idx value to tell us how far we got. So, remember when I mentioned the odd V_ prefix? Those are per-vnet variables. They work a bit like thread-local variables. Each vnet (virtual network stack) keeps its state separate from the others, and the V_ variables use a pointer that's changed whenever we change the currently active vnet (say with CURVNETSET() or CURVNETRESTORE()). That's tracked in the 'curvnet' variable. In other words: there are as many Vpfvnetactive variables as there are vnets: number of vnet jails plus one (for the host system). Why is that relevant here? Note that idx is not a per-vnet variable, but we handle multiple pf instances here. We run through all of them in fact. That means that we end up checking the first X states in the first vnet, then check the second X states in the second vnet, the third X states in the third and so on and so on. That of course means that we think we've run through all of the states in a vnet while we really only checked some of them. So when pfpurgeunlinkedrules() runs it can end up free()ing rules that actually are still in use because pfpurgethread() skipped over the state(s) that actually used the rule. The problem only happened if we reloaded rules in the host, because the active ruleset is never free()d, even if there are no states pointing to the rule. That explains the panic, and the fix is actually quite straightforward: idx needs to be a per-vnet variable, Vpfpurge_idx, and then the problem is gone. As is often the case, the solution to a fairly hard problem turns out to be really simple. As you might expect, finding the problem takes a lot more work that fixing it Thanks to Kristof for writing up this detailed post explaining how the problem was found, and what caused it. *** vBSDcon 2017: BSD at Work (https://www.ixsystems.com/blog/vbsdcon-2017-dexter/) The third biennial vBSDcon hosted by Verisign took place September 7th through 9th with the FreeBSD Developer Summit taking place the first day. vBSDcon and iXsystems' MeetBSD event have been alternating between the East and West coasts of the U.S.A. and these two events play vital roles in reaching Washington, DC-area and Bay Area/Silicon Valley audiences. Where MeetBSD serves many BSD Vendors, vBSDcon attracts a unique government and security industry demographic that isn't found anywhere else. Conference time and travel budgets are always limited and bringing these events to their attendees is a much-appreciated service provided by their hosts. The vBSDcon FreeBSD DevSummit had a strong focus on OpenZFS, the build system and networking with the FreeBSD 12 wish list of features in mind. How to best incorporate the steady flow of new OpenZFS features into FreeBSD such as dataset-level encryption was of particular interest. This feature from a GNU/Linux-based storage vendor is tribute to the growth of the OpenZFS community which is vital in light of the recent “Death of Solaris and ZFS” at Oracle. There has never been more demand for OpenZFS on FreeBSD and the Oracle news further confirms our collective responsibility to meet that demand. The official conference opened with my talk on “Isolated BSD Build Environments” in which I explained how the bhyve hypervisor can be used to effortlessly tour FreeBSD 5.0-onward and build specific source releases on demand to trace regressions to their offending commit. I was followed by a FreeNAS user who made the good point that FreeNAS is an exemplary “entry vector” into Unix and Enterprise Storage fundamentals, given that many of the vectors our generation had are gone. Where many of us discovered Unix and the Internet via console terminals at school or work, smart phones are only delivering the Internet without the Unix. With some irony, both iOS and Android are Unix-based yet offer few opportunities for their users to learn and leverage their Unix environments. The next two talks were The History and Future of Core Dumps in FreeBSD by Sam Gwydir and Using pkgsrc for multi-platform deployments in heterogeneous environments by G. Clifford Williams. I strongly recommend that anyone wanting to speak at AsiaBSDCon read Sam's accompanying paper on core dumps because I consider it the perfect AsiaBSDCon topic and his execution is excellent. Core dumps are one of those things you rarely think about until they are a DROP EVERYTHING! priority. G. Clifford's talk was about what I consider a near-perfect BSD project: pkgsrc, the portable BSD package manager. I put it up there with OpenSSH and mandoc as projects that have provided significant value to other Open Source operating systems. G. Clifford's real-world experiences are perfectly inline with vBSDcon's goal to be more production-oriented than other BSDCons. Of the other talks, any and all Dtrace talks are always appreciated and George Neville-Neil's did not disappoint. He based it on his experiences with the Teach BSD project which is bringing FreeBSD-based computer science education to schools around the world. The security-related talks by John-Mark Gurney, Dean Freeman and Michael Shirk also represented vBSDcon's consideration of the local community and made a convincing point that the BSDs should make concerted efforts to qualify for Common Criteria, FIPS, and other Government security requirements. While some security experts will scoff at these, they are critical to the adoption of BSD-based products by government agencies. BSD Now hosts Allan Jude and Benedict Reuschling hosted an OpenZFS BoF and Ansible talk respectively and I hosted a bhyve hypervisor BoF. The Hallway Track and food at vBSDcon were excellent and both culminated with an after-dinner dramatic reading of Michael W. Lucas' latest book that raised money for the FreeBSD Foundation. A great time was had by all and it was wonderful to see everyone! News Roundup FreeBSD 10.4-RC2 Available (https://lists.freebsd.org/pipermail/freebsd-stable/2017-September/087848.html) FreeBSD 10.4 will be released soon, this is the last chance to find bugs before the official release is cut. Noteworthy Changes Since 10.4-RC1: Given that the amd64 disc1 image was overflowing, more of the base components installed into the disc1 (live) file systems had to be disabled. Most notably, this removed the compiler toolchain from the disc1 images. All disabled tools are still available with the dvd1 images, though. The aesni(4) driver now no longer shares a single FPU context across multiple sessions in multiple threads, addressing problems seen when employing aesni(4) for ipsec(4). Support for netmap(4) by the ixgbe(4) driver has been brought into line with the netmap(4) API present in stable/10. Also, ixgbe(4) now correctly handles VFs in its netmap(4) support again instead of treating these as PFs. During the creation of amd64 and i386 VM images, etcupdate(8) and mergemaster(8) databases now are bootstrapped, akin to what happens along the extraction of base.txz as part of a new installation via bsdinstall(8). This change allows for both of these tools to work out-of-box on the VM images and avoids errors seen when upgrading these images via freebsd-update(8). If you are still on the stable/10 branch, you should test upgrading to 10.4, and make sure there are no problems with your workload Additional testing specifically of the features that have changed since 10.4-BETA1 would also be most helpful This will be the last release from the stable/10 branch *** OpenBSD changes of note 628 (https://www.tedunangst.com/flak/post/openbsd-changes-of-note-628) EuroBSDCon in two weeks. Be sure to attend early and often. Many and various documentation improvements for libcrypto. New man pages, rewrites, expanded bugs sections, and more. Only allow upward migration in vmd. There's a README for the syspatch build system if you want to run your own. Move the kernel relinking code from /etc/rc into a seperate script usable by syspatch. Kernel patches can now be reduced to just the necessary files. Make the callers of sogetopt() responsible for allocating memory. Now allocation and free occur in the same place. Use waitpid() instead of wait() in most programs to avoid accidentally collecting the wrong child. Have cu call isatty() before making assumptions. Switch mandoc rendering of mathematical symbols and greek letters from trying to imitate the characters' graphical shapes, which resulted in unintelligible renderings in many cases, to transliterations conveying the characters' meanings. Update libexpat to 2.2.4. Fix copying partial UTF-8 characters. Sigh, here we go again. Work around bug in F5's handling of the supported elliptic curves extension. RFC 4492 only defines elliptic_curves for ClientHello. However, F5 is sending it in ServerHello. We need to skip over it since our TLS extension parsing code is now more strict. After a first install, run syspatch -c to check for patches. If SMAP is present, clear PSL_AC on kernel entry and interrupt so that only the code in copy{in,out}* that need it run with it set. Panic if it's set on entry to trap() or syscall(). Prompted by Maxime Villard's NetBSD work. Errata. New drivers for arm: rktemp, mvpinctrl, mvmpic, mvneta, mvmdio, mvpxa, rkiic, rkpmic. No need to exec rm from within mandoc. We know there's exactly one file and directory to remove. Similarly with running cmp. Revert to Mesa 13.0.6 to hopefully address rendering issues a handful of people have reported with xpdf/fvwm on ivy bridge with modesetting driver. Rewrite ALPN extension using CBB/CBS and the new extension framework. Rewrite SRTP extension using CBB/CBS and the new extension framework. Revisit 2q queue sizes. Limit the hot queue to 1/20th the cache size up to a max of 4096 pages. Limit the warm and cold queues to half the cache. This allows us to more effectively notice re-interest in buffers instead of losing it in a large hot queue. Add glass console support for arm64. Probably not yet for your machine, though. Replace heaps of hand-written syscall stubs in ld.so with a simpler framework. 65535 is a valid port to listen on. When xinit starts an X server that listens only on UNIX socket, prefer DISPLAY=unix:0 rather than DISPLAY=:0. This will prevent applications from ever falling back to TCP if the UNIX socket connection fails (such as when the X server crashes). Reverted. Add -z and -Z options to apmd to auto suspend or hibernate when low on battery. Remove the original (pre-IETF) chacha20-poly1305 cipher suites. Add urng(4) which supports various USB RNG devices. Instead of adding one driver per device, start bundling them into a single driver. Remove old deactivated pledge path code. A replacement mechanism is being brewed. Fix a bug from the extension parsing rewrite. Always parse ALPN even if no callback has been installed to prevent leaving unprocessed data which leads to a decode error. Clarify what is meant by syslog priorities being ordered, since the numbers and priorities are backwards. Remove a stray setlocale() from ksh, eliminating a lot of extra statically linked code. Unremove some NPN symbols from libssl because ports software thinks they should be there for reasons. Fix saved stack location after resume. Somehow clang changed it. Resume works again on i386. Improve error messages in vmd and vmctl to be more informative. Stop building the miniroot installer for OMAP3 Beagleboards. It hasn't worked in over a year and nobody noticed. Have the callers of sosetopt() free the mbuf for symmetry. On octeon, let the kernel use the hardware FPU even if emulation is compiled in. It's faster. Fix support for 486DX CPUs by not calling cpuid. I used to own a 486. Now I don't. Merge some drm fixes from linux. Defer probing of floppy drives, eliminating delays during boot. Better handling of probes and beacons and timeouts and scans in wifi stack to avoid disconnects. Move mutex, condvar, and thread-specific data routes, pthreadonce, and pthreadexit from libpthread to libc, along with low-level bits to support them. Let's thread aware (but not actually threaded) code work with just libc. New POSIX xlocale implementation. Complete as long as you only use ASCII and UTF-8, as you should. Round and round it goes; when 6.2 stops, nobody knows. A peak at the future? *** Screencasting with OpenBSD (http://eradman.com/posts/screencasting.html) USB Audio Any USB microphone should appear as a new audio device. Here is the dmesg for my mic by ART: uaudio0 at uhub0 port 2 configuration 1 interface 0 "M-One USB" rev 1.10/0.01 addr 2 uaudio0: audio rev 1.00, 8 mixer controls audio1 at uaudio0 audioctl can read off all of the specific characterisitcs of this device $ audioctl -f /dev/audio1 | grep record mode=play,record record.rate=48000 record.channels=1 record.precision=16 record.bps=2 record.msb=1 record.encoding=slinear_le record.pause=0 record.active=0 record.block_size=1960 record.bytes=0 record.errors=0 Now test the recording from the second audio device using aucat(1) aucat -f rsnd/1 -o file.wav If the device also has a headset audio can be played through the same device. aucat -f rsnd/1 -i file.wav Screen Capture using Xvfb The rate at which a framebuffer for your video card is a feature of the hardware and software your using, and it's often very slow. x11vnc will print an estimate of the banwidth for the system your running. x11vnc ... 09/05/2012 22:23:45 fb read rate: 7 MB/sec This is about 4fps. We can do much better by using a virtual framebuffer. Here I'm setting up a new screen, setting the background color, starting cwm and an instance of xterm Xvfb :1 -screen 0 720x540x16 & DISPLAY=:1 xsetroot -solid steelblue & DISPLAY=:1 cwm & DISPLAY=:1 xterm +sb -fa Hermit -fs 14 & Much better! Now we're up around 20fps. x11vnc -display :1 & ... 11/05/2012 18:04:07 fb read rate: 168 MB/sec Make a connection to this virtual screen using raw encoding to eliminate time wasted on compression. vncviewer localhost -encodings raw A test recording with sound then looks like this ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 ~/out.avi Note: always stop the recording and playback using q, not Ctrl-C so that audio inputs are shut down properly. Screen Capture using Xephyr Xephyr is perhaps the easiest way to run X with a shadow framebuffer. This solution also avoids reading from the video card's RAM, so it's reasonably fast. Xephyr -ac -br -noreset -screen 800x600 :1 & DISPLAY=:1 xsetroot -solid steelblue & DISPLAY=:1 cwm & DISPLAY=:1 xrdb -load ~/.Xdefaults & DISPLAY=:1 xterm +sb -fa "Hermit" -fs 14 & Capture works in exactally the same way. This command tries to maintain 12fps. ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi To capture keyboard and mouse input press Ctrl then Shift. This is very handy for using navigating a window manager in the nested X session. Arranging Windows I have sometimes found it helpful to launch applications and arrange them in a specific way. This will open up a web browser listing the current directory and position windows using xdotool DISPLAY=:1 midori "file:///pwd" & sleep 2 DISPLAY=:1 xdotool search --name "xterm" windowmove 0 0 DISPLAY=:1 xdotool search --class "midori" windowmove 400 0 DISPLAY=:1 xdotool search --class "midori" windowsize 400 576 This will position the window precisely so that it appears to be in a tmux window on the right. Audio/Video Sync If you find that the audio is way out of sync with the video, you can ajust the start using the -ss before the audio input to specify the number of seconds to delay. My final recording command line, that delays the audio by 0.5 seconds, writing 12fps ffmpeg -ss 0.5 -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi Sharing a Terminal with tmux If you're trying to record a terminal session, tmux is able to share a session. In this way a recording of an X framebuffer can be taken without even using the screen. Start by creating the session. tmux -2 -S /tmp/tmux0 Then on the remote side connect on the same socket tmux -2 -S /tmp/tmux0 attach Taking Screenshots Grabbing a screenshots on Xvfb server is easily accomplished with ImageMagick's import command DISPLAY=:1 import -window root screenshot.png Audio Processing and Video Transcoding The first step is to ensure that the clip begins and ends where you'd like it to. The following will make a copy of the recording starting at time 00:00 and ending at 09:45 ffmpeg -i interactive-sql.avi -vcodec copy -acodec copy -ss 00:00:00 -t 00:09:45 interactive-sql-trimmed.avi mv interactive-sql-trimmed.avi interactive-sql.avi Setting the gain correctly is very important with an analog mixer, but if you're using a USB mic there may not be a gain option; simply record using it's built-in settings and then adjust the levels afterwards using a utility such as normalize. First extact the audio as a raw PCM file and then run normalize ffmpeg -i interactive-sql.avi -c:a copy -vn audio.wav normalize audio.wav Next merge the audio back in again ffmpeg -i interactive-sql.avi -i audio.wav -map 0:0 -map 1:0 -c copy interactive-sql-normalized.avi The final step is to compress the screencast for distribution. Encoding to VP8/Vorbis is easy: ffmpeg -i interactive-sql-normalized.avi -c:v libvpx -b:v 1M -c:a libvorbis -q:a 6 interactive-sql.webm H.264/AAC is tricky. For most video players the color space needs to be set to yuv420p. The -movflags puts the index data at the beginning of the file to enable streaming/partial content requests over HTTP: ffmpeg -y -i interactive-sql-normalized.avi -c:v libx264 -preset slow -crf 14 -pix_fmt yuv420p -movflags +faststart -c:a aac -q:a 6 interactive-sql.mp4 TrueOS @ Ohio Linuxfest '17! (https://www.trueos.org/blog/trueos-ohio-linuxfest-17/) Dru Lavigne and Ken Moore are both giving presentations on Saturday the 30th. Sit in and hear about new developments for the Lumina and FreeNAS projects. Ken is offering Lumina Rising: Challenging Desktop Orthodoxy at 10:15 am in Franklin A. Hear his thoughts about the ideas propelling desktop environment development and how Lumina, especially Lumina 2, is seeking to offer a new model of desktop architecture. Elements discussed include session security, application dependencies, message handling, and operating system integration. Dru is talking about What's New in FreeNAS 11 at 2:00 pm in Franklin D. She'll be providing an overview of some of the new features added in FreeNAS 11.0, including: Alert Services Starting specific services at boot time AD Monitoring to ensure the AD service restarts if disconnected A preview of the new user interface support for S3-compatible storage and the bhyve hypervisor She's also giving a sneak peek of FreeNAS 11.1, which has some neat features: A complete rewrite of the Jails/Plugins system as FreeNAS moves from warden to iocage Writing new plugins with just a few lines of code A brand new asynchronous middleware API Who's going? Attending this year are: Dru Lavigne (dlavigne): Dru leads the technical documentation team at iX, and contributes heavily to open source documentation projects like FreeBSD, FreeNAS, and TrueOS. Ken Moore (beanpole134): Ken is the lead developer of Lumina and a core contributor to TrueOS. He also works on a number of other Qt5 projects for iXsystems. J.T. Pennington (q5sys): Some of you may be familiar with his work on BSDNow, but J.T. also contributes to the TrueOS, Lumina, and SysAdm projects, helping out with development and general bug squashing. *** Beastie Bits Lumina Development Preview: Theme Engine (https://www.trueos.org/blog/lumina-development-preview-theme-engine/) It's happening! Official retro Thinkpad lappy spotted in the wild (https://www.theregister.co.uk/2017/09/04/retro_thinkpad_spotted_in_the_wild/) LLVM libFuzzer and SafeStack ported to NetBSD (https://blog.netbsd.org/tnf/entry/llvm_libfuzzer_and_safestack_ported) Remaining 2017 FreeBSD Events (https://www.freebsdfoundation.org/news-and-events/event-calendar/2017-openzfs-developer-summit/) *** Feedback/Questions Andrew - BSD Teaching Material (http://dpaste.com/0YTT0VP) Seth - Switching to Tarsnap after Crashplan becomes no more (http://dpaste.com/1SK92ZX#wrap) Thomas - Native encryption in ZFS (http://dpaste.com/02KD5FX#wrap) Coding Cowboy - Coding Cowboy - Passwords and clipboards (http://dpaste.com/31K0E40#wrap) ***
© 2020-2025 Ivy Podcast Discovery LLC
