Podcasts about bsdcan

The Do-Not-Stab flag in the HTTP Header, FreeBSD jail host with multiple local networks, Generative AI is for the idea guys, Static dual stack networking on OmniOS Solaris Zones, FRAME sockets added to OpenBSD, The problem with combining DNS CNAME records and anything else, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines (due to excessive use of the F-bomb, perhaps we should somewhat censor it... You can do so in words... or I can use Tom's favorite Frequency tone to do it in post). You decide and let me know what you think would be funnier.) Also I'm hoping for some good commentary from you guys on this one. :P The Do-Not-Stab flag in the HTTP Header (https://www.5snb.club/posts/2023/do-not-stab/) FreeBSD jail host with multiple local networks (https://savagedlight.me/2014/03/07/freebsd-jail-host-with-multiple-local-networks/) News Roundup Generative AI is for the idea guys (https://rachsmith.com/ai-is-for-the-idea-guys/) Static dual stack networking on OmniOS Solaris Zones (https://www.tumfatig.net/2024/static-dual-stack-networking-on-omnios-solaris-zones/) FRAME sockets added to OpenBSD (https://www.undeadly.org/cgi?action=article;sid=20241219080430) The problem with combining DNS CNAME records and anything else (https://utcc.utoronto.ca/~cks/space/blog/tech/DNSCNAMEAndOthersWhyNot) Conference Bits BSD-NL (https://bsdnl.nl/) BSDCan (https://www.bsdcan.org/2025/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)

Security Audit of the Capsicum and bhyve Subsystems, ZFS on Linux and block IO limits show some limits of being out of the kernel, NetBSD on a ROCK64 Board, Domain Naming, BSDCan 2025 CFP, The Internet Gopher from Minnesota, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Roundup Storage and Network Diagnostics (https://klarasystems.com/articles/winter_2024_roundup_storage_and_network_diagnostics/?utm_source=BSD%20Now&utm_medium=Podcast) Security Audit of the Capsicum and bhyve Subsystems (https://freebsdfoundation.org/wp-content/uploads/2024/11/2024_Code_Audit_Capsicum_Bhyve_FreeBSD_Foundation.pdf) News Roundup ZFS on Linux and block IO limits show some limits of being out of the kernel (https://utcc.utoronto.ca/~cks/space/blog/linux/ZFSOnLinuxVersusBlockIOLimits) NetBSD on a ROCK64 Board (https://simonevellei.com/blog/posts/netbsd-on-a-rock64-board/) Domain Naming (https://ambient.institute/domain-naming/) BSDCan 2025 CFP (https://www.bsdcan.org/2025/papers.html) The Internet Gopher from Minnesota (https://www.abortretry.fail/p/the-internet-gopher-from-minnesota) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Brendan - MinIO (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/594/feedback/Brendan%20-%20minio.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)

Limiting Process Priority in a FreeBSD Jail, Why You Should Use FreeBSD, The web fun fact that domains can end in dots and canonicalization failures, Replacing postfix with dma + auth, modern unix tool list, Smol KVM, The Computers of Voyager NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Tips and Tricks: Limiting Process Priority in a FreeBSD Jail (https://it-notes.dragas.net/2024/07/11/limiting-process-priority-in-freebsd-jail/) Why You Should Use FreeBSD (https://freebsdfoundation.org/blog/why-you-should-use-freebsd/) News Roundup The web fun fact that domains can end in dots and canonicalization failures (https://utcc.utoronto.ca/~cks/space/blog/web/DomainDotsAndCanonicalization) Replacing postfix with dma + auth (https://dan.langille.org/2024/08/02/replacing-postfix-with-dma-auth/) modern unix tool list (https://notes.billmill.org/computer_usage/cli_tips_and_tools/modern_unix_tool_list.html) Smol KVM (https://adventurist.me/posts/00324) The Computers of Voyager (https://hackaday.com/2024/05/06/the-computers-of-voyager/) Beastie Bits No unmodified files remain from original import of OpenBSD (https://www.undeadly.org/cgi?action=article;sid=20240824114631) The BSDCan 2024 Playlist is now complete (https://www.undeadly.org/cgi?action=article;sid=20240814053159) UDP parallel input committed to -current (http://undeadly.org/cgi?action=article;sid=20240727110501) Your browser is your Computer (https://www.exaequos.com) For the member-berries (https://defrag98.com) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)

My personal BSDCan Devsummit and Schedule, Syncthing, Paperless-ngx, neovim, Things we always remind ourselves while coding, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines FreeBSD Devsummit 2024 Schedule (https://freebsdfoundation.org/news-and-events/event-calendar/may-2024-freebsd-developer-summit/) BSDCan 2024 Schedule (https://indico.bsdcan.org/event/1/timetable/?#20240531.detailed) News Roundup A list of things I was drawn deeper into, got excited about, and wanted to tell you more about. Syncthing (https://syncthing.net) Paperless-ngx (https://docs.paperless-ngx.com) FreeBSD ports man page (https://man.freebsd.org/cgi/man.cgi?query=paperless&apropos=0&sektion=0&manpath=FreeBSD+14.0-RELEASE+and+Ports&arch=default&format=html) Neovim (https://neovim.io) List of popular plugins and themes (https://neovimcraft.com) Neovim for Newbs (by the Typecraft guy) (https://www.youtube.com/playlist?list=PLsz00TDipIffreIaUNk64KxTIkQaGguqn) Josean Martinez does a step by step tutorial (https://www.youtube.com/watch?v=6pAG3BHurdM&list=PLnu5gT9QrFg36OehOdECFvxFFeMHhb_07&index=11&pp=iAQB) Blog post about the setup (https://www.josean.com/posts/how-to-setup-neovim-2024) TJ DeVries (Neovim developer) reads the entire manual in 9:27:42 (https://www.youtube.com/watch?v=rT-fbLFOCy0) Things we always remind ourselves while coding (https://changelog.com/posts/things-we-always-remind-ourselves-while-coding) Beastie Bits Me giving a ZFS intro talk, Sci-fi style (German) (https://media.ccc.de/v/fsck-2024-66-disk-space-the-final-frontier-) Gulaschprogrammiernacht (GPN) 22 (some English talks, but most in German) (https://cfp.gulas.ch/gpn22/schedule/) A RAM-disk based workflow (https://people.freebsd.org/~dch/posts/2014-09-05-a-ramdisk-based-workflow/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)

NetBSD 9.4, FreeBSD SSDF Attestation to Support Cybersecurity Compliance, The Lost Worlds of Telnet, alter file ownership and permissions with a feedback information, parallel raw IP input, OpenBSD routers on AliExpress mini PCs, FreeBSD for Devs. Plus a special interview with the organizers of BSDCAN 2024. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines NetBSD 9.4 (https://www.netbsd.org/releases/formal-9/NetBSD-9.4.html) FreeBSD Foundation Delivers V1 of FreeBSD SSDF Attestation to Support Cybersecurity Compliance (https://freebsdfoundation.org/blog/freebsd-foundation-delivers-v1-of-freebsd-ssdf-attestation-to-support-cybersecurity-compliance/) News Roundup The Lost Worlds of Telnet (https://thenewstack.io/the-lost-worlds-of-telnet/) How to alter file ownership and permissions with a feedback information (https://sleeplessbeastie.eu/2024/04/18/how-to-alter-file-ownership-and-permissions-with-a-feedback-information/) Coming soon to a -current system near you: parallel raw IP input (https://www.undeadly.org/cgi?action=article;sid=20240418050520) OpenBSD routers on AliExpress mini PCs (https://www.srcbeat.com/2024/02/aliexpress-openbsd-router/) FreeBSD for Devs (https://dev.to/scovl/freebsd-for-devs-3n0k) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Daniel - jail issue (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/556/feedback/Daniel%20-%20jail%20issue.md) Rick - ZFS (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/556/feedback/Rick%20-%20ZFS.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)

GPL 3: The Controversial Licensing Model and Potential Solutions, The Geeks way of checking what the outside weather is like, Alpine on a FreeBSD Jail, DragonFly BSD on a Thinkpad T480s, Dealing with USB Storage devices on OmniOS, Creating a Time Capsule instance using Samba, FreeBSD, and ZFS NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines GPL 3: The Controversial Licensing Model and Potential Solutions (https://klarasystems.com/articles/gpl-3-the-controversial-licensing-model-and-potential-solutions/) The Geeks way of checking what the outside wheather is like (https://blog.netbsd.org/tnf/entry/the_geeks_way_of_checking) News Roundup Alpine on a FreeBSD Jail (https://it-notes.dragas.net/2024/01/18/installing-alpine-linux-on-a-freebsd-jail/) DragonFly BSD on a Thinkpad T480s (https://git.sr.ht/~tomh/dragonflybsd-on-a-laptop/tree/master/item/README.md) Dealing with USB Storage devices on OmniOS (https://www.tumfatig.net/2024/dealing-with-usb-storage-devices-on-omnios/) Creating a Time Capsule instance using Samba, FreeBSD, and ZFS (https://dan.langille.org/2024/01/06/creating-a-time-capsule-instance-using-samba-freebsd-and-zfs-2/) Conferences FOSDEM (https://fosdem.org/2024/) AsiaBSDCon (https://2024.asiabsdcon.org/program.html) BSDCan (https://www.bsdcan.org/2024/papers.php) EuroBSDcon (https://2024.eurobsdcon.org/) Southeast Linuxfest (https://southeastlinuxfest.org/2024/01/self-2024-call-for-participation/) Dont let the name fool you, SELF is BSD friendly and they'd love to have BSD/Unix Talks if you're in the area. JT is staff at SELF, so he can put in a good word for you. ;) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)

Microsoft's rudimentary error that allowed an attacker access to its executives' emails, Pixel phones have another serious storage bug, hidden malware payload found at Ars Technica, and when to upgrade your hardware for Windows 11.   Plugs Support us on patreon and get an ad-free RSS feed with early episodes sometimes BSDCan 2024 – Call […]

Microsoft's rudimentary error that allowed an attacker access to its executives' emails, Pixel phones have another serious storage bug, hidden malware payload found at Ars Technica, and when to upgrade your hardware for Windows 11.   Plugs Support us on patreon and get an ad-free RSS feed with early episodes sometimes BSDCan 2024 – Call... Read More

FreeBSD on the RISC-V Architecture, A bit of XENIX history, pkgbase: Official packages, recover lost text by coredumping firefox, FuguIta 7.4 has been released, LibreSSL 3.8.2 Released, OpenSMTPD 7.4.0p0 Released NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Looking Towards the Future: FreeBSD on the RISC-V Architecture (https://klarasystems.com/articles/looking-towards-the-future-freebsd-on-the-risc-v-architecture/) A bit of XENIX history (http://seefigure1.com/2014/04/15/xenixtime.html) News Roundup Official packages (https://lists.freebsd.org/archives/freebsd-pkgbase/2023-October/000221.html) recover lost text by coredumping firefox (https://j3s.sh/thought/recover-lost-text-by-coredumping-firefox.html) FuguIta 7.4 has been released (https://fuguita.org/?FuguIta/7.4&utm_source=bsdweekly) LibreSSL 3.8.2 Released (https://undeadly.org/cgi?action=article;sid=20231103065952) OpenSMTPD 7.4.0p0 Released (https://undeadly.org/cgi?action=article;sid=20231026121132) Conference News AsiaBSDCon 2024 (https://2024.asiabsdcon.org) BSDCan 2024 (https://www.bsdcan.org) EuroBSDCon 2024 (https://2024.eurobsdcon.org) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)

AsiaBSDCon 2023 Trip Report, Converting My X201 ThinkPad into a Slabtop, Stream your OpenBSD desktop audio to other devices, The Gnome and Its "Secret Place", ttyload, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines AsiaBSDCon 2023 Trip Report (https://freebsdfoundation.org/blog/asiabsdcon-2023-trip-report/) Converting My X201 ThinkPad into a Slabtop (https://bt.ht/slabtop/) News Roundup Stream your OpenBSD desktop audio to other devices (https://dataswamp.org/~solene/2023-05-05-openbsd-sound-streaming.html) The Gnome and Its "Secret Place" (https://www.tuhs.org/pipermail/tuhs/2023-May/028363.html) ttyload - Linux/Unix color-coded graphical tracking tool for load average in a terminal (https://www.cyberciti.biz/open-source/command-line-hacks/ttyload-color-coded-graphical-tracking-tool-for-unixlinux-load-average-in-a-terminal/) Beastie Bits • [OpenIndiana with a Sun Microsystems 22" LCD monitor. Running on a 1.8GHz quad core AMD Phenom 9100e processor, 4Gb RAM, nVidia GEForce GT630.](https://www.reddit.com/r/unix/comments/13otjnt/openindiana_with_a_sun_microsystems_22_lcd/) • [cron(8) now supports random ranges with steps](https://www.undeadly.org/cgi?action=article;sid=20230507122935&utm_source=bsdweekly) • [BSDCan 2024 Reorganization](https://mwl.io/archives/22799) • [Depenguin me](https://depenguin.me/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)

Comparing Modern Open-Source Storage Solutions, FreeBSD Q1 Status Report, Hello Systems 0.8.1 Release, OpenBSD: Managing an inverter/converter with NUT, Tips for Running a Greener FreeBSD, BSDCAN Registration open NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Comparing Modern Open-Source Storage Solutions OpenZFS vs. The Rest (https://klarasystems.com/articles/openzfs-comparing-modern-open-source-storage-solutions/) FreeBSD Q1 Status Report (https://www.freebsd.org/status/report-2023-01-2023-03/) News Roundup Hello Systems 0.8.1 Release (https://github.com/helloSystem/ISO/releases/tag/r0.8.1) OpenBSD: Managing an inverter/converter with NUT (https://doc.huc.fr.eu.org/en/sys/openbsd/nut/) Celebrating Earth Day: Tips for Running a Greener FreeBSD (https://freebsdfoundation.org/blog/celebrating-earth-day-tips-for-running-a-greener-freebsd/) BSDCAN Registration (https://www.bsdcan.org/2023/registration.php) Beastie Bits • [SimCity 2000 running on OpenBSD 7.3 via DOSBox 0.74-3](https://www.reddit.com/r/openbsd_gaming/comments/12k9zt2/simcity_2000_running_on_openbsd_73_via_dosbox_0743/) • [OpenBSD Webzine #13](https://webzine.puffy.cafe/issue-13.html) • [AWS Gazo bot](https://github.com/csaltos/aws-gazo-bot) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)

OpenBSD 7.3 released, Accelerating Datacenter Energy Efficiency by Leveraging FreeBSD as Your Server OS, install Cinnamon as a Desktop environment, xmonad FreeBSD set up from scratch, Burgr books in your terminal, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines OpenBSD 7.3 released (http://undeadly.org/cgi?action=article;sid=20230410140049) BSDCan 2023 Schedule posted (https://www.bsdcan.org/events/bsdcan_2023/schedule/) Accelerating Datacenter Energy Efficiency by Leveraging FreeBSD as Your Server OS (https://klarasystems.com/articles/accelerating-datacenter-energy-efficiency-by-leveraging-freebsd-as-your-server-os/) News Roundup FreeBSD – How to install Cinnamon as a Desktop environment (https://byte-sized.de/linux-unix/freebsd-cinnamon-als-gui-installieren/#english) xmonad FreeBSD set up from scratch (https://forums.FreeBSD.org/threads/xmonad-freebsd-set-up-from-scratch.75911/) Burgr books in your terminal (https://blubsblog.bearblog.dev/burgr-books-in-your-terminal/) Pros and Cons of FreeBSD for virtual Servers (https://www.hostzealot.com/blog/about-vps/pros-and-cons-of-freebsd-for-virtual-servers) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Reese - Dans Interview (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/505/feedback/Reese%20-%20Dans%20Interview.md) jj - looking for help (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/505/feedback/jj%20-%20looking%20for%20help.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***

Nextcloud + OpenBSD =

We're interviewing Dan Langille about his new server project. He'll talk to us about the things he's building, some of which are a bit out of the ordinary. We're also talking about BSDCan 2023 and what to expect after returning to an in-presence conference format. Enjoy! NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Interview - Dan Langille - dan@langille.org (mailto:dan@langille.org) / @twitter (https://twitter.com/dlangille) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Special Guest: Dan Langille.

OpenZFS auditing for storage Performance, Privilege drop; privilege separation; and restricted-service operating mode in OpenBSD, OPNsense 23.1.1 release, Cloning a System with Ansible, FOSDEM 2023, BSDCan 2023 Travel Grants NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines OpenZFS auditing for storage Performance (https://klarasystems.com/articles/openzfs-auditing-for-storage-performance/) Privilege drop, privilege separation, and restricted-service operating mode in OpenBSD (https://sha256.net/privsep.html) News Roundup OPNsense 23.1.1 released (https://forum.opnsense.org/index.php?topic=32484.0) Cloning a System with Ansible (https://kernelpanic.life/software/cloning-a-system-with-ansible.html) FOSDEM 2023 (http://blog.netbsd.org/tnf/entry/fosdem_2023) BSDCan 2023 Travel Grant Application Now Open (https://freebsdfoundation.org/blog/bsdcan-2023-travel-grant-application-now-open/) The Undeadly Bits Game of Trees milestone (http://undeadly.org/cgi?action=article;sid=20230120073530) Game of Trees Daemon - video and slides (May make the older game of trees obsolete) (http://undeadly.org/cgi?action=article;sid=20230210065830) amd64 execute-only committed to -current (http://undeadly.org/cgi?action=article;sid=20230121125423) Using /bin/eject with USB flash drives (http://undeadly.org/cgi?action=article;sid=20230214061952) Tunneling vxlan(4) over WireGuard wg(4) (http://undeadly.org/cgi?action=article;sid=20230214061330) Console screendumps (http://undeadly.org/cgi?action=article;sid=20230128183032) Execute-only status report (http://undeadly.org/cgi?action=article;sid=20230130061324) OpenBSD in Canada (http://undeadly.org/cgi?action=article;sid=20230226065006) Privilege drop, privilege separation, and restricted-service operating mode in OpenBSD (http://undeadly.org/cgi?action=article;sid=20230219234206) Theo de Raadt on pinsyscall(2) (http://undeadly.org/cgi?action=article;sid=20230222064027) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Kevin - PLUG (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/498/feedback/Kevin%20-%20PLUG.md) Luna - FOSDEM (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/498/feedback/Luna%20-%20FOSDEM.md) *** Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***

FreeBSD Foundation's Software Development review of 2022, what can we learn from Vintage Computing, OpenBSD KDE Status Report 2022, a Decade of HardenedBSD, In Praise of Plan9, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines 2022 in Review: Software Development (https://freebsdfoundation.org/blog/2022-in-review-software-development/) What can we learn from Vintage Computing (https://github.com/readme/featured/vintage-computing) News Roundup OpenBSD KDE Status Report 2022 (https://www.sizeofvoid.org/posts/2022-26-12-openbsd-kde-status-report-2022/) A Decade of HardenedBSD (https://git.hardenedbsd.org/shawn.webb/articles/-/blob/master/hardenedbsd/2023-01_decade/article.md) In Praise of Plan9 (https://drewdevault.com/2022/11/12/In-praise-of-Plan-9.html) Beastie Bits LibreSSL 3.7.0 Released (https://undeadly.org/cgi?action=article;sid=20221212183516) OPNsense 22.7.10 released (https://opnsense.org/opnsense-22-7-10-released/) BSDCan 2023 call for papers (https://lists.bsdcan.org/pipermail/bsdcan-announce/2022-December/000194.html) How to lock OpenSSH authentication agent (https://sleeplessbeastie.eu/2022/12/28/how-to-lock-openssh-authentication-agent/) Once upon a time long ago, I was sitting alone in the UCLA ARPANET site... (https://mastodon.laurenweinstein.org/@lauren/109588605178700335) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***

Evaluating FreeBSD CURRENT for Production Use, Time Machine-like Backups on OpenBSD, FreeBSD on the Graviton 3, Compiling the NetBSD kernel as a benchmark, Network Management with the OpenBSD Packet Filter Toolset from BSDCan 2022, Hardware Detection & Diagnostics for New FreeBSD Users, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Evaluating FreeBSD CURRENT for Production Use (https://klarasystems.com/articles/evaluating-freebsd-current-for-production-use/) Time Machine like Backups on OpenBSD (https://xosc.org/timemachine.html) News Roundup FreeBSD on the Graviton 3 (https://www.daemonology.net/blog/2022-05-23-FreeBSD-Graviton-3.html) Compiling the NetBSD kernel as a benchmark (https://blog.anotherhomepage.org/post/2022/05/25/Compiling-the-NetBSD-kernel-as-a-benchmark/) Network Management with the OpenBSD Packet Filter Toolset from BSDCan 2022 (http://undeadly.org/cgi?action=article;sid=20220607112236) Hardware Detection & Diagnostics for New FreeBSD Users & PCs (https://forums.FreeBSD.org/threads/hardware-detection-diagnostics-for-new-freebsd-users-pcs.84596/) Beastie Bits • [NetBSD - Announcing Google Summer of Code 2022 projects](https://blog.netbsd.org/tnf/entry/announcing_google_summer_of_code3) • [Welcome FreeBSD Google Summer of Code Participants](https://freebsdfoundation.org/blog/welcome-freebsd-google-summer-of-code-participants/) • [Network from Scratch](https://www.networksfromscratch.com) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***

Fundamentals of the FreeBSD Shell, Spammers in the Public Cloud, locking user accounts properly, overgrowth on NetBSD, moreutils, ctwm & spleen, interpreting a traceroute, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Fundamentals of the FreeBSD Shell (https://klarasystems.com/articles/interacting-with-freebsd-learning-the-fundamentals-of-the-freebsd-shell-2/) Spammers in the Public Cloud, Protected by SPF; Intensified Password Groping Still Ongoing; Spamware Hawked to Spamtraps (https://bsdly.blogspot.com/2022/04/spammers-in-public-cloud-protected-by.html) News Roundup A cautionary tale about locking Linux & FreeBSD user accounts (https://www.cyberciti.biz/networking/a-cautionary-tale-about-locking-linux-freebsd-user-accounts/) Overgrowth runs on NetBSD (https://www.reddit.com/r/openbsd_gaming/comments/ucgavg/i_was_able_to_build_overgrowth_on_netbsd/) moreutils (https://joeyh.name/code/moreutils/) NetBSD, CTWM, and Spleen (https://www.cambus.net/netbsd-ctwm-and-spleen/) How to properly interpret a traceroute or mtr (https://phil.lavin.me.uk/2022/03/how-to-properly-interpret-a-traceroute-or-mtr/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Lets talk a bit about some of the events happening this year, BSDCan in virtual this weekend, emfcamp is this weekend too and in person, MCH is this summer and eurobsdcon is in september. How were the postgres conferences benedict? Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***

Dan Langille Dan is a sysadmin at Cisco's Talos Intelligence Group, He's the organizer of BSDCan, the man behind the FreeBSD Diary, Fresh Ports, and Fresh Source. Twitter: https://twitter.com/DLangille LinkedIn: https://www.linkedin.com/in/danlangille/ Notes: Sites Dan Runs: https://www.freebsddiary.org https://www.freshports.org https://www.freshsource.org https://www.bsdcan.org/ https://www.langille.org Credits: Music by ikson: https://www.iksonmusic.com Special Guest: Dan Langille.

AsiaBSDcon review, Meltdown and Spectre Patches in FreeBSD stable, Interview with MidnightBSD founder, 8 months with TrueOS, mysteries of GNU and BSD split This episode was brought to you by Headlines AsiaBSDCon 2018 has concluded (https://2018.asiabsdcon.org/) We have just returned from AsiaBSDCon in Tokyo, Japan last weekend Please excuse our jetlag The conference consisted two days of meeting followed by 2 days of paper presentations We arrived a few days early to see some sights and take a few extra delicious meals in Tokyo The first day of meetings was a FreeBSD developer summit (while Benedict was teaching his two tutorials) where we discussed the FreeBSD release cycle and our thoughts on improving it, the new Casper capsicum helper service, and developments in SDIO which will eventually enable WiFi and SD card readers on more embedded devices The second day of meetings consisted of bhyvecon, a miniconf that covered development in all hypervisors on all BSDs. It also included presentations on the porting of bhyve to IllumOS. Then the conference started There were a number of great presentations, plus an amazing hallway track as usual It was great to see many old friends and to spend time discussing the latest happenings in BSD. A couple of people came by and asked to take a picture with us and we were happy to do that. *** FreeBSD releases Spectre and Meltdown mitigations for 11.1 (https://www.freebsd.org/security/advisories/FreeBSD-SA-18:03.speculative_execution.asc) Speculative execution vulnerability mitigation is a work in progress. This advisory addresses the most significant issues for FreeBSD 11.1 on amd64 CPUs. We expect to update this advisory to include 10.x for amd64 CPUs. Future FreeBSD releases will address this issue on i386 and other CPUs. freebsd-update will include changes on i386 as part of this update due to common code changes shared between amd64 and i386, however it contains no functional changes for i386 (in particular, it does not mitigate the issue on i386). Many modern processors have implementation issues that allow unprivileged attackers to bypass user-kernel or inter-process memory access restrictions by exploiting speculative execution and shared resources (for example, caches). An attacker may be able to read secret data from the kernel or from a process when executing untrusted code (for example, in a web browser). + Meltdown: The mitigation is known as Page Table Isolation (PTI). PTI largely separates kernel and user mode page tables, so that even during speculative execution most of the kernel's data is unmapped and not accessible. A demonstration of the Meltdown vulnerability is available at https://github.com/dag-erling/meltdown. A positive result is definitive (that is, the vulnerability exists with certainty). A negative result indicates either that the CPU is not affected, or that the test is not capable of demonstrating the issue on the CPU (and may need to be modified). A patched kernel will automatically enable PTI on Intel CPUs. The status can be checked via the vm.pmap.pti sysctl PTI introduces a performance regression. The observed performance loss is significant in microbenchmarks of system call overhead, but is much smaller for many real workloads. + Spectre V2: There are two common mitigations for Spectre V2. This patch includes a mitigation using Indirect Branch Restricted Speculation, a feature available via a microcode update from processor manufacturers. The alternate mitigation, Retpoline, is a feature available in newer compilers. The feasibility of applying Retpoline to stable branches and/or releases is under investigation. The patch includes the IBRS mitigation for Spectre V2. To use the mitigation the system must have an updated microcode; with older microcode a patched kernel will function without the mitigation. IBRS can be disabled via the hw.ibrsdisable sysctl (and tunable), and the status can be checked via the hw.ibrsactive sysctl. IBRS may be enabled or disabled at runtime. Additional detail on microcode updates will follow. + Wiki tracking the vulnerabilities and mitigations on different platforms (https://wiki.freebsd.org/SpeculativeExecutionVulnerabilities) Interview with MidnightBSD Founder and Lead Dev Lucas Holt (https://itsfoss.com/midnightbsd-founder-lucas-holt/) Recently, I have taken a little dip into the world of BSD. As part of my attempt to understand the BSD world a little better, I connected with Lucas Holt (MidnightBSD founder and lead developer) to ask him a few questions about his project. Here are his answers. It's FOSS: Please explain MidnightBSD in a nutshell. How is it different than other BSDs? Lucas Holt: MidnightBSD is a desktop focused operating system. When it's considered stable, it will provide a full desktop experience. This differs from other efforts such as TrueOS or GhostBSD in that it's not a distro of FreeBSD, but rather a fork. MidnightBSD has its own package manager, mport as well as unique package cluster software and several features built into user land such as mDNSresponder, libdispatch, and customizations throughout the system. It's FOSS: Who is MidnightBSD aimed at? Lucas Holt: The goal with MidnightBSD has always been to provide a desktop OS that's usable for everyday tasks and that even somewhat non technical people can use. Early versions of Mac OS X were certainly an inspiration. In practice, we're rather far from that goal at this point, but it's been an excellent learning opportunity. It's FOSS: What is your background in computers? Lucas Holt: I started in technical support at a small ISP and moved into web design and system administration. While there, I learned BSDi, Solaris and Linux. I also started tinkering with programming web apps in ASP and a little perl CGI. I then did a mix of programming and system administration jobs through college and graduated with a bachelors in C.S. from Eastern Michigan University. During that time, I learned NetBSD and FreeBSD. I started working on several projects such as porting Apple's HFS+ code to FreeBSD 6 and working on getting the nforce2 chipset SATA controller working with FreeBSD 6, with the latter getting committed. I got a real taste for BSD and after seeing the lack of interest in the community for desktop BSDs, I started MidnightBSD. I began work on it in late 2005. Currently, I'm a Senior Software Engineer focusing on backend rest services by day and a part-time graduate student at the University of Michigan Flint. It's FOSS: I recently installed TrueOS. I was disappointed that a couple of the programs I wanted were not available. The FreeBSD port system looked mildly complicated for beginners. I'm used to using pacman to get the job done quickly. How does MidnightBSD deal with ports? Lucas Holt: MidnightBSD has it's own port system, mports, which shared similarities with FreeBSD ports as well as some ideas from OpenBSD. We decided early on that decent package management was essential for regular users. Power users will still use ports for certain software, but it's just so time consuming to build everything. We started work on our own package manager, mport. Every package is a tar lzma archive with a sqlite3 manifest file as well as a sqlite 3 index that's downloaded from our server. This allows users to query and customize the package system with standard SQL queries. We're also building more user friendly graphical tools. Package availability is another issue that most BSDs have. Software tends to be written for one or two operating systems and many projects are reluctant to support other systems, particularly smaller projects like MidnightBSD. There are certainly gaps. All of the BSD projects need more volunteers to help with porting software and keeping it up to date. It's FOSS: During your June 2015 interview on BSDNow, you mentioned that even though you support both i386 and amd64, that you recommend people choose amd64. Do you have any plans to drop i386 support in the future, like many have done? Lucas Holt: Yes, we do plan to drop i386 support, mostly because of the extra work needed to build and maintain packages. I've held off on this so far because I had a lot of feedback from users in South America that they still needed it. For now, the plan is to keep i386 support through 1.0 release. That's probably a year or two out. It's FOSS: What desktop environments does MidnightBSD support? Lucas Holt: The original plan was to use Etoile as a desktop environment, but that project changed focus. We currently support Xfce, Gnome 3, WindowMaker + GNUstep + Gworkspace as primary choices. We also have several other window managers and desktop environments available such as Enlightenment, rat poison, afterstep, etc. Early versions offered KDE 3.x but we had some issues with KDE 4. We may revisit that with newer versions. It's FOSS: What is MidnightBSD's default filesystem? Do you support DragonflyBSD's HAMMER filesystem? What other filesystems? Lucas Holt: Boot volumes are UFS2. We also support ZFS for additional storage. We have read support for ExFat, NTFS, ext2, CD9660. NFS v3 and v4 are also supported for network file systems. We do not support HAMMER, although it was considered. I would love to see HAMMER2 get added to MidnightBSD eventually. It's FOSS: Is MidnightBSD affected by the recent Spectre and Meltdown issues? Lucas Holt: Yes. Most operating systems were affected by these issues. We were not informed of the issue until the general public became aware. Work is ongoing to come up with appropriate mitigations. Unfortunately, we do not have a patch yet. It's FOSS: The Raspberry Pi and its many clones have made the ARM platform very popular. Are there any plans to make MidnightBSD available on that platform? Lucas Holt: No immediate plans. ARM is an interesting architecture, but by the very nature of SoC designs, takes a lot of work to support a broad number of devices. It might be possible when we stop supporting i386 or if someone volunteers to work on the ARM port. Eventually, I think most hobby systems will need to run ARM chips. Intel's planning on locking down hardware with UEFI 3 and this may make it difficult to run on commodity hardware in the future not only for MidnightBSD but other systems as well. At one point, MidinightBSD ran on sparc64. When workstations were killed off, we dropped support. A desktop OS on a server platform makes little sense. It's FOSS: Does MidnightBSD offer support for Linux applications? Lucas Holt: Yes, we offer Linux emulation. It's emulating a 2.6.16 kernel currently and that needs to be updated so support newer apps. It's possible to run semi-recent versions of Firefox, Thunderbird, Java, and OpenOffice on it though. I've also used it to host game servers in the past and play older games such as Quake 3, enemy territory, etc. It's FOSS: Could you comment on the recent dust-up between the Pale Moon browser developers and the team behind the OpenBSD ports system? [Author's Note: For those who haven't heard about this, let me summarize. Last month, someone from the OpenBSD team added the Pale Moon browser to their ports collection. A Pale Moon developer demanded that they include Pale Moon's libraries instead of using system libraries. As the conversation continued, it got more hostile, especially on the Pale Moon side. The net result is that Pale Moon will not be available on OpenBSD, MidnightBSD, or FreeBSD.] Lucas Holt: I found this discussion frustrating. Many of the BSD projects hear a lot of complaints about browser availability and compatibility. With Firefox moving to Rust, it makes it even more difficult. Then you get into branding issues. Like Firefox, the Pale Moon developers have decided to protect their brand at the cost of users. Unlike the Firefox devs, they've made even stranger requirements for branding. It is not possible to use a system library version of anything with Pale Moon and keep their branding requirements. As such, we cannot offer Pale Moon in MidnightBSD. The reason this is an issue for an open source project is that many third party libraries are used in something as complex as a web browser. For instance, Gecko-based browsers use several multimedia libraries, sqlite3 (for bookmarks), audio and video codecs, etc. Trying to maintain upstream patches for each of these items is difficult. That's why the BSDs have ports collections to begin with. It allows us to track and manage custom patches to make all these libraries work. We go through a lot of effort in keeping these up to date. Sometimes upstream patches don't get included. That means our versions are the only working copies. With pale moon's policy, we'd need to submit separate patches to their customized versions of all these libraries too and any new release of the browser would not be available as changes occur. It might not even be possible to compile pale moon without a patch locally. With regard to Rust, it requires porting the language, as well as an appropriate version of LLVM before you can even start on the browser. It's FOSS: If someone wanted to contribute to your project, both financial and technical, how can they do that? Lucas Holt: Financial assistance for the project can be submitted online. We have a page outlining how to make donations with Patreon, Paypal or via bitcoin. Donations are not tax deductible. You can learn more at http://www.midnightbsd.org/donate/ We also need assistance with translations, porting applications, and working on the actual OS. Interested parties can contact us on the mailing list or through IRC on freenode #midnightbsd We also could use assistance with mirroring ISOs and packages. I would like to thank Lucas for taking the time to reply to my many questions. For more information about MidnightBSD or to download it, please visit their website. The most recent version of MidnightBSD is 0.8.6. News Roundup 8 months with TrueOS (https://inflo.ws/blog/post/2018-03-03-trueos-8th-month-review/) Purpose of this review - what it is and what it is not. I vowed to write down what I felt about TrueOS if I ever got to the six month mark of usage. This is just that. This is neither a tutorial, nor a piece of evangelism dedicated towards it. This is also not a review of specific parts of TrueOS such as Lumina or AppCafe, since I don't use them at all. In the spirit of presenting a screen shot, here is my i3wm displaying 4 windows in one screen - a configuration that I never use. https://inflo.ws/blog/images/trues-screenshot.png The primary tasks I get done with my computer. I need a tiling wm with multi-desktop capability. As regards what I do with a computer, it is fairly straightforward to describe if I just list down my most frequently used applications. xterm (CLI) Emacs (General editing and org mode) Intellij IDEA (Java, Kotlin, SQL) Firefox (Main web browser, with Multi-Account Containers) Thunderbird (Work e-mail) Notmuchmail (Personal e-mail) Chromium/Iridium (Dumb web browser) Telegram Desktop weechat (with wee-slack) cmus (Music player) mpv (Video player) mps-youtube (Youtube client) transmission-gtk Postgresql10 (daemon) Rabbitmq (daemon) Seafile (file sync) Shotwell (manage pictures) GIMP (Edit pictures) Calibre (Manage e-books) VirtualBox All of these are available as binary packages from the repository. Since I use Intellij Ultimate edition, I decided to download the no-jdk linux version from the website rather than install it. This would make sure that it gets updated regularly. Why did I pick TrueOS ? I ran various Linux distributions from 2001 all the way till 2009, till I discovered Arch, and continued with it till 2017. I tried out Void for two months before I switched to TrueOS. Over the last few years, I started feeling like no matter which Linux distribution I touched, they all just stopped making a lot of sense. Generally in the way things were organised, and particularly in terms of software like systemd, which just got pushed down my throat. I couldn't wrap my head around half the things going on in my computer. Mostly I found that Linux distributions stopped becoming a collection of applications that got developed together to something more coupled by software mechanisms like systemd - and that process was more and more opaque. I don't want to talk about the merits and de-merits of systemd, lets just say that I found it of no use and an unnecessary hassle. In February, I found myself in charge of the entire technology stack of a company, and I was free to make choices. A friend who was a long time FreeBSD user convinced me to try it on the servers. My requirement then was to run Postgres, Rabbitmq, Nginx and a couple of JVM processes. The setup was zero hassle and it hasn't changed much in a year. About three months of running FreeBSD-11.x on servers was enough for me to consider it for my laptop. I was very apprehensive of hardware support, but luckily my computer is a Thinkpad, and Thinkpads sort of work out of the box with various BSDs. My general requirements were: Must run Intellij IDEA. Must have proper graphics and sound driver support. Must be able to run VirtualBox. I had to pick from FreeBSD, NetBSD and OpenBSD, since these were the major BSDs that I was familiar with. One of my requirements was that I needed to be able to run VMs just in case I needed to test something on Windows/Linux. This ruled out OpenBSD. Then I was left with NetBSD and FreeBSD. NetBSD's driver support for newer Intel chip-sets were questionable, and FreeBSD was the only choice then. When I was digging through FreeBSD forums, I found out that running the 11.x RELEASE on my laptop was out of the question since it didn't have proper drivers for my chip-set either. A few more hours of digging led me to GhostBSD and TrueOS. I picked TrueOS straightaway because - well because TrueOS came from the old PC-BSD and it was built off FreeBSD-12-CURRENT with the latest drivers integrated. I downloaded the UNSTABLE version available in June 2017, backed up ALL my data and home directory, and then installed it. There were no glitches during installation - I simply followed the installation as described in the handbook and everything was fine. My entire switch from Arch/Void to TrueOS took about an hour, discounting the time it took to backup my data to an external hard disk. It was that easy. Everything I wanted to work just worked, everything was available in the repo. Tweaks from cooltrainer.org : I discovered this excellent tutorial that describes setting up a FreeBSD 11 desktop. It documents several useful tweaks, some of which I applied. A few examples - Fonts, VirtualBox, Firewall, UTF-8 sections. TrueOS (and FreeBSD) specific things I liked Open-rc The open-rc init system is familiar and is well documented. TrueOS specific parts are described here. When I installed postgresql10-server, there was no open-rc script for it, but I could cobble one together in two hours with zero prior experience writing init scripts. Later on I figured out that the init script for postgresql9 would work for 10 as well, and used that. Boot Environments This was an alien concept to me, but the first time I did an update without waiting for a CDN sync to finish, my computer booted into the shell and remained there. The friendly people at TrueOS discourse asked me to roll back to an older BE and wait for sync to finish. I dug through the forums and found "ZFS / Snapshots basics & How-To's for those new to TrueOS". This describes ZFS and BEs, and is well worth reading. ZFS My experience with boot environments was enough to convince me about the utility of ZFS. I am still reading about it and trying things out, and whatever I read just convinces me more about why it is good. File-system layout Coming from the Linux world, how the FreeBSD file-system is laid out seemed odd at first. Then I realised that it was the Linux distros that were doing the odd thing. e.g : The whole OS is split into base system and applications. All the non base system configurations and apps go into /usr/local. That made a lot of sense. The entire OS is developed along with its applications as a single coherent entity, and that shows. Documentation The handbooks for both TrueOS and FreeBSD are really really good. For e.g, I kept some files in an LUKS encrypted drive (when I used Arch Linux). To find an equivalent, all I had to do was read the handbook and look at the GELI section. It is actually nice being able to go to a source like Handbook and things from there just work. Arch Linux and Gentoo has excellent documentation as well, if anyone is wondering about Linux distros. Community The TrueOS community on both Telegram as well as on Discourse are very friendly and patient. They help out a lot and do not get upset when I pose really stupid questions. TrueOS core developers hangout in the Telegram chat-room too, and it is nice being able to talk to them directly about things. What did not work in TrueOS ? The following things that worked during my Linux tenure doesn't work in TrueOS. Netflix Google Hangouts Electron based applications (Slack, Skype) These are not major concerns for the kind of work I do, so it doesn't bother me much. I run a WinXP VM to play some old games, and a Bunsenlabs installation for Linux things like Hangouts/Netflix. I don't have a video calling system setup in TrueOS because I use my phone for both voice and video calls exclusively. Why am I staying on TrueOS ? Great community - whether on Discourse or on the telegram channel, the people make you feel welcome. If things go unanswered, someone will promise to work on it/file a bug/suggest work-arounds. Switching to TrueOS was philosophical as well - I thought a lot more about licenses, and I have arrived at the conclusion that I like BSD more than GPL. I believe it is a more practical license. I believe TrueOS is improving continuously, and is a great desktop UNIX if you put some time into it. AsiaBSDCon 2016 videos now available (https://www.youtube.com/playlist?list=PLnTFqpZk5ebD-FfVScL-x6ZnZSecMA1jI) The videos from AsiaBSDCon 2016 have been posted to youtube, 30 videos in all We'll cover the videos from 2017 next week The videos from 2018 should be posted in 4-6 weeks I are working on a new version of https://papers.freebsd.org/ that will make it easier to find the papers, slides, and videos of all talks related to FreeBSD *** syspatches will be provided for both supported releases (https://undeadly.org/cgi?action=article;sid=20180307234243) Good news for people doing upgrades only once per year: syspatches will be provided for both supported releases. The commit from T.J. Townsend (tj@) speaks for itself: ``` Subject: CVS: cvs.openbsd.org: www From: T.J. Townsend Date: 2018-03-06 22:09:12 CVSROOT: /cvs Module name: www Changes by: tj@cvs.openbsd.org 2018/03/06 15:09:12 Modified files: . : errata61.html stable.html faq : faq10.html Log message: syspatches will now be provided for both supported releases. ``` Thanks to all the developers involved in providing these! Update: An official announcement has been released: ``` I'm happy to announce that we are now able to provide two releases worth of syspatches on the amd64 and i386 platforms. The binary patches for 6.1 will hit the mirrors shortly, so you will be able to catch up with the errata on https://www.openbsd.org/errata61.html using the syspatch utility. People running amd64 will thus get the meltdown workaround. This means in particular that 6.2 will remain supported by syspatch when 6.3 comes out. Thanks to robert and ajacoutot for their amazing work on syspatch and for all their help. Thanks also to tj and the volunteers from #openbsd for their timely tests and of course to Theo for overseeing it all. ``` Exploring permutations and a mystery with BSD and GNU split filenames (https://www.lorainekv.com/permutations_split_and_gsplit/) Recently, I was playing around with the split command-line tool on Mac OS X, and I decided to chop a 4000-line file into 4000 separate single-line files. However, when I attempted to run split -l1, I ran into a funny error: split: too many files Curious to see if any splitting had occurred, I ran ls and sure enough, a huge list of filenames appeared, such as: xaa xab ... xzy xzz Now I could see why you'd run out of unique filenames - there are only 26 letters in the alphabet and these filenames were only three letters long. Also, they all seemed to begin with the letter "x". BSD split's filename defaults I checked the manual for split's defaults and confirmed what I was seeing: each file into which the file is split is named by the prefix followed by a lexically ordered suffix using suffix_length characters in the range 'a-z'. If -a is not specified, two letters are used as the suffix....with the prefix 'x' and with suffixes as above. Got it, so running split with the defaults for prefix name and suffix length will give me filenames that always start with the letter "x" followed by two-letter alphabetical permutations composed of a-z letters, with repeats allowed. I say "repeats allowed" because I noticed filenames such as xaa and xbb in the output. Side node: The reason why I say "permutations" rather than "combinations" is because letter order matters. For example, xab and xba are two distinct and legitimate filenames. Here's a nice explanation about the difference between permutations and combinations. Some permutation math So how many filenames can you get from the BSD split tool using the defaults? There are permutation formulas out there for repeating values and non-repeating values. Based on split's behavior, I wanted to use the repeating values formula: n^r where n equals the number of possible values (26 for a-z) and r equals the number of values (2, since there are only 2 letters after "x" in the filename). 26^2 = 676 So the total number of filename permutations allowed with BSD split's defaults should be 676. To double check, I ran ls | wc -l to get the total number of files in my split_test directory. The output was 677. If you subtract my original input file, input.txt, then you have 676, or the number of permutations split would allow before running out of filenames! Neat. But I still wanted my 4000 files. Moar permutations pls While 26^2 permutations doesn't support 4000 different filenames, I wondered if I could increase r to 3. Then, I'd have 17,576 different filename permutations to play with - more than enough. Earlier, I remembered the manual mentioning suffix length: -a suffixlength Use suffixlength letters to form the suffix of the file name. So I passed 3 in with the -a flag and guess what? I got my 4000 files! split -l1 -a3 input.txt ls | wc -l 4001 But that was a lot of work. It would be great if split would just handle these permutations and suffix lengths by default! In fact, I vaguely remember splitting large files into smaller ones with numerical filenames, which I prefer. I also remember not having to worry about suffixes in the past. But numerical filenames didn't seem to be an option with split installed on Mac OS X - there was no mention of it in the manual. Turns out that I was remembering GNU split from using the Debian OS two years ago, a different flavor of the split tool with different defaults and behaviors. Beastie Bits Michael Lucas is speaking at mug.org 10 April 2018 (https://blather.michaelwlucas.com/archives/3121) PkgsrcCon 2018 July 7+8 Berlin (http://pkgsrc.org/pkgsrcCon/2018/) Tint2 rocks (http://www.vincentdelft.be/post/post_20180310) Open Source Summit Europe 2018 Call for Proposals (https://www.freebsdfoundation.org/news-and-events/call-for-papers/open-source-summit-europe-2018-call-for-proposals/) Travel Grants for BSDCan 2018 (https://www.freebsdfoundation.org/blog/bsdcan-2018-travel-grant-application-now-open/) BSDCan 2018 FreeBSD Developers Summit Call for Proposals (https://www.freebsdfoundation.org/news-and-events/call-for-papers/bsdcan-2018-freebsd-developers-summit-call-for-proposals/) OpenBSD vmm(4) update, by Mike Larkin (https://www.openbsd.org/papers/asiabsdcon2018-vmm-slides.pdf) Feedback/Questions Morgan ZFS Install Question (http://dpaste.com/3NZN49P#wrap) Andre - Splitting ZFS Array, or not (http://dpaste.com/3V09BZ5#wrap) Jake - Python Projects (http://dpaste.com/2CY5MRE#wrap) Dave - Screen Sharing & Video Conference (http://dpaste.com/257WGCB#wrap) James - ZFS disk id switching (http://dpaste.com/3HAPZ90#wrap)

We cover an interview about Unix Architecture Evolution, another vBSDcon trip report, how to teach an old Unix about backspace, new NUMA support coming to FreeBSD, and stack pointer checking in OpenBSD. This episode was brought to you by Headlines Unix Architecture Evolution from the 1970 PDP-7 to the 2017 FreeBSD (https://fosdem.org/2018/interviews/diomidis-spinellis/) Q: Could you briefly introduce yourself? I'm a professor of software engineering, a programmer at heart, and a technology author. Currently I'm also the editor in chief of the IEEE Software magazine. I recently published the book Effective Debugging, where I detail 66 ways to debug software and systems. Q: What will your talk be about, exactly? I will describe how the architecture of the Unix operating system evolved over the past half century, starting from an unnamed system written in PDP-7 assembly language and ending with a modern FreeBSD system. My talk is based, first, on a GitHub repository where I tried to record the system's history from 1970 until today and, second, on the evolution of documented facilities (user commands, system calls, library functions) across revisions. I will thus present the early system's defining architectural features (layering, system calls, devices as files, an interpreter, and process management) and the important ones that followed in subsequent releases: the tree directory structure, user contributed code, I/O redirection, the shell as a user program, groups, pipes, scripting, and little languages. Q: Why this topic? Unix stands out as a major engineering breakthrough due to its exemplary design, its numerous technical contributions, its impact, its development model, and its widespread use. Furthermore, the design of the Unix programming environment has been characterized as one offering unusual simplicity, power, and elegance. Consequently, there are many lessons that we can learn by studying the evolution of the Unix architecture, which we can apply to the design of new systems. I often see modern systems that suffer from a bloat of architectural features and a lack of clear form on which functionality can be built. I believe that many of the modern Unix architecture defining features are excellent examples of what we should strive toward as system architects. Q: What do you hope to accomplish by giving this talk? What do you expect? I'd like FOSDEM attendees to leave the talk with their mind full with architectural features of timeless quality. I want them to realize that architectural elegance isn't derived by piling design patterns and does not need to be expensive in terms of resources. Rather, beautiful architecture can be achieved on an extremely modest scale. Furthermore, I want attendees to appreciate the importance of adopting flexible conventions rather than rigid enforcement mechanisms. Finally, I want to demonstrate through examples that the open source culture was part of Unix from its earliest days. Q: What are the most significant milestones in the development of Unix? The architectural development of Unix follows a path of continuous evolution, albeit at a slowing pace, so I don't see here the most important milestones. I would however define as significant milestones two key changes in the way Unix was developed. The first occurred in the late 1970s when significant activity shifted from a closely-knit team of researchers at the AT&T Bell Labs to the Computer Science Research Group in the University of California at Berkeley. This opened the system to academic contributions and growth through competitive research funding. The second took place in the late 1980s and the 1990s when Berkeley open-sourced the the code it had developed (by that time a large percentage of the system) and enthusiasts built on it to create complete open source operating system distributions: 386BSD, and then FreeBSD, NetBSD, OpenBSD, and others. Q: In which areas has the development of Unix stalled? The data I will show demonstrate that there were in the past some long periods where the number of C library functions and system calls remained mostly stable. Nowadays there is significant growth in the number of all documented facilities with the exception of file formats. I'm looking forward to a discussion regarding the meaning of these growth patterns in the Q&A session after the talk. Q: What are the core features that still link the 1970 PDP-7 system to the latest FreeBSD 11.1 release, almost half a century apart? Over the past half-century the Unix system has grown by four orders of magnitude from a few thousand lines of code to many millions. Nevertheless, looking at a 1970s architecture diagram and a current one reveals that the initial architectural blocks are still with us today. Furthermore, most system calls, user programs, and C library functions of that era have survived until today with essentially similar functionality. I've even found in modern FreeBSD some lines of code that have survived unchanged for 40 years. Q: Can we still add innovative changes to operating systems like FreeBSD without breaking the ‘Unix philosophy'? Will there be a moment where FreeBSD isn't recognizable anymore as a descendant of the 1970 PDP-7 system? There's a saying that “form liberates”. So having available a time-tested form for developing operating system functionality allows you to innovate in areas that matter rather than reinventing the wheel. Such concepts include having commands act as a filter, providing manual pages with a consistent structure, supplying build information in the form of a Makefile, installing files in a well-defined directory hierarchy, implementing filesystems with an standardized object-oriented interface, and packaging reusable functions as a library. Within this framework there's ample space for both incremental additions (think of jq, the JSON query command) and radical innovations (consider the Solaris-derived ZFS and dtrace functionality). For this reason I think that BSD and Linux systems will always be recognizable as direct or intellectual descendants of the 1970s Research Unix editions. Q: Have you enjoyed previous FOSDEM editions? Immensely! As an academic I need to attend many scientific conferences and meetings in order to present research results and interact with colleagues. This means too much time spent traveling and away from home, and a limited number of conferences I'm in the end able to attend. Nevertheless, attending FOSDEM is an easy decision due to the world-changing nature of its theme, the breadth of the topics presented, the participants' enthusiasm and energy, as well as the exemplary, very efficient conference organization. Another vBSDCon trip report we just found (https://www.weaponizedawesome.com/blog/?cat=53) We just got tipped about another trip report from vBSDCon, this time from one of the first time speakers: W. Dean Freeman Recently I had the honor of co-presenting on the internals of FreeBSD's Kernel RNG with John-Mark Gurney at the 3rd biennial vBSDCon, hosted in Reston, VA hosted by Verisign. I've been in and out of the FreeBSD community for about 20 years. As I've mentioned on here before, my first Unix encounter was FreeBSD 2.2.8 when I was in the 7th or 8th grade. However, for all that time I've never managed to get out to any of the cons. I've been to one or two BUG meetings and I've met some folks from IRC before, but nothing like this. A BSD conference is a very different experience than anything else out there. You have to try it, it is the only way to truly understand it. I'd also not had to do a stand-up presentation really since college before this. So, my first BSD con and my first time presenting rolled into one made for an interesting experience. See, he didn't say terrifying. It went very well. You should totally submit a talk for the next conference, even if it is your first. That said, it was amazing and invigorating experience. I got to meet a few big names in the FreeBSD community, discuss projects, ideas for FreeBSD, etc. I did seem to spend an unusual amount of time talking about FIPS and Common Criteria with folks, but to me that's a good sign and indicative that there is interest in working to close gaps between FreeBSD and the current requirements so that we can start getting FreeBSD and more BSD-based products into the government and start whittling away the domination of Linux (especially since Oracle has cut Solaris, SPARC and the ZFS storage appliance business units). There is nothing that can match the high bandwidth interchange of ideas in person. The internet has made all kinds of communication possible, and we use it all the time, but every once in a while, getting together in person is hugely valuable. Dean then went on to list some of the talks he found most valuable, including DTrace, Capsicum, bhyve, *BSD security tools, and Paul Vixie's talk about gets() I think the talk that really had the biggest impact on me, however, was Kyle Kneisl's talk on BSD community dynamics. One of the key points he asked was whether the things that drew us to the BSD community in the first place would be able to happen today. Obviously, I'm not a 12 or 13 year old kid anymore, but it really got me thinking. That, combined with getting face time with people I'd previously only known as screen names has recently drawn me back into participating in IRC and rejoining mailing lists (wdf on freenode. be on the lookout!) Then Dean covered some thoughts on his own talk: JMG and my talk seems to have been well received, with people paying lots of attention. I don't know what a typical number of questions is for one of these things, but on day one there weren't that many questions. We got about 5 during our question time and spent most of the rest of the day fielding questions from interested attendees. Getting a “great talk!” from GNN after coming down from the stage was probably one of the major highlights for me. I remember my first solo talk, and GNN asking the right question in the middle to get me to explain a part of it I had missed. It was very helpful. I think key to the interest in our presentation was that JMG did a good job framing a very complicated topic's importance in terms everyone could understand. It also helped that we got to drop some serious truth bombs. Final Thoughts: I met a lot of folks in person for the first time, and met some people I'd never known online before. It was a great community and I'm glad I got a chance to expand my network. Verisign were excellent hosts and they took good care of both speakers (covering airfare, rooms, etc.) and also conference attendees at large. The dinners that they hosted were quite good as well. I'm definitely interested in attending vBSDCon again and now that I've had a taste of meeting IRL with the community on scale of more than a handful, I have every intention of finally making it to BSDCan next year (I'd said it in 2017, but then moved to Texas for a new job and it wasn't going to be practical). This year for sure, though! Teaching an Almost 40-year Old UNIX about Backspace (https://virtuallyfun.com/2018/01/17/teaching_an_almost_40-year_old_unix_about_backspace/) Introduction I have been messing with the UNIX® operating system, Seventh Edition (commonly known as UNIX V7 or just V7) for a while now. V7 dates from 1979, so it's about 40 years old at this point. The last post was on V7/x86, but since I've run into various issues with it, I moved on to a proper installation of V7 on SIMH. The Internet has some really good resources on installing V7 in SIMH. Thus, I set out on my own journey on installing and using V7 a while ago, but that was remarkably uneventful. One convenience that I have been dearly missing since the switch from V7/x86 is a functioning backspace key. There seem to be multiple different definitions of backspace: BS, as in ASCII character 8 (010, 0x08, also represented as ^H), and DEL, as in ASCII character 127 (0177, 0x7F, also represented as ^?). V7 does not accept either for input by default. Instead, # is used as the erase character and @ is used as the kill character. These defaults have been there since UNIX V1. In fact, they have been “there” since Multics, where they got chosen seemingly arbitrarily. The erase character erases the character before it. The kill character kills (deletes) the whole line. For example, “ba##gooo#d” would be interpreted as “good” and “bad line@good line” would be interpreted as “good line”. There is some debate on whether BS or DEL is the correct character for terminals to send when the user presses the backspace key. However, most programs have settled on DEL today. tmux forces DEL, even if the terminal emulator sends BS, so simply changing my terminal to send BS was not an option. The change from the defaults outlined here to today's modern-day defaults occurred between 4.1BSD and 4.2BSD. enf on Hacker News has written a nice overview of the various conventions Getting the Diff For future generations as well as myself when I inevitably majorly break this installation of V7, I wanted to make a diff. However, my V7 is installed in SIMH. I am not a very intelligent man, I didn't keep backup copies of the files I'd changed. Getting data out of this emulated machine is an exercise in frustration. In the end, I printed everything on screen using cat(1) and copied that out. Then I performed a manual diff against the original source code tree because tabs got converted to spaces in the process. Then I applied the changes to clean copies that did have the tabs. And finally, I actually invoked diff(1). Closing Thoughts Figuring all this out took me a few days. Penetrating how the system is put together was surprisingly fairly hard at first, but then the difficulty curve eased up. It was an interesting exercise in some kind of “reverse engineering” and I definitely learned something about tty handling. I was, however, not pleased with using ed(1), even if I do know the basics. vi(1) is a blessing that I did not appreciate enough until recently. Had I also been unable to access recursive grep(1) on my host and scroll through the code, I would've probably given up. Writing UNIX under those kinds of editing conditions is an amazing feat. I have nothing but the greatest respect for software developers of those days. News Roundup New NUMA support coming to FreeBSD CURRENT (https://lists.freebsd.org/pipermail/freebsd-current/2018-January/068145.html) Hello folks, I am working on merging improved NUMA support with policy implemented by cpuset(2) over the next week. This work has been supported by Dell/EMC's Isilon product division and Netflix. You can see some discussion of these changes here: https://reviews.freebsd.org/D13403 https://reviews.freebsd.org/D13289 https://reviews.freebsd.org/D13545 The work has been done in user/jeff/numa if you want to look at svn history or experiment with the branch. It has been tested by Peter Holm on i386 and amd64 and it has been verified to work on arm at various points. We are working towards compatibility with libnuma and linux mbind. These commits will bring in improved support for NUMA in the kernel. There are new domain specific allocation functions available to kernel for UMA, malloc, kmem, and vmpage*. busdmamem consumers will automatically be placed in the correct domain, bringing automatic improvements to some device performance. cpuset will be able to constrains processes, groups of processes, jails, etc. to subsets of the system memory domains, just as it can with sets of cpus. It can set default policy for any of the above. Threads can use cpusets to set policy that specifies a subset of their visible domains. Available policies are first-touch (local in linux terms), round-robin (similar to linux interleave), and preferred. For now, the default is round-robin. You can achieve a fixed domain policy by using round-robin with a bitmask of a single domain. As the scheduler and VM become more sophisticated we may switch the default to first-touch as linux does. Currently these features are enabled with VMNUMAALLOC and MAXMEMDOM. It will eventually be NUMA/MAXMEMDOM to match SMP/MAXCPU. The current NUMA syscalls and VMNUMAALLOC code was 'experimental' and will be deprecated. numactl will continue to be supported although cpuset should be preferred going forward as it supports the full feature set of the new API. Thank you for your patience as I deal with the inevitable fallout of such sweeping changes. If you do have bugs, please file them in bugzilla, or reach out to me directly. I don't always have time to catch up on all of my mailing list mail and regretfully things slip through the cracks when they are not addressed directly to me. Thanks, Jeff Stack pointer checking – OpenBSD (https://marc.info/?l=openbsd-tech&m=151572838911297&w=2) Stefan (stefan@) and I have been working for a few months on this diff, with help from a few others. At every trap and system call, it checks if the stack-pointer is on a page that is marked MAPSTACK. execve() is changed to create such mappings for the process stack. Also, libpthread is taught the new MAPSTACK flag to use with mmap(). There is no corresponding system call which can set MAP_FLAG on an existing page, you can only set the flag by mapping new memory into place. That is a piece of the security model. The purpose of this change is to twart stack pivots, which apparently have gained some popularity in JIT ROP attacks. It makes it difficult to place the ROP stack in regular data memory, and then perform a system call from it. Workarounds are cumbersome, increasing the need for far more gadgetry. But also the trap case -- if any memory experiences a demand page fault, the same check will occur and potentially also kill the process. We have experimented a little with performing this check during device interrupts, but there are some locking concerns and performance may then become a concern. It'll be best to gain experience from handle of syncronous trap cases first. chrome and other applications I use run fine! I'm asking for some feedback to discover what ports this breaks, we'd like to know. Those would be ports which try to (unconventionally) create their stacks in malloc()'d memory or inside another Data structure. Most of them are probably easily fixed ... Qt 5.9 on FreeBSD (https://euroquis.nl/bobulate/?p=1768) Tobias and Raphael have spent the past month or so hammering on the Qt 5.9 branch, which has (finally!) landed in the official FreeBSD ports tree. This brings FreeBSD back up-to-date with current Qt releases and, more importantly, up-to-date with the Qt release KDE software is increasingly expecting. With Qt 5.9, the Elisa music player works, for instance (where it has run-time errors with Qt 5.7, even if it compiles). The KDE-FreeBSD CI system has had Qt 5.9 for some time already, but that was hand-compiled and jimmied into the system, rather than being a “proper” ports build. The new Qt version uses a new build system, which is one of the things that really slowed us down from a packaging perspective. Some modules have been reshuffled in the process. Some applications depending on Qt internal-private headers have been fixed along the way. The Telegram desktop client continues to be a pain in the butt that way. Following on from Qt 5.9 there has been some work in getting ready for Clang 6 support; in general the KDE and Qt stack is clean and modern C++, so it's more infrastructural tweaks than fixing code. Outside of our silo, I still see lots of wonky C++ code being fixed and plenty of confusion between pointers and integers and strings and chars and .. ugh. Speaking of ugh, I'm still planning to clean up Qt4 on ARM aarch64 for FreeBSD; this boils down to stealing suitable qatomic implementations from Arch Linux. For regular users of Qt applications on FreeBSD, there should be few to no changes required outside the regular upgrade cycle. For KDE Plasma users, note that development of the ports has changed branches; as we get closer to actually landing modern KDE bits, things have been renamed and reshuffled and mulled over so often that the old plasma5 branch wasn't really right anymore. The kde5-import branch is where it's at nowadays, and the instructions are the same: the x11/kde5 metaport will give you all the KDE Frameworks 5, KDE Plasma Desktop and modern KDE Applications you need. Adding IPv6 to an Nginx website on FreeBSD / FreshPorts (https://dan.langille.org/2018/01/13/adding-ipv6-to-an-nginx-website-on-freebsd-freshports/) FreshPorts recently moved to an IPv6-capable server but until today, that capability has not been utilized. There were a number of things I had to configure, but this will not necessarily be an exhaustive list for you to follow. Some steps might be missing, and it might not apply to your situation. All of this took about 3 hours. We are using: FreeBSD 11.1 Bind 9.9.11 nginx 1.12.2 Fallout I expect some monitoring fallout from this change. I suspect some of my monitoring assumes IP4 and now that IPv6 is available, I need to monitor both IP addresses. ZFS on TrueOS: Why We Love OpenZFS (https://www.trueos.org/blog/zfs-trueos-love-openzfs/) TrueOS was the first desktop operating system to fully implement the OpenZFS (Zettabyte File System or ZFS for short) enterprise file system in a stable production environment. To fully understand why we love ZFS, we will look back to the early days of TrueOS (formerly PC-BSD). The development team had been using the UFS file system in TrueOS because of its solid track record with FreeBSD-based computer systems and its ability to check file consistency with the built-in check utility fsck. However, as computing demands increased, problems began to surface. Slow fsck file verification on large file systems, slow replication speeds, and inconsistency in data integrity while using UFS logging / journaling began to hinder users. It quickly became apparent that TrueOS users would need a file system that scales with evolving enterprise storage needs, offers the best data protection, and works just as well on a hobbyist system or desktop computer. Kris Moore, the founder of the TrueOS project, first heard about OpenZFS in 2007 from chatter on the FreeBSD mailing lists. In 2008, the TrueOS development team was thrilled to learn that the FreeBSD Project had ported ZFS. At the time, ZFS was still unproven as a graphical desktop solution, but Kris saw a perfect opportunity to offer ZFS as a cutting-edge file system option in the TrueOS installer, allowing the TrueOS project to act as an indicator of how OpenZFS would fair in real-world production use. The team was blown away by the reception and quality of OpenZFS on FreeBSD-based systems. By its nature, ZFS is a copy-on-write (CoW) file system that won't move a block of data until it both writes the data and verifies its integrity. This is very different from most other file systems in use today. ZFS is able to assure that data stays consistent between writes by automatically comparing write checksums, which mitigates bit rot. ZFS also comes with native RaidZ functionality that allows for enterprise data management and redundancy without the need for expensive traditional RAID cards. ZFS snapshots allow for system configuration backups in a split-second. You read that right. TrueOS can backup or restore snapshots in less than a second using the ZFS file system. Given these advantages, the TrueOS team decided to use ZFS as its exclusive file system starting in 2013, and we haven't looked back since. ZFS offers TrueOS users the stable workstation experience they want, while simultaneously scaling to meet the increasing demands of the enterprise storage market. TrueOS users are frequently commenting on how easy it is to use ZFS snapshots with our built-in snapshot utility. This allows users the freedom to experiment with their system knowing they can restore it in seconds if anything goes wrong. If you haven't had a chance to try ZFS with TrueOS, browse to our download page and make sure to grab a copy of TrueOS. You'll be blown away by the ease of use, data protection functionality, and incredible flexibility of RaidZ. Beastie Bits Source Code Podcast Interview with Michael W Lucas (https://blather.michaelwlucas.com/archives/3099) Operating System of the Year 2017: NetBSD Third place (https://w3techs.com/blog/entry/web_technologies_of_the_year_2017) OPNsense 18.1-RC1 released (https://opnsense.org/opnsense-18-1-rc1-released/) Personal OpenBSD Wiki Notes (https://balu-wiki.readthedocs.io/en/latest/security/openbsd.html) BSD section can use some contribution (https://guide.freecodecamp.org/bsd-os/) The Third Research Edition Unix Programmer's Manual (now available in PDF) (https://github.com/dspinellis/unix-v3man) Feedback/Questions Alex - my first freebsd bug (http://dpaste.com/3DSV7BC#wrap) John - Suggested Speakers (http://dpaste.com/2QFR4MT#wrap) Todd - Two questions (http://dpaste.com/2FQ450Q#wrap) Matthew - CentOS to FreeBSD (http://dpaste.com/3KA29E0#wrap) Brian - Brian - openbsd 6.2 and enlightenment .17 (http://dpaste.com/24DYF1J#wrap) ***

We review the information about Spectre & Meltdown thus far, we look at NetBSD memory sanitizer progress, Postgres on ZFS & show you a bit about NomadBSD. This episode was brought to you by Headlines Meltdown Spectre Official Site (https://meltdownattack.com/) Kernel-memory-leaking Intel processor design flaw forces Linux, Windows redesign (https://www.theregister.co.uk/2018/01/02/intel_cpu_design_flaw/) Intel's official response (https://newsroom.intel.com/news/intel-responds-to-security-research-findings/) The Register mocks intels response with pithy annotations (https://www.theregister.co.uk/2018/01/04/intel_meltdown_spectre_bugs_the_registers_annotations/) Intel's Analysis PDF (https://newsroom.intel.com/wp-content/uploads/sites/11/2018/01/Intel-Analysis-of-Speculative-Execution-Side-Channels.pdf) XKCD (https://xkcd.com/1938/) Response from FreeBSD (https://lists.freebsd.org/pipermail/freebsd-security/2018-January/009719.html) FreeBSD's patch WIP (https://reviews.freebsd.org/D13797) Why Raspberry Pi isn't vulnerable to Spectre or Meltdown (https://www.raspberrypi.org/blog/why-raspberry-pi-isnt-vulnerable-to-spectre-or-meltdown/) Xen mitigation patches (https://lists.xenproject.org/archives/html/xen-devel/2018-01/msg00110.html) Overview of affected FreeBSD Platforms/Architectures (https://wiki.freebsd.org/SpeculativeExecutionVulnerabilities) Groff's response (https://twitter.com/GroffTheBSDGoat/status/949372300368867328) ##### We'll cover OpenBSD, NetBSD, and DragonflyBSD's responses in next weeks episode. *** ###The LLVM Memory Sanitizer support work in progress (https://blog.netbsd.org/tnf/entry/the_llvm_memory_sanitizer_support) > In the past 31 days, I've managed to get the core functionality of MSan to work. This is an uninitialized memory usage detector. MSan is a special sanitizer because it requires knowledge of every entry to the basesystem library and every entry to the kernel through public interfaces. This is mandatory in order to mark memory regions as initialized. Most of the work has been done directly for MSan. However, part of the work helped generic features in compiler-rt. Sanitizers > Changes in the sanitizer are listed below in chronological order. Almost all of the changes mentioned here landed upstream. A few small patches were reverted due to breaking non-NetBSD hosts and are rescheduled for further investigation. I maintain these patches locally and have moved on for now to work on the remaining features. NetBSD syscall hooks > I wrote a large patch (815kb!) adding support for NetBSD syscall hooks for use with sanitizers. NetBSD ioctl(2) hooks > Similar to the syscall hooks, there is need to handle every ioctl(2) call. I've created the needed patch, this time shorter - for less than 300kb. New patches still pending for upstream review > There are two corrections that I've created, and they are still pending upstream for review: Add MSan interceptor for fstat(2)](https://reviews.llvm.org/D41637) Correct the setitimer interceptor on NetBSD)](https://reviews.llvm.org/D41502) > I've got a few more local patches that require cleanup before submitting to review. NetBSD basesystem corrections Sanitizers in Go The MSan state as of today Solaris support in sanitizers > I've helped the Solaris team add basic support for Sanitizers (ASan, UBsan). This does not help NetBSD directly, however indirectly it improves the overall support for non-Linux hosts and helps to catch more Linuxisms in the code. Plan for the next milestone > I plan to continue the work on MSan and correct sanitizing of the NetBSD basesystem utilities. This mandates me to iterate over the basesystem libraries implementing the missing interceptors and correcting the current support of the existing ones. My milestone is to build all src/bin programs against Memory Sanitizer and when possible execute them cleanly. This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: http://netbsd.org/donations/#how-to-donate (http://netbsd.org/donations/#how-to-donate) *** ##News Roundup ###MWL's 2017 Wrap-Up (https://blather.michaelwlucas.com/archives/3078) > The obvious place to start is my 2016 wrap-up post](https://blather.michaelwlucas.com/archives/2822), where I listed goals for 2017. As usual, these goals were wildly delusional. > The short answer is, my iron was back up to normal. My writing speed wasn't, though. I'd lost too much general health, and needed hard exercise to recover it. Yes, writing requires physical endurance. Maintaining that level of concentration for several hours a day demands a certain level of blood flow to the brain. I could have faked it in a day job, but when self-employed as an artist? Not so much. > Then there's travel. I did my usual BSDCan trip, plus two educational trips to Lincoln City, Oregon. The current political mayhem convinced me that if I wanted to hit EuroBSDCon any time in the next few years, I should do it in the very near future. So I went to Paris, where I promptly got pickpocketed. (Thankfully, they didn't get my passport.) I was actively writing the third edition of Absolute FreeBSD, so I visited BSDCam in Cambridge to get the latest information and a sense of where FreeBSD was going. I also did weekends at Kansas LinuxFest (because they asked and paid for my trip) and Penguicon. > (Because people will ask: why EuroBSDCon and not AsiaBSDCon? A six-hour transatlantic flight requires that I take a substantial dose of heavy-grade tranquilizers. I'm incapable of making intelligent decisions while on those drugs, or for several hours afterward. They don't last long enough for twelve-hour flight to Japan, so I need to be accompanied by someone qualified to tell me when I need to take the next dose partway through the flight. This isn't a predetermined time that I can set an alarm for; it depends on how the clonazepam affects me at those altitudes. A drug overdose while flying over the North Pole would be bad. When I can arrange that qualified companion, I'll make the trip.) > I need most of the preceding week to prepare for long trips. I need the following week to recover from time shifts and general exhaustion. Additionally, I have to hoard people juice for a few weeks beforehand so I can deal with folks during these expeditions. Travel disrupts my dojo time as well, which impacts my health. > Taken as a whole: I didn't get nearly as much done as I hoped. I wrote more stories, but Kris Rusch bludgeoned me into submitting them to trad markets. (The woman is a brute, I tell you. Cross her at your peril.) Among my 2017 titles, my fiction outsold the tech books. No, not Prohibition Orcs–all four of the people who buy those love them, but the sales tell me I've done something wrong with those tales. My cozy mystery git commit murder outsold Relayd and Httpd Mastery. But what outdid them both, as well as most of my older books? What title utterly dominated my sales for the last quarter of the year? It was of course, my open source software political satire disguised as porn Savaged by Systemd: an Erotic Unix Encounter. (https://www.michaelwarrenlucas.com/index.php/romance#sbs) > I can't believe I just wrote that paragraph. The good news is, once I recovered from EuroBSDCon, my writing got better. I finished Absolute FreeBSD, 3rd edition and submitted it to the publisher. I wrote the second edition of SSH Mastery (no link, because you can't order it yet.) I'm plowing through git sync murder, the sequel to git commit murder. I don't get to see the new Star Wars movie until I finish GSM, so hopefully that'll be this month. All in all, I wrote 480,200 words in 2017. Most of that was after September. It's annoyingly close to breaking half a million, but after 2016's scandalous 195,700, I'll take it. *** ###PG Phriday: Postgres on ZFS (https://blog.2ndquadrant.com/pg-phriday-postgres-zfs/) > ZFS is a filesystem originally created by Sun Microsystems, and has been available for BSD over a decade. While Postgres will run just fine on BSD, most Postgres installations are historically Linux-based systems. ZFS on Linux has had much more of a rocky road to integration due to perceived license incompatibilities. > As a consequence, administrators were reluctant or outright refused to run ZFS on their Linux clusters. It wasn't until OpenZFS was introduced in 2013 that this slowly began to change. These days, ZFS and Linux are starting to become more integrated, and Canonical of Ubuntu fame even announced direct support for ZFS in their 16.04 LTS release. > So how can a relatively obscure filesystem designed by a now-defunct hardware and software company help Postgres? Let's find out! Eddie waited til he finished high school > Old server hardware is dirt cheap these days, and make for a perfect lab for testing suspicious configurations. This is the server we'll be using for these tests for those following along at home, or want some point of reference: Dell R710 x2 Intel X5660 CPUs, for up to 24 threads 64GB RAM x4 1TB 7200RPM SATA HDDs H200 RAID card configured for Host Bus Adapter (HBA) mode 250GB Samsung 850 EVO SSD > The H200 is particularly important, as ZFS acts as its own RAID system. It also has its own checksumming and other algorithms that don't like RAID cards getting in the way. As such, we put the card itself in a mode that facilitates this use case. > Due to that, we lose out on any battery-backed write cache the RAID card might offer. To make up for it, it's fairly common to use an SSD or other persistent fast storage to act both as a write cache, and a read cache. This also transforms our HDDs into hybrid storage automatically, which is a huge performance boost on a budget. She had a guitar and she taught him some chords > First things first: we need a filesystem. This hardware has four 1TB HDDs, and a 250GB SSD. To keep this article from being too long, we've already placed GPT partition tables on all the HDDs, and split the SSD into 50GB for the OS, 32GB for the write cache, and 150GB for the read cache. A more robust setup would probably use separate SSDs or a mirrored pair for these, but labs are fair game. They moved into a place they both could afford > Let's start by getting a performance baseline for the hardware. We might expect peak performance at 12 or 24 threads because the server has 12 real CPUs and 24 threads, but query throughput actually topped out at concurrent 32 processes. We can scratch our heads over this later, for now, we can consider it the maximum capabilities of this hardware. Here's a small sample: ``` $> pgbench -S -j 32 -c 32 -M prepared -T 20 pgbench ... tps = 264661.135288 (including connections establishing) tps = 264849.345595 (excluding connections establishing) ``` So far, this is pretty standard behavior. 260k prepared queries per second is great read performance, but this is supposed to be a filesystem demonstration. Let's get ZFS involved. + The papers said Ed always played from the heart Let's repeat that same test with writes enabled. Once that happens, filesystem syncs, dirty pages, WAL overhead, and other things should drastically reduce overall throughput. That's an expected result, but how much are we looking at, here? ``` $> pgbench -j 32 -c 32 -M prepared -T 10 pgbench ... tps = 6153.877658 (including connections establishing) tps = 6162.392166 (excluding connections establishing) ``` SSD cache or not, storage overhead is a painful reality. Still, 6000 TPS with writes enabled is a great result for this hardware. Or is it? Can we actually do better? Consider the Postgres fullpagewrites parameter. Tomas Vondra has written about it in the past as a necessity to prevent WAL corruption due to partial writes. The WAL is both streaming replication and crash recovery, so its integrity is of utmost importance. As a result, this is one parameter almost everyone should leave alone. ZFS is Copy on Write (CoW). As a result, it's not possible to have a torn page because a page can't be partially written without reverting to the previous copy. This means we can actually turn off fullpagewrites in the Postgres config. The results are some fairly startling performance gains: $> pgbench -j 32 -c 32 -M prepared -T 10 pgbench tps = 10325.200812 (including connections establishing) tps = 10336.807218 (excluding connections establishing) That's nearly a 70% improvement. Due to write amplification caused by full page writes, Postgres produced 1.2GB of WAL files during a 1-minute pgbench test, but only 160MB with full page writes disabled. To be fair, a 32-thread pgbench write test is extremely abusive and certainly not a typical usage scenario. However, ZFS just ensured our storage a much lower write load by altering one single parameter. That means the capabilities of the hardware have also been extended to higher write workloads as IO bandwidth is not being consumed by WAL traffic. + They both met movie stars, partied and mingled Astute readers may have noticed we didn't change the default ZFS block size from 128k to align with the Postgres default of 8kb. As it turns out, the 128kb blocks allow ZFS to better combine some of those 8kb Postgres pages to save space. That will allow our measly 2TB to go a lot further than is otherwise possible. Please note that this is not de-duplication, but simple lz4 compression, which is nearly real-time in terms of CPU overhead. De-duplication on ZFS is currently an uncertain bizzaro universe populated with misshapen horrors crawling along a broken landscape. It's a world of extreme memory overhead for de-duplication tables, and potential lost data due to inherent conflicts with the CoW underpinnings. Please don't use it, let anyone else use it, or even think about using it, ever. + They made a record and it went in the chart We're still not done. One important aspect of ZFS as a CoW filesystem, is that it has integrated snapshots. Consider the scenario where a dev is connected to the wrong system and drops what they think is a table in a QA environment. It turns out they were in the wrong terminal and just erased a critical production table, and now everyone is frantic. + The future was wide open It's difficult to discount an immediately observable reduction in write overhead. Snapshots have a multitude of accepted and potential use cases, as well. In addition to online low-overhead compression, and the hybrid cache layer, ZFS boasts a plethora of features we didn't explore. Built-in checksums with integrated self-healing suggest it isn't entirely necessary to re-initialize an existing Postgres instance to enable checksums. The filesystem itself ensures checksums are validated and correct, especially if we have more than one drive resource in our pool. It even goes the extra mile and actively corrects inconsistencies when encountered. I immediately discounted ZFS back in 2012 because the company I worked for at the time was a pure Linux shop. ZFS was only available using the FUSE driver back then, meaning ZFS only worked through userspace with no real kernel integration. It was fun to tinker with, but nobody sane would use that on a production server of any description. Things have changed quite drastically since then. I've stopped waiting for btrfs to become viable, and ZFS has probably taken the throne away from XFS as my filesystem of choice. Future editions of the Postgres High Availability Cookbook will reflect this as well. Postgres MVCC and ZFS CoW seem made for each other. I'm curious to see what will transpire over the next few years now that ZFS has reached mainstream acceptance in at least one major Linux distribution. NomadBSD (https://github.com/mrclksr/NomadBSD) About NomadBSD is a live system for flash drives, based on FreeBSD. Screenshots http://freeshell.de/~mk/download/nomadbsd-ss1.png http://freeshell.de/~mk/download/nomadbsd-ss2.png Requirements for building the image A recent FreeBSD system Requirements for running NomadBSD A 4GB (or more) flash drive A System capable running FreeBSD 11.1 (amd64) Building the image ~~ csh # make image ~~ Writing the image to an USB memory stick ~~ csh # dd if=nomadbsd.img of=/dev/da0 bs=10240 conv=sync ~~ Resize filesystem to use the entire USB memory Boot NomadBSD into single user mode, and execute: ~~ # gpart delete -i 2 da0s1 # gpart resize -i 1 da0 # gpart commit da0s1 ~~ Determine the partition size in megabytes using fdisk da0 and calculate the remaining size of da0s1a: = - . ~~ # gpart resize -i 1 -s M da0s1 # gpart add -t freebsd-swap -i 2 da0s1 # glabel label NomadBSDsw da0s1b # service growfs onestart # reboot ~~ FreeBSD forum thread (https://forums.freebsd.org/threads/63888/) A short screen capture video of the NomadBSD system running in VirtualBox (https://freeshell.de/~mk/download/nomad_capture.mp4) *** ##Beastie Bits Coolpkg, a package manager inspired by Nix for OpenBSD (https://github.com/andrewchambers/coolpkg) zrepl - ZFS replication (https://zrepl.github.io/) OpenBSD hotplugd automount script (https://bijanebrahimi.github.io/blog/openbsd-hotplugd-scripting.html) Ancient troff sources vs. modern-day groff (https://virtuallyfun.com/2017/12/22/learn-ancient-troff-sources-vs-modern-day-groff/) Paypal donation balance and status.. thanks everyone! (http://lists.dragonflybsd.org/pipermail/users/2017-December/313752.html) Supervised FreeBSD rc.d script for a Go daemon (updated in last few days) (https://redbyte.eu/en/blog/supervised-freebsd-init-script-for-go-deamon/) A Brief History of sed (https://blog.sourcerer.io/a-brief-history-of-sed-6eaf00302ed) Flamegraph: Why does my AWS instance boot so slow? (http://www.daemonology.net/timestamping/tslog-c5.4xlarge.svg) *** ##Feedback/Questions Jeremy - Replacing Drive in a Zpool (http://dpaste.com/319593M#wrap) Dan's Blog (https://dan.langille.org/2017/08/16/swapping-5tb-in-3tb-out/) Tim - Keeping GELI key through reboot (http://dpaste.com/11QTA06) Brian - Mixing 2.5 and 3.5 drives (http://dpaste.com/2JQVD10#wrap) Troels - zfs swap on FreeBSD (http://dpaste.com/147WAFR#wrap) ***

TrueOS stable 17.12 is out, we have an OpenBSD workstation guide for you, learnings from the PDP-11, FreeBSD 2017 Releng recap and Duo SSH. This episode was brought to you by Headlines TrueOS stable release 17.12 (https://www.trueos.org/blog/trueos-17-12-release/) We are pleased to announce a new release of the 6-month STABLE version of TrueOS! This release cycle focused on lots of cleanup and stabilization of the distinguishing features of TrueOS: OpenRC, boot speed, removable-device management, SysAdm API integrations, Lumina improvements, and more. We have also been working quite a bit on the server offering of TrueOS, and are pleased to provide new text-based server images with support for Virtualization systems such as bhyve! This allows for simple server deployments which also take advantage of the TrueOS improvements to FreeBSD such as: Sane service management and status reporting with OpenRC Reliable, non-interactive system update mechanism with fail-safe boot environment support. Graphical management of remote TrueOS servers through SysAdm (also provides a reliable API for administrating systems remotely). LibreSSL for all base SSL support. Base system managed via packages (allows for additional fine-tuning). Base system is smaller due to the removal of the old GCC version in base. Any compiler and/or version may be installed and used via packages as desired. Support for newer graphics drivers and chipsets (graphics, networking, wifi, and more) TrueOS Version 17.12 (2017, December) is now available for download from the TrueOS website. Both the STABLE and UNSTABLE package repositories have also been updated in-sync with each other, so current users only need to follow the prompts about updating their system to run the new release. We are also pleased to announce the availability of TrueOS Sponsorships! If you would like to help contribute to the project financially we now have the ability to accept both one-time donations as well as recurring monthly donations which wil help us advocate for TrueOS around the world. Thank you all for using and supporting TrueOS! Notable Changes: Over 1100 OpenRC services have been created for 3rd-party packages. This should ensure the functionality of nearly all available 3rd-party packages that install/use their own services. The OpenRC services for FreeBSD itself have been overhauled, resulting in significantly shorter boot times. Separate install images for desktops and servers (server image uses a text/console installer) Bhyve support for TrueOS Server Install FreeBSD base is synced with 12.0-CURRENT as of December 4th, 2017 (Github commit: 209d01f) FreeBSD ports tree is synced as of November 30th (pre-FLAVOR changes) Lumina Desktop has been updated/developed from 1.3.0 to 1.4.1 PCDM now supports multiple simultaneous graphical sessions Removable devices are now managed through the “automounter” service. Devices are “announced” as available to the system via *.desktop shortcuts in /media. These shortcuts also contain a variety of optional “Actions” that may be performed on the device. Devices are only mounted while they are being used (such as when browsing via the command line or a file manager). Devices are automatically unmounted as soon as they stop being accessed. Integrated support for all major filesystems (UFS, EXT, FAT, NTFS, ExFAT, etc..) NOTE: The Lumina desktop is the only one which supports this functionality at the present time. The TrueOS update system has moved to an “active” update backend. This means that the user will need to actually start the update process by clicking the “Update Now” button in SysAdm, Lumina, or PCDM (as well as the command-line option). The staging of the update files is still performed automatically by default but this (and many other options) can be easily changed in the “Update Manager” settings as desired. Known Errata: [VirtualBox] Running FreeBSD within a VirtualBox VM is known to occasionally receive non-existent mouse clicks – particularly when using a scroll wheel or two-finger scroll. Quick Links: TrueOS Forums (https://discourse.trueos.org/) TrueOS Bugs (https://github.com/trueos/trueos-core/issues) TrueOS Handbook (https://www.trueos.org/handbook/trueos.html) TrueOS Community Chat on Telegram (https://t.me/TrueOSCommunity) *** OpenBSD Workstation Guide (https://begriffs.com/posts/2017-05-17-linux-workstation-guide.html) Design Goals User actions should complete instantaneously. While I understand if compiling code and rendering videos takes time, opening programs and moving windows should have no observable delay. The system should use minimalist tools. Corollary: cache data offline when possible. Everything from OpenStreetMaps to StackExchange can be stored locally. No reason to repeatedly hit the internet to query them. This also improves privacy because the initial download is indiscriminate and doesn't reveal personal queries or patterns of computer activity. No idling program should use a perceptible amount of CPU. Why does CalendarAgent on my Macbook sometimes use 150% CPU for fifteen minutes? Who knows. Why are background ChromeHelpers chugging along at upper-single-digit CPU? I didn't realize that holding a rendered DOM could be so challenging. Avoid interpreted languages, web-based desktop apps, and JavaScript garbage. There, I said it. Take your Electron apps with you to /dev/null! Stability. Old fashioned programs on a conservative OS on quality mainstream hardware. There are enough challenges to tackle without a bleeding edge system being one of them. Delegate to quality hardware components. Why use a janky ncurses software audio mixer when you can use…an actual audio mixer? Hardware privacy. No cameras or microphones that I can't physically disconnect. Also real hardware protection for cryptographic keys. Software privacy. Commercial software and operating systems have gotten so terrible about this. I even catch Mac command line tools trying to call Google Analytics. Sorry homebrew, your cute emojis don't make up for the surveillance. The Hardware Core To get the best hardware for the money I'm opting for a desktop computer. Haven't had one since the early 2000s and it feels anachronistic, but it will outperform a laptop of similar cost. After much searching, I found the HP Z240 Tower Workstation. It's no-nonsense and supports exactly the customizations I was looking for: No operating system pre-loaded (Cut out the “Windows tax”) Intel Xeon E3-1270 v6 processor (Supports ECC ram) 16 GB (2x8 GB) DDR4-2400 ECC Unbuffered memory (2400Mhz is the full memory clock speed supported by the Xeon) 256 GB HP Z Turbo Drive G2 PCIe SSD (Uses NVMe rather than SATA for faster throughput, supported by nvme(4)) No graphics card (We'll add our own) Intel® Ethernet I210-T1 PCIe (Supported by em(4)) A modest discrete video card will enable 2D Glamor acceleration on X11. The Radeon HD 6450 (sold separately) is fanless and listed as supported by radeon(4). Why build a solid computer and not protect it? Externally, the APC BR1300G UPS will protect the system from power surges and abrupt shutdowns. Peripherals The Matias Ergo Pro uses mechanical switches for that old fashioned clicky sound. It also includes dedicated buttons along the side for copying and pasting. Why is that cool? Well, it improves secondary selection, a technique that Sun computers used but time forgot. Since we're talking about a home office workstation, you may want a printer. The higher quality printers speak PostScript and PDF natively. Unix machines connect to them on TCP port 9100 and send PostScript commands directly. (You can print via telnet if you know the commands!) The Brother HL-L5100DN is a duplex LaserJet which allows that “raw” TCP printing. Audio/Video I know a lot of people enjoy surrounding themselves with a wall of monitors like they're in the heart of NASA Mission Control, but I find multi-monitor setups slightly disorienting. It introduces an extra bit of cognitive overhead to determine which monitor is for what exactly. That's why I'd go with a modest, crisp Dell UltraSharp 24" U2417H. It's 1080p and yeah there are 4k monitors nowadays, but text and icons are small enough as it is for me! If I ever considered a second monitor it would be e-ink for comfortably reading electronic copies of books or long articles. The price is currently too high to justify the purchase, but the most promising monitor seems to be the Dasung Paperlike. In the other direction, video input, it's more flexible to use a general-purpose HDMI capture box like the Rongyuxuan than settle on a particular webcam. This allows hooking up a real camera, or any other video device. Although the motherboard for this system has built-in audio, we should use a card with better OpenBSD support. The WBTUO PCIe card uses a C-Media CMI8768 chipset, handled by cmpci(4). The card provides S/PDIFF in and out ports if you ever want to use an external DAC or ADC. The way to connect it with other things is with a dedicated hardware mixer. The Behringer Xenyx 802 has all the connections needed, and the ability to route audio to and from the computer and a variety of devices at once. The mixer may seem an odd peripheral, but I want to mix the computer with an old fashioned CD player, ham radio gear, and amplifier so this unifies the audio setup. When doing remote pair programming or video team meetings it's nice to have a quality microphone. The best ones for this kind of work are directional, with a cardioid reception pattern. The MXL 770 condenser mic is perfect, and uses a powered XLR connection supplied by the mixer. Backups We're going dead simple and old-school, back to tapes. There are a set of tape standards called LTO-n. As n increases the tape capacity gets bigger, but the tape drive gets more expensive. In my opinion the best balance these days for the home user is LTO-3. You can usually find an HP Ultrium 960 LTO-3 on eBay for 150 dollars. The cartridges hold 800GB and are about 15 dollars apiece. Hard drives keep coming down in price, but these tapes are very cheap and simpler than keeping a bunch of disk drives. Also tape has proven longevity, and good recoverability. To use old fashioned tech like this you need a SCSI host bus adapter like the Adaptec 29320LPE, supported by ahd(4). Cryptography You don't want to generate and store secret keys on a general purpose network attached computer. The attack surface is a mile wide. Generating or manipulating “offline” secret keys needs to happen on a separate computer with no network access. Little boards like the Raspberry Pi would be good except they use ARM processors (incompatible with Tails OS) and have wifi. The JaguarBoard is a small x86 machine with no wireless capability. Just switch the keyboard and monitor over to this machine for your “cleanroom.” jaguar board: Generating keys requires entropy. The Linux kernel on Tails samples system properties to generate randomness, but why not help it out with a dedicated true random number generator (TRNG)? Bit Babbler supplies pure randomness at a high bitrate through USB. (OneRNG works better on the OpenBSD main system, via uonerng(4).) bit babbler: This little computer will save its results onto a OpenPGP Smartcard V2.1. This card provides write-only access to keys, and computes cryptographic primitives internally to sign and encrypt messages. To use it with a regular computer, hook up a Cherry ST2000 card reader. This reader has a PIN pad built in, so no keylogger on the main computer could even obtain your decryption PIN. The Software We take the beefed up hardware above and pair it with ninja-fast software written in C. Some text-based, others raw X11 graphical apps unencumbered by ties to any specific window manager. I'd advise OpenBSD for the underlying operating system, not a Linux. OpenBSD has greater internal consistency, their man pages are impeccable, and they make it a priority to prune old code to keep the system minimal. What Have We Learned from the PDP-11? (https://dave.cheney.net/2017/12/04/what-have-we-learned-from-the-pdp-11) The paper I have chosen tonight is a retrospective on a computer design. It is one of a series of papers by Gordon Bell, and various co-authors, spanning the design, growth, and eventual replacement of the companies iconic line of PDP-11 mini computers. This year represents the 60th anniversary of the founding of the company that produced the PDP-11. It is also 40 years since this paper was written, so I thought it would be entertaining to review Bell's retrospective through the lens of our own 20/20 hindsight. To set the scene for this paper, first we should talk a little about the company that produced the PDP-11, the Digital Equipment Corporation of Maynard, Massachusetts. Better known as DEC. It's also worth noting that the name PDP is an acronym for “Programmed Data Processor”, as at the time, computers had a reputation of being large, complicated, and expensive machines, and DEC's venture capitalists would not support them if they built a “computer” A computer is not solely determined by its architecture; it reflects the technological, economic, and human aspects of the environment in which it was designed and built. […] The finished computer is a product of the total design environment. “Right from the get go, Bell is letting us know that the success of any computer project is not abstractly building the best computer but building the right computer, and that takes context.” It is the nature of computer engineering to be goal-oriented, with pressure to produce deliverable products. It is therefore difficult to plan for an extensive lifetime. Because of the open nature of the PDP-11, anything which interpreted the instructions according to the processor specification, was a PDP-11, so there had been a rush within DEC, once it was clear that the PDP-11 market was heating up, to build implementations; you had different groups building fast, expensive ones and cost reduced slower ones The first weakness of minicomputers was their limited addressing capability. The biggest (and most common) mistake that can be made in a computer design is that of not providing enough address bits for memory addressing and management. A second weakness of minicomputers was their tendency not to have enough registers. This was corrected for the PDP-11 by providing eight 16-bit registers. Later, six 32-bit registers were added for floating-point arithmetic. […] More registers would increase the multiprogramming context switch time and confuse the user. “It's also interesting to note Bell's concern that additional registers would confuse the user. In the early 1970's the assumption that the machine would be programmed directly in assembly was still the prevailing mindset.” A third weakness of minicomputers was their lack of hardware stack capability. In the PDP-11, this was solved with the autoincrement/autodecrement addressing mechanism. This solution is unique to the PDP-11 and has proven to be exceptionally useful. (In fact, it has been copied by other designers.) “Nowadays it's hard to imagine hardware that doesn't have a notion of a stack, but consider that a stack isn't important if you don't need recursion.” “The design for the PDP-11 was laid down in 1969 and if we look at the programming languages of the time, FORTRAN and COBOL, neither supported recursive function calls. The function call sequence would often store the return address at a blank word at the start of the procedure making recursion impossible.” A fourth weakness, limited interrupt capability and slow context switching, was essentially solved with the device of UNIBUS interrupt vectors, which direct device interrupts. The basic mechanism is very fast, requiring only four memory cycles from the time an interrupt request is issued until the first instruction of the interrupt routine begins execution. A fifth weakness of prior minicomputers, inadequate character-handling capability, was met in the PDP-11 by providing direct byte addressing capability. “Strings and character handling were of increasing importance during the 1960's as scientific and business computing converged. The predominant character encodings at the time were 6 bit character sets which provided just enough space for upper case letters, the digits 0 to 9, space, and a few punctuation characters sufficient for printing financial reports.” “Because memory was so expensive, placing one 6 bit character into a 12 or 18 bit word was simply unacceptable so characters would be packed into words. This proved efficient for storage, but complex for operations like move, compare, and concatenate, which had to account for a character appearing in the top or bottom of the word, expending valuable words of program storage to cope.” “The problem was addressed in the PDP-11 by allowing the machine to operate on memory as both a 16-bit word, and the increasingly popular 8-bit byte. The expenditure of 2 additional bits per character was felt to be worth it for simpler string handling, and also eased the adoption of the increasingly popular 7-bit ASCII standard of which DEC were a proponent at the time. Bell concludes this point with the throw away line:” Although string instructions are not yet provided in the hardware, the common string operations (move, compare, concatenate) can be programmed with very short loops. A sixth weakness, the inability to use read-only memories, was avoided in the PDP-11. Most code written for the PDP-11 tends to be pure and reentrant without special effort by the programmer, allowing a read-only memory (ROM) to be used directly. A seventh weakness, one common to many minicomputers, was primitive I/O capabilities. A ninth weakness of minicomputers was the high cost of programming them. Many users program in assembly language, without the comfortable environment of editors, file systems, and debuggers available on bigger systems. The PDP-11 does not seem to have overcome this weakness, although it appears that more complex systems are being built successfully with the PDP-11 than with its predecessors, the PDP-8 and PDP-15. The problems faced by computer designers can usually be attributed to one of two causes: inexperience or second-systemitis Before the PDP-11, there was no UNIX. Before the PDP-11, there was no C, this is the computer that C was designed on. If you want to know why the classical C int is 16 bits wide, it's because of the PDP-11. UNIX bought us ideas such as pipes, everything is a file, and interactive computing. UNIX, which had arrived at Berkley in 1974 aboard a tape carried by Ken Thompson, would evolve into the west coast flavoured Berkley Systems Distribution. Berkeley UNIX had been ported to the VAX by the start of the 1980's and was thriving as the counter cultural alternative to DEC's own VMS operating system. Berkeley UNIX spawned a new generation of hackers who would go on to form companies like Sun micro systems, and languages like Self, which lead directly to the development of Java. UNIX was ported to a bewildering array of computer systems during the 80's and the fallout from the UNIX wars gave us the various BSD operating systems who continue to this day. The article, and the papers it is summarizing, contain a lot more than we could possibly dig into even if we dedicated the entire show to the topic *** News Roundup Two-factor authentication SSH with Duo in FreeBSD 11 (https://www.teachnix.com/2017/11/29/configuring-two-factor-authentication-on-freebsd-with-duo/) This setup uses an SSH key as the first factor of authentication. Please watch Part 1 on setting up SSH keys and how to scp it to your server. Video guide (https://www.youtube.com/watch?v=E5EuvF-iaV0) Register for a free account at Duo.com Install the Duo package on your FreeBSD server pkg install -y duo Log into the Duo site > Applications > Protect an Application > Search for Unix application > Protect this Application This will generate the keys we need to configure Duo. Edit the Duo config file using the course notes template vi /usr/local/etc/pam_duo.conf Example config [duo] ; Duo integration key ikey = Integration key goes here ; Duo secret key skey = Secret key goes here ; Duo API host host = API hostname goes here Change the permissions of the Duo config file. If the permissions are not correct then the service will not function properly. chmod 600 /usr/local/etc/pam_duo.conf Edit the SSHD config file using the course notes template vi /etc/ssh/sshd_config Example config ListenAddress 0.0.0.0 Port 22 PasswordAuthentication no UsePAM yes ChallengeResponseAuthentication yes UseDNS no PermitRootLogin yes AuthenticationMethods publickey,keyboard-interactive Edit PAM to configure SSHD for Duo using the course notes template Example config ``` # auth auth sufficient pamopie.so nowarn nofakeprompts auth requisite pamopieaccess.so nowarn allowlocal auth required /usr/local/lib/security/pamduo.so # session # session optional pamssh.so wantagent session required pam_permit.so # password # password sufficient pamkrb5.so nowarn tryfirstpass password required pamunix.so nowarn tryfirstpass ``` Restart the sshd service service sshd restart SSH into your FreeBSD server and follow the link it outputs to enroll your phone with Duo. ssh server.example.com SSH into your server again ssh server.example.com Choose your preferred method and it should log you into your server. FreeBSD 2017 Release Engineering Recap (https://www.freebsdfoundation.org/blog/2017-release-engineering-recap/) This past year was undoubtedly a rather busy and successful year for the Release Engineering Team. Throughout the year, development snapshot builds for FreeBSD-CURRENT and supported FreeBSD-STABLE branches were continually provided. In addition, work to package the base system using pkg(8) continued throughout the year and remains ongoing. The FreeBSD Release Engineering Team worked on the FreeBSD 11.1-RELEASE, with the code slush starting mid-May. The FreeBSD 11.1-RELEASE cycle stayed on schedule, with the final release build starting July 21, and the final release announcement following on July 25, building upon the stability and reliability of 11.0-RELEASE. Milestones during the 11.1-RELEASE cycle can be found on the 11.1 schedule page (https://www.freebsd.org/releases/11.1R/schedule.html). The final announcement is available here (https://www.freebsd.org/releases/11.1R/announce.html). The FreeBSD Release Engineering Team started the FreeBSD 10.4-RELEASE cycle, led by Marius Strobl. The FreeBSD 10.4-RELEASE cycle continued on schedule, with the only adjustments to the schedule being the addition of BETA4 and the removal of RC3. FreeBSD 10.4-RELEASE builds upon the stability and reliability of FreeBSD 10.3-RELEASE, and is planned to be the final release from the stable/10 branch. Milestones during the 10.4-RELEASE cycle can be found on the 10.4 schedule page (https://www.freebsd.org/releases/10.4R/schedule.html). The final announcement is available here (https://www.freebsd.org/releases/10.4R/announce.html). In addition to these releases, support for additional arm single-board computer images were added, notably Raspberry Pi 3 and Pine64. Additionally, release-related documentation effective 12.0-RELEASE and later has been moved from the base system repository to the documentation repository, making it possible to update related documentation as necessary post-release. Additionally, the FreeBSD Release Engineering article in the Project Handbook had been rewritten to outline current practices used by the Release Engineering Team. For more information on the procedures and processes the FreeBSD Release Engineering Team follows, the new article is available here and continually updated as procedures change. Finally, following the availability of FreeBSD 11.1-RELEASE, Glen Barber attended the September Developer Summit hosted at vBSDCon in Reston, VA, USA, where he gave a brief talk comprising of several points relating directly to the 11.1-RELEASE cycle. In particular, some of the points covered included what he felt went well during the release cycle, what did not go as well as it could have, and what we, as a Project, could do better to improve the release process. The slides from the talk are available in the FreeBSD Wiki. During the question and answer time following the talk, some questions asked included: Q: Should developers use the ‘Relnotes' tag in the Subversion commit template more loosely, at risk of an increase in false positives. A: When asked when the tag in the template was initially added, the answer would have been “no”, however in hindsight it is easier to sift through the false positives, than to comb through months or years of commit logs. Q: What issues are present preventing moving release-related documentation to the documentation repository? A: There were some rendering issues last time it was investigated, but it is really nothing more than taking the time to fix those issues. (Note, that since this talk, the migration of the documentation in question had moved.) Q: Does it make sense to extend the timeframe between milestone builds during a release cycle from one week to two weeks, to allow more time for testing, for example, RC1 versus RC2? A: No. It would extend the length of the release cycle with no real benefit between milestones since as we draw nearer to the end of a given release cycle, the number of changes to that code base significantly reduce. FLIMP - GIMP Exploit on FreeBSD (https://flimp.fuzzing-project.org) In 2014, when starting the Fuzzing Project (https://fuzzing-project.org/), Hanno Böck did some primitive fuzzing on GIMP and reported two bugs. They weren't fixed and were forgotten in the public bug tracker. Recently Tobias Stöckmann found one of these bugs (https://bugzilla.gnome.org/show_bug.cgi?id=739133) (CVE-2017-17785) and figured out that it's easy to exploit. What kind of bug is that? It's a classic heap buffer overflow in the FLIC parser. FLIC is a file format for animations and was introduced by Autodesk Animator. How does the exploit work? Tobias has created a detailed writeup (https://flimp.fuzzing-project.org/exploit.html). The exploit doesn't work for me! We figured out it's unreliable and the memory addresses are depending on many circumstances. The exploit ZIP comes with two variations using different memory addresses. Try both of them. We also noticed putting the files in a subdirectory sometimes made the exploit work. Anything more to tell about the GIMP? There's a wide variety of graphics formats. GIMP tries to support many of them, including many legacy formats that nobody is using any more today. While this has obvious advantages - you can access the old images you may find on a backup CD from 1995 - it comes with risks. Support for many obscure file formats means many parsers that hardly anyone ever looks at. So... what about the other parsers? The second bug (https://bugzilla.gnome.org/show_bug.cgi?id=739134) (CVE-2017-17786), which is a simple overread, was in the TGA parser. Furthermore we found buffer overreads in the XCF parser (https://bugzilla.gnome.org/show_bug.cgi?id=790783) (CVE-2017-17788), the Gimp Brush (GBR) parser (https://bugzilla.gnome.org/show_bug.cgi?id=790784) (CVE-2017-17784) and the Paint Shop Pro (PSP) parser (https://bugzilla.gnome.org/show_bug.cgi?id=790849) (CVE-2017-17789). We found another Heap buffer overflow (https://bugzilla.gnome.org/show_bug.cgi?id=790849) in the Paint Shop Pro parser (CVE-2017-17787) which is probably also exploitable. In other words: The GIMP import parsers are full of memory safety bugs. What should happen? First of all obviously all known memory safety bugs should be fixed. Furthermore we believe the way GIMP plugins work is not ideal for security testing. The plug-ins are separate executables, however they can't be executed on their own, as they communicate with the main GIMP process. Ideally either these plug-ins should be changed in a way that allows running them directly from the command line or - even better - they should be turned into libraries. The latter would also have the advantage of making the parser code useable for other software projects. Finally it might be a good idea to sandbox the import parsers. Dell FS12-NV7 Review – Bargain FreeBSD/ZFS box (http://blog.frankleonhardt.com/2017/dell-fs12-nv7-review-bargain-freebsdzfs-box/) It seems just about everyone selling refurbished data centre kit has a load of Dell FS12-NV7's to flog. Dell FS-what? You won't find them in the Dell catalogue, that's for sure. They look a bit like C2100s of some vintage, and they have a lot in common. But on closer inspection they're obviously a “special” for an important customer. Given the number of them knocking around, it's obviously a customer with big data, centres stuffed full of servers with a lot of processing to do. Here's a hint: It's not Google or Amazon. So, should you be buying a weirdo box with no documentation whatsoever? I'd say yes, definitely. If you're interests are anything like mine. In a 2U box you can get twin 4-core CPUs and 64Gb of RAM for £150 or less. What's not to like? Ah yes, the complete lack of documentation. Over the next few weeks I intend to cover that. And to start off this is my first PC review for nearly twenty years. As I mentioned, it's a 2U full length heavy metal box on rails. On the back there are the usual I/O ports: a 9-way RS-232, VGA, two 1Gb Ethernet, two USB2 and a PS/2 keyboard and mouse. The front is taken up by twelve 3.5″ hard drive bays, with the status lights and power button on one of the mounting ears to make room. Unlike other Dell servers, all the connections are on the back, only. So, in summary, you're getting a lot for your money if its the kind of thing you want. It's ideal as a high-performance Unix box with plenty of drive bays (preferably running BSD and ZFS). In this configuration it really shifts. Major bang-per-buck. Another idea I've had is using it for a flight simulator. That's a lot of RAM and processors for the money. If you forego the SAS controllers in the PCIe slots and dump in a decent graphics card and sound board, it's hard to see what's could be better (and you get jet engine sound effects without a speaker). So who should buy one of these? BSD geeks is the obvious answer. With a bit of tweaking they're a dream. It can build-absolutely-everything in 20-30 minutes. For storage you can put fast SAS drives in and it goes like the wind, even at 3Gb bandwidth per drive. I don't know if it works with FreeNAS but I can't see why not – I'm using mostly FreeBSD 11.1 and the generic kernel is fine. And if you want to run a load of weird operating systems (like Windows XP) in VM format, it seems to work very well with the Xen hypervisor and Dom0 under FreeBSD. Or CentOS if you prefer. So I shall end this review in true PCW style: Pros: Cheap Lots of CPUs, Lots of RAM Lots of HD slots Great for BSD/ZFS or VMs Cons: Noisy no AES-NI SAS needs upgrading Limited PCI slots As I've mentioned, the noise and SAS are easy and relatively cheap to fix, and thanks to BitCoin miners, even the PCI slot problem can be sorted. I'll talk about this in a later post. Beastie Bits Reflections on Hackathons (https://undeadly.org/cgi?action=article;sid=20171126090055) 7-Part Video Crash Course on SaltStack For FreeBSD (https://www.youtube.com/watch?v=HijG0hWebZk&list=PL5yV8umka8YQOr1wm719In5LITdGzQMOF) The LLVM Thread Sanitizer has been ported to NetBSD (https://blog.netbsd.org/tnf/entry/the_llvm_thread_sanitizer_has) The First Unix Port (1998) (http://bitsavers.informatik.uni-stuttgart.de/bits/Interdata/32bit/unix/univWollongong_v6/miller.pdf) arm64 platform now officially supported [and has syspatch(8)] (https://undeadly.org/cgi?action=article;sid=20171208082238) BSDCan 2018 Call for Participation (https://www.freebsdfoundation.org/news-and-events/call-for-papers/bsdcan-2018-call-for-participation/) AsiaBSDCon 2018 Call for Papers (https://www.freebsdfoundation.org/news-and-events/call-for-papers/asiabsdcon-2018-call-for-papers/) *** Feedback/Questions Shawn - DragonFlyBSD vagrant images (http://dpaste.com/3PRPJHG#wrap) Ben - undermydesk (http://dpaste.com/0AZ32ZB#wrap) Ken - Conferences (http://dpaste.com/3E8FQC6#wrap) Ben - ssh keys (http://dpaste.com/0E4538Q#wrap) SSH Chaining (https://www.bsdnow.tv/tutorials/ssh-chaining) ***

Allan reports on his trip to BSD Taiwan, new versions of Lumina and GhostBSD are here, a bunch of OpenBSD p2k17 hackathon reports. This episode was brought to you by Headlines Allan's Trip Report from BSD Taiwan (https://bsdtw.org/) BSD TW and Taiwan in general was a fun and interesting experience I arrived Thursday night and took the high speed train to Taipei main station, and then got on the Red line subway to the venue. The dorm rooms were on par with BSDCan, except the mattress was better. I spent Friday with a number of other FreeBSD developers doing touristy things. We went to Taipei 101, the world's tallest building from 2004 - 2010. It also features the world's fastest elevator (2004 - 2016), traveling at 60.6 km/h and transporting passengers from the 5th to 89th floor in 37 seconds. We also got to see the “tuned mass damper”, a 660 tonne steel pendulum suspended between the 92nd and 87th floors. This device resists the swaying of the building caused by high winds. There are interesting videos on display beside the damper, of its reaction during recent typhoons and earthquakes. The Taipei 101 building sits just 200 meters from a major fault line. Then we had excellent dumplings for lunch After walking around the city for a few more hours, we retired to a pub to escape the heat of the sunny Friday afternoon. Then came the best part of each day in Taipei, dinner! We continued our efforts to cause a nation wide shortage of dumplings Special thanks to Scott Tsai (https://twitter.com/scottttw) who took detailed notes for each of the presentations Saturday marked the start of the conference: Arun Thomas provided background and then a rundown of what is happening with the RISC-V architecture. Notes (https://docs.google.com/document/d/1yrnhNTHaMDr4DG-iviXN0O9NES9Lmlc7sWVQhnios6g/edit#heading=h.kcm1n3yzl35q) George Neville-Neil talked about using DTrace in distributed systems as an in-depth auditing system (who did what to whom and when). Notes (https://docs.google.com/document/d/1qut6tMVF8NesrGHd6bydLDN-aKBdXMgHx8Vp3_iGKjQ/edit#heading=h.qdghsgk1bgtl) Baptiste Daroussin presented Poudrière image, an extension of everyone's favourite package building system, to build custom images of FreeBSD. There was discussion of making this generate ZFS based images as well, making it mesh very well with my talk the next day. Notes (https://docs.google.com/document/d/1LceXj8IWJeTRHp9KzOYy8tpM00Fzt7fSN0Gw83B9COE/edit#heading=h.incfzi6bnzxr) Brooks Davis presented his work on an API design for a replacement for mmap. It started with a history of address space management in the BSD family of operating systems going all the way back to the beginning. This overview of the feature and how it evolved filled in many gaps for me, and showed why the newer work would be beneficial. The motivation for the work includes further extensions to support the CHERI hardware platform. Notes (https://docs.google.com/document/d/1LceXj8IWJeTRHp9KzOYy8tpM00Fzt7fSN0Gw83B9COE/edit#heading=h.incfzi6bnzxr) Johannes M Dieterich gave an interesting presentation about using FreeBSD and GPU acceleration for high performance computing. One of the slides showed that amd64 has taken almost the entire market for the top 500 super computers, and that linux dominates the list, with only a few remaining non-linux systems. Sadly, at the supercomputing conference the next week, it was announced that linux has achieved 100% saturation of the top 500 super computers list. Johannes detailed the available tools, what ports are missing, what changes should be made to the base system (mostly OpenMP), and generally what FreeBSD needs to do to become a player in the supercomputer OS market. Johannes' perspective is interesting, as he is a computational chemist, not a computer scientist. Those interested in improving the numerical libraries and GPU acceleration frameworks on FreeBSD should join the ports team. Notes (https://docs.google.com/document/d/1uaJiqtPk8WetST6_GnQwIV49bj790qx7ToY2BHC9zO4/edit#heading=h.nvsz1n6w3gyq) The final talk of the day was Peter Grehan, who spoke about how graphics support in bhyve came to be. He provided a history of how the feature evolved, and where it stands today. Notes (https://docs.google.com/document/d/1LqJQJUwdUwWZ0n5KwCH1vNI8jiWGJlI1j0It3mERN80/edit#heading=h.sgeixwgz7bjs) Afterwards, we traveled as a group to a large restaurant for dinner. There was even Mongolian Vodka, provided by Ganbold Tsagaankhuu of the FreeBSD project. Sunday: The first talk of the day Sunday was mine. I presented “ZFS: Advanced Integration”, mostly talking about how boot environments work, and the new libbe and be(1) tools that my GSoC student Kyle Kneitinger created to manage them. I talked about how they can be used for laptop and developer systems, but also how boot environments can be used to replace nanobsd for appliances (as already done in FreeNAS and pfSense). I also presented about zfsbootcfg (zfs nextboot), and some future extensions to it to make it even more useful in appliance type workloads. I also provided a rundown of new developments out of the ZFS developer summit, two weeks previous. Notes (https://docs.google.com/document/d/1Blh3Dulf0O91A0mwv34UnIgxRZaS_0FU2lZ41KRQoOU/edit#heading=h.gypim387e8hy) Theo de Raadt presented “Mitigations and other real Security Features”, and made his case for changing to a ‘fail closed' mode of interoperability. Computer's cannot actually self heal, so lets stop pretending that they can. Notes (https://docs.google.com/document/d/1fFHzlxJjbHPsV9t_Uh3PXZnXmkapAK5RkJsfaHki7kc/edit#heading=h.192e4lmbl70c) Ruslan Bukin talked about doing the port of FreeBSD for RISC-V and writing the Device Drivers. Ruslan walked through the process step by step, leading members of the audience to suggest he turn it into a developer's handbook article, explaining how to do the initial bringup on new hardware. Ruslan also showed off a FreeBSD/MIPS board he designed himself and had manufactured in China. Notes (https://docs.google.com/document/d/1kRhRr3O3lQ-0dS0kYF0oh_S0_zFufEwrdFjG1QLyk8Y/edit#heading=h.293mameym7w1) Mariusz Zaborski presented Case studies on sandboxing the base system with Capsicum. He discussed the challenges encountered as existing programs are modified to sandbox them, and recent advancements in the debugging tools available during that process. Mariusz also discussed the Casper service at length, including the features that are planned for 2018 and onwards. Notes (https://docs.google.com/document/d/1_0BpAE1jGr94taUlgLfSWlJOYU5II9o7Y3ol0ym1eZQ/edit#heading=h.xm9mh7dh6bay) The final presentation of the day was Mark Johnston on Memory Management Improvements in FreeBSD 12.0. This talk provided a very nice overview of the memory management system in FreeBSD, and then detailed some of the recent improvements. Notes (https://docs.google.com/document/d/1gFQXxsHM66GQGMO4-yoeFRTcmOP4NK_ujVFHIQJi82U/edit#heading=h.uirc9jyyti7w) The conference wrapped up with the Work-in-Progress session, including updates on: multi-device-at-once GELI attach, MP-safe networking on NetBSD, pkgsrc, NetBSD in general, BSD on Microsoft Azure, Mothra (send-pr for bugzilla), BSDMizer a machine learning compiler optimizer, Hyperledger Sawtooth (blockchain), and finally VIMAGE and pf testing on FreeBSD. Notes (https://docs.google.com/document/d/1miHZEPrqrpCTh8JONmUKWDPYUmTuG2lbsVrWDtekvLc/edit#heading=h.orhedpjis5po) Group Photo (https://pbs.twimg.com/media/DOh1txnVoAAFKAa.jpg:large) BSDTW was a great conference. They are still considering if it should be an annual thing, trade off every 2nd year with AsiaBSDCon, or something else. In order to continue, BSD Taiwan requires more organizers and volunteers. They have regular meetups in Taipei if you are interested in getting involved. *** Lumina 1.4.0 released (https://lumina-desktop.org/version-1-4-0-released/) The Lumina Theme Engine (and associated configuration utility) The Lumina theme engine is a new component of the “core” desktop, and provides enhanced theming capabilities for the desktop as well as all Qt5 applications. While it started out life as a fork of the “qt5ct” utility, it quickly grew all sorts of new features and functionality such as system-defined color profiles, modular theme components, and built-in editors/creators for all components. The backend of this engine is a standardized theme plugin for the Qt5 toolkit, so that all Qt5 applications will now present a unified appearance (if the application does not enforce a specific appearance/theme of it's own). Users of the Lumina desktop will automatically have this plugin enabled: no special action is required. Please note that the older desktop theme system for Lumina has been rendered obsolete by the new engine, but a settings-conversion path has already been implemented which should transition your current settings to the new engine the first time you login to Lumina 1.4.0. Custom themes for the older system may not be converted though, but it is trivial to copy/paste any custom stylesheets from the old system into the editor for the new theme engine to register/re-apply them as desired. Lumina-Themes Repository I also want to give a shout-out to the trueos/lumina-themes github repository contributors. All of the wallpapers in the 1.4.0 screenshots I posted come from that package, and they are working on making more wallpapers, color palettes, and desktop styles for use with the Lumina Theme Engine. If your operating system does not currently provide a package for lumina-themes, I highly recommend that you make one as soon as possible! The Lumina PDF Viewer (lumina-pdf) This is a new, stand-alone desktop utility for viewing/printing/presenting PDF documents. It uses the poppler-qt5 library in the backend for rendering the document, but uses multi-threading in many ways (such as to speed up the loading of pages) to give the user a nice, streamlined utility for viewing PDF documents. There is also built-in presentation functionality which allows users to easily cast the document to a separate screen without mucking about in system menus or configuration utilities. Lumina PDF Viewer (1.4.0) Important Packaging Changes One significant change of note for people who are packaging Lumina for their particular operating system is that the minimum supported versions of Qt for Lumina have been changed with this release: lumina-core: Qt 5.4+ lumina-mediaplayer: Qt 5.7+ Everything else: Qt 5.2+ Of course, using the latest version of the Qt5 libraries is always recommended. When packaging for Linux distributions, the theme engine also requires the availability of some of the “-dev” packages for Qt itself when compiling the theme plugin. For additional information (specifically regarding Ubuntu builds), please take a look at a recent ticket on the Lumina repository. + The new lumina-pdf utility requires the availability of the “poppler-qt5” library. The includes for this library on Ubuntu 17.10 were found to be installed outside of the normal include directories, so a special rule for it was added to our OS-Detect file in the Lumina source tree. If your particular operating system also places the the poppler include files in a non-standard place, please patch that file or send us the information and we can add more special rules for your particular OS. Other Changes of Note (in no particular order) lumina-config: Add a new page for changing audio theme (login, logout, low battery) Add option to replace fluxbox with some other WM (with appropriate warnings) Have the “themes” page redirect to launching the Lumina theme engine configuration utility. start-lumina-desktop: Auto-detect the active X11 displays and create a new display for the Lumina session (prevent conflict with prior graphical sessions). Add a process-failure counter & restart mechanism. This is particularly useful for restarting Fluxbox from time to time (such as after any monitor addition/removal) lumina-xconfig: Restart fluxbox after making any monitor changes with xrandr. This ensures a more reliable session. Implement a new 2D monitor layout mechanism. This allows for the placement of monitors anywhere in the X/Y plane, with simplification buttons for auto-tiling the monitors in each dimension based on their current location. Add the ability to save/load monitor profiles. Distinguish between the “default” monitor arrangement and the “current” monitor arrangement. Allow the user to set the current arrangement as the new default. lumina-desktop: Completely revamp the icon loading mechanisms so it should auto-update when the theme changes. Speed up the initialization of the desktop quite a bit. Prevent loading/probing files in the “/net/” path for existence (assume they exist in the interest of providing shortcuts). On FreeBSD, these are special paths that actually pause the calling process in order to mount/load a network share before resuming the process, and can cause significant “hangs” in the desktop process. Add the ability to take a directory as a target for the wallpaper. This will open/probe the directory for any existing image files that it can use as a wallpaper and randomly select one. Remove the popup dialog prompting about system updates, and replace it with new “Restart (with updates)” buttons on the appropriate menus/windows instead. If no wallpapers selection is provided, try to use the “lumina-nature” wallpaper directory as the default, otherwise fall back on the original default wallpaper if the “lumina-themes” package is not installed. lumina-open: Make the *.desktop parsing a bit more flexible regarding quoted strings where there should not be any. If selecting which application to use, only overwrite the user-default app if the option is explicitly selected. lumina-fileinfo: Significant cleanup of this utility. Now it can be reliably used for creating/registering XDG application shortcuts. Add a whole host of new ZFS integrations: If a ZFS dataset is being examined, show all the ZFS properties for that dataset. If the file being examined exists within ZFS snapshots, show all the snapshots of the file lumina-fm: Significant use of additional multi-threading. Makes the loading of directories much faster (particularly ones with image files which need thumbnails) Add detection/warning when running as root user. Also add an option to launch a new instance of lumina-fm as the root user. [FreeBSD/TrueOS] Fix up the detection of the “External Devices” list to also list available devices for the autofs system. Fix up some drag and drop functionality. Expose the creation, extraction, and insertion of files into archives (requires lumina-archiver at runtime) Expand the “Open With” option into a menu of application suggestions in addition to the “Other” option which runs “lumina-open” to find an application. Provide an option to set the desktop wallpaper to the selected image file(s). (If the running desktop session is Lumina). lumina-mediaplayer: Enable the ability to playback local video files. (NOTE: If Qt5 is set to use the gstreamer multimedia backend, make sure you have the “GL” plugin installed for smooth video playback). lumina-archiver: Add CLI flags for auto-archive and auto-extract. This allows for programmatic/scriptable interactions with archives. That is not mentioning all of the little bugfixes, performance tweaks, and more that are also included in this release. *** The strongest KASLR, ever? (https://blog.netbsd.org/tnf/entry/the_strongest_kaslr_ever) Re: amd64: kernel aslr support (https://mail-index.netbsd.org/tech-kern/2017/11/14/msg022594.html) So, I did it. Now the kernel sections are split in sub-blocks, and are all randomized independently. See my drawing [1]. What it means in practice, is that Kernel ASLR is much more difficult to defeat: a cache attack will at most allow you to know that a given range is mapped as executable for example, but you don't know which sub-block of .text it is; a kernel pointer leak will at most allow you to reconstruct the layout of one sub-block, but you don't know the layout and address of the remaining blocks, and there can be many. The size and number of these blocks is controlled by the split-by-file parameter in Makefile.amd64. Right now it is set to 2MB, which produces a kernel with ~23 allocatable (ie useful at runtime) sections, which is a third of the total number supported (BTSPACENSEGS = 64). I will probably reduce this parameter a bit in the future, to 1.5MB, or even 1MB. All of that leaves us with about the most advanced KASLR implementation available out there. There are ways to improve it even more, but you'll have to wait a few weeks for that. If you want to try it out you need to make sure you have the latest versions of GENERICKASLR / prekern / bootloader. The instructions are still here, and haven't changed. Initial design As I said in the previous episode, I added in October a Kernel ASLR implementation in NetBSD for 64bit x86 CPUs. This implementation would randomize the location of the kernel in virtual memory as one block: a random VA would be chosen, and the kernel ELF sections would be mapped contiguously starting from there. This design had several drawbacks: one leak, or one successful cache attack, could be enough to reconstruct the layout of the entire kernel and defeat KASLR. NetBSD's new KASLR design significantly improves this situation. New design In the new design, each kernel ELF section is randomized independently. That is to say, the base addresses of .text, .rodata, .data and .bss are not correlated. KASLR is already at this stage more difficult to defeat, since you would need a leak or cache attack on each of the kernel sections in order to reconstruct the in-memory kernel layout. Then, starting from there, several techniques are used to strengthen the implementation even more. Sub-blocks The kernel ELF sections are themselves split in sub-blocks of approximately 1MB. The kernel therefore goes from having: { .text .rodata .data .bss } to having { .text .text.0 .text.1 ... .text.i .rodata .rodata.0 ... .rodata.j ... .data ...etc } As of today, this produces a kernel with ~33 sections, each of which is mapped at a random address and in a random order. This implies that there can be dozens of .text segments. Therefore, even if you are able to conduct a cache attack and determine that a given range of memory is mapped as executable, you don't know which sub-block of .text it is. If you manage to obtain a kernel pointer via a leak, you can at most guess the address of the section it finds itself in, but you don't know the layout of the remaining 32 sections. In other words, defeating this KASLR implementation is much more complicated than in the initial design. Higher entropy Each section is put in a 2MB-sized physical memory chunk. Given that the sections are 1MB in size, this leaves half of the 2MB chunk unused. Once in control, the prekern shifts the section within the chunk using a random offset, aligned to the ELF alignment constraint. This offset has a maximum value of 1MB, so that once shifted the section still resides in its initial 2MB chunk: The prekern then maps these 2MB physical chunks at random virtual addresses; but addresses aligned to 2MB. For example, the two sections in Fig. A will be mapped at two distinct VAs: There is a reason the sections are shifted in memory: it offers higher entropy. If we consider a .text.i section with a 64byte ELF alignment constraint, and give a look at the number of possibilities for the location of the section in memory: The prekern shifts the 1MB section in its 2MB chunk, with an offset aligned to 64 bytes. So there are (2MB-1MB)/(64B)=214 possibilities for the offset. Then, the prekern uses a 2MB-sized 2MB-aligned range of VA, chosen in a 2GB window. So there are (2GB-2MB)/(2MB)=210-1 possibilities for the VA. Therefore, there are 214x(210-1)˜224 possible locations for the section. As a comparison with other systems: OS # of possibilities Linux 2^6 MacOS 2^8 Windows 2^13 NetBSD 2^24 Of course, we are talking about one .text.i section here; the sections that will be mapped afterwards will have fewer location possibilities because some slots will be already occupied. However, this does not alter the fact that the resulting entropy is still higher than that of the other implementations. Note also that several sections have an alignment constraint smaller than 64 bytes, and that in such cases the entropy is even higher. Large pages There is also a reason we chose to use 2MB-aligned 2MB-sized ranges of VAs: when the kernel is in control and initializes itself, it can now use large pages to map the physical 2MB chunks. This greatly improves memory access performance at the CPU level. Countermeasures against TLB cache attacks With the memory shift explained above, randomness is therefore enforced at both the physical and virtual levels: the address of the first page of a section does not equal the address of the section itself anymore. It has, as a side effect, an interesting property: it can mostly mitigate TLB cache attacks. Such attacks operate at the virtual-page level; they will allow you to know that a given large page is mapped as executable, but you don't know where exactly within that page the section actually begins. Strong? This KASLR implementation, which splits the kernel in dozens of sub-blocks, randomizes them independently, while at the same time allowing for higher entropy in a way that offers large page support and some countermeasures against TLB cache attacks, appears to be the most advanced KASLR implementation available publicly as of today. Feel free to prove me wrong, I would be happy to know! WIP Even if it is in a functional state, this implementation is still a work in progress, and some of the issues mentioned in the previous blog post haven't been addressed yet. But feel free to test it and report any issue you encounter. Instructions on how to use this implementation can still be found in the previous blog post, and haven't changed since. See you in the next episode! News Roundup GhostBSD 11.1 Finally Ready and Available! (http://www.ghostbsd.org/11.1_release_announcement) Screenshots (https://imgur.com/a/Mu8xk) After a year of development, testing, debugging and working on our software package repository, we are pleased to announce the release of GhostBSD 11.1 is now available on 64-bit(amd64) architecture with MATE and XFCE Desktop on direct and torrent download. With 11.1 we drop 32-bit i386 supports, and we currently maintain our software packages repository for more stability. What's new on GhostBSD 11.1 GhostBSD software repository Support VMware Workstation Guest Features New UFS full disk mirroring option on the installer New UFS full disk MBR and GPT option on the installer New UFS full disk swap size option on the installer Whisker Menu as default Application menu on XFCE All software developed by GhostBSD is now getting updated ZFS configuration for disk What has been fixed on 11.1? Fix XFCE sound plugin Installer ZFS configuration file setting Installer ZFS setup appears to be incomplete The installer was not listing ZFS disk correctly. The installer The partition list was not deleted when pressing back XFCE and MATE shutdown/suspend/hibernate randomly missing Clicking 'GhostBSD Bugs' item in the Main menu -> 'System Tools' brings up 'Server not found' page XFCE installation - incorrect keyboard layout Locale setting not filling correctly Update Station tray icon The image checksum's, hybrid ISO(DVD, USB) images are available at GhostBSD (http://www.ghostbsd.org/download). *** p2k17 Hackathon Reports p2k17 Hackathon Report: Matthias Kilian on xpdf, haskell, and more (https://undeadly.org/cgi?action=article;sid=20171107034258) p2k17 Hackathon Report: Herzliche grusse vom Berlin (espie@ on mandoc, misc packages progress) (https://undeadly.org/cgi?action=article;sid=20171107185122) p2k17 Hackathon Report: Paul Irofti (pirofti@) on hotplugd(8), math ports, xhci(4) and other kernel advancements (https://undeadly.org/cgi?action=article;sid=20171107225258) p2k17 Hackathon report: Jeremy Evans on ruby progress, postgresql and webdriver work (https://undeadly.org/cgi?action=article;sid=20171108072117) p2k17 Hackathon report: Christian Weisgerber on random devices, build failures and gettext (https://undeadly.org/cgi?action=article;sid=20171109171447) p2k17 Hackathon report: Sebastian Reitenbach on Puppet progress (https://undeadly.org/cgi?action=article;sid=20171110124645) p2k17 Hackathon Report: Anthony J. Bentley on firmware, games and securing pkg_add runs (https://undeadly.org/cgi?action=article;sid=20171110124656) p2k17 Hackathon Report: Landry Breuil on Mozilla things and much more (https://undeadly.org/cgi?action=article;sid=20171113091807) p2k17 Hackathon report: Florian Obser on network stack progress, kernel relinking and more (https://undeadly.org/cgi?action=article;sid=20171113235334) p2k17 Hackathon report: Antoine Jacoutot on ports+packages progress (https://undeadly.org/cgi?action=article;sid=20171120075903) *** TrueOS Talks Tech and Open Source at Pellissippi State (https://www.trueos.org/blog/trueos-talks-tech-open-source-pellissippi-state/) Ken Moore of the TrueOS project presented a talk to the AITP group at Pellissippi State today entitled “It's A Unix(-like) system? An Introduction to TrueOS and Open source”. Joshua Smith of the TrueOS project was also in attendance. We were happy to see a good attendance of about 40 individuals that came to hear more about TrueOS and how we continue to innovate along with the FreeBSD project. Many good questions were raised about development, snapshots, cryptocurrency, and cyber-security. We've included a copy of the slides if you'd like to have a look at the talk on open source. We'd like to offer a sincere thanks to everyone who attended and offer an extended invitation for you to join us at our KnoxBUG group on October 30th @ the iXsystems offices! We hope to see you soon! Open Source Talk – Slideshare PDF (https://web.trueos.org/wp-content/uploads/2017/10/Open-Source-Talk.pdf) KnoxBug - Lumina Rising : Challenging Desktop Orthodoxy (http://knoxbug.org/content/octobers-talk-available-youtube) Ken gave his talk about the new Lumina 2.0 Window Manager that he gave at Ohio LinuxFest 2017 KnoxBUG October 2017 (https://youtu.be/w3ZrqxLTnIU) (OLF 2017) Lumina Rising: Challenging Desktop Orthodoxy (https://www.slideshare.net/beanpole135/olf-2017-lumina-rising-challenging-desktop-orthodoxy) *** Official OpenBSD 6.2 CD set - the only one to be made! (https://undeadly.org/cgi?action=article;sid=20171118190325) Our dear friend Bob Beck (beck@) writes: So, again this release the tradition of making Theo do art has continued! Up for sale by auction to the highest bidder on Ebay is the only OpenBSD 6.2 CD set to be produced. The case and CD's feature the 6.2 artwork, custom drawn and signed by Theo. All proceeds to support OpenBSD Go have a look at the auction As with previous OpenBSD auctions, if you are not the successful bidder, we would like to encourage you to donate the equivalent of you highest bid to the project. The Auction (https://www.ebay.ca/itm/Official-OpenBSD-6-2-CD-Set/253265944606) *** Beastie Bits HAMMER2 userspace on Linux (http://lists.dragonflybsd.org/pipermail/users/2017-October/313646.html) OpenBSD Porting Workshop (now changed to January 3, 2018) (http://www.nycbug.org/index.cgi?action=view&id=10655) Matt Ahrens on when Native Encryption for ZFS will land (https://twitter.com/mahrens1/status/921204908094775296) The first successful build of OpenBSD base system (http://nanxiao.me/en/the-first-successful-build-of-openbsd-base-system/) KnoxBug November Meeting (https://www.meetup.com/KnoxBUG-BSD-Linux-and-FOSS-Users-Unite/events/245291204/) Absolute FreeBSD, 3rd Edition, pre-orders available (https://www.michaelwlucas.com/os/af3e) Feedback/Questions Jon - Jails and Networking (http://dpaste.com/2BEW0HB#wrap) Nathan - bhyve Provisioning (http://dpaste.com/1GHSYJS#wrap) Lian - OpenSSL jumping the Shark (http://dpaste.com/18P8D8C#wrap) Kim - Suggestions (http://dpaste.com/1VE0K9E#wrap) ***

EuroBSDcon trip report, how to secure OpenBSD's LDAP server, ZFS channel programs in FreeBSD HEAD and why software is storytelling. This episode was brought to you by Headlines EuroBSDcon Trip Report This is from Frank Moore, who has been supplying us with collections of links for the show and who we met at EuroBSDcon in Paris for the first time. Here is his trip report. My attendance at the EuroBSDCon 2017 conference in Paris was sprinkled with several 'firsts'. My first visit to Paris, my first time travelling on a EuroTunnel Shuttle train and my first time at any BSD conference. Hopefully, none of these will turn out to be 'lasts'. I arrived on the Wednesday afternoon before the conference started on Thursday morning. My hotel was conveniently located close to the conference centre in Paris' 3rd arrondissement. This area is well-known as a buzzy enclave of hip cafes, eateries, independent shops, markets, modern galleries and museums. It certainly lived up to its reputation. Even better, the weather held over the course of the conference, only raining once, with the rest of the time being both warm and sunny. The first two days were taken up with attending Dr Kirk McKusick's excellent tutorial 'An Introduction to the FreeBSD Open-Source Operating System'. This is training "straight from the horse's mouth". Kirk has worked extensively on The FreeBSD operating system since the 1980's, helping to design the original BSD filesystem (FFS) and later working on UFS as well. Not only is Kirk an engaging speaker, making what could be a dry topic very interesting, he also sprinkles liberal doses of history and war stories throughout his lectures. Want to know why a protocol was designed the way that it was? Or why a system flag has a particular value or position in a record? Kirk was there and has the first-hand answer. He reminisces about his meetings and work with other Unix and BSD luminaries and debunks and confirms common myths in equal measure. Kirk's teaching style and knowledge are impressive. Every section starts with an overview and a big picture diagram before drilling down into the nitty-gritty detail. Nothing feels superfluous, and everything fits together logically. It's easy to tell that the material and its delivery have been honed over many years, but without feeling stale. Topics covered included the kernel, processes, virtual memory, threads, I/O, devices, FFS, ZFS, and networking. The slides were just as impressive, with additional notes written by a previous student and every slide containing a reference back to the relevant page(s) in the 2nd edition of Kirk's operating system book. As well as a hard copy for those that requested it, Kirk also helpfully supplied soft copies of all the training materials. The breaks in between lectures were useful for meeting the students from the other tutorials and for recovering from the inevitable information overload. It's not often that you can get to hear someone as renowned as Dr McKusick give a lecture on something as important as the FreeBSD operating system. If you have any interest in FreeBSD, Unix history, or operating systems in general, I would urge you to grab the opportunity to attend one of his lectures. You won't be disappointed. The last two days of the conference consisted of various hour-long talks by members of each of the main BSD systems. All of them were fairly evenly represented except Dragonfly BSD which unfortunately only had one talk. With three talks going on at any one time, it was often difficult to pick which one to go to. At other times there might be nothing to pique the interest. Attendance at a talk is not mandatory, so for those times when no talks looked inviting, just hanging out in one of the lobby areas with other attendees was often just as interesting and informative. The conference centre itself was certainly memorable with the interior design of an Egyptian temple or pyramid. All the classrooms were more than adequate while the main auditorium was first-class and easily held the 300+ attendees comfortably. All in all, the facilities, catering and organisation were excellent. Kudos to the EuroBSDCon team, especially Bapt and Antoine for all their hard work and hospitality. As a long-time watcher and occasional contributor to the BSD Now podcast it was good to meet both Allan and Benedict in the flesh. And having done some proofreading for Michael Lucas previously, it was nice to finally meet him as well. My one suggestion to the organisers of the next conference would be to provide more hand-holding for newbies. As a first-time attendee at a BSD conference it would have been nice to have been formally introduced to various people within the projects as the goto people for their areas. I could do this myself, but it's not always easy finding the right person and wrangling an introduction. I also think it was a missed opportunity for each project to recruit new developers to their cause. Apparently, this is already in place at BSDCan, but should probably be rolled out across all BSD conferences. Having said all that, my aims for the conference were to take Dr McKusick's course, meet a few BSD people and make contacts within one of the BSD projects to start contributing. I was successful on all these fronts, so for me this was mission accomplished. Another first! autoconf/clang (No) Fun and Games (https://undeadly.org/cgi?action=article;sid=20170930133438) Robert Nagy (robert@) wrote in with a fascinating story of hunting down a recent problem with ports: You might have been noticing the amount of commits to ports regarding autoconf and nested functions and asking yourself… what the hell is this all about? I was hanging out at my friend Antoine (ajacoutot@)'s place just before EuroBSDCon 2017 started and we were having drinks and he told me that there is this weird bug where Gnome hangs completely after just a couple of seconds of usage and the gnome-shell process just sits in the fsleep state. This started to happen at the time when inteldrm(4) was updated, the default compiler was switched to clang(1) and futexes were turned on by default. The next day we started to have a look at the issue and since the process was hanging in fsleep, it seemed clear that the cause must be futexes, so we had to start bisecting the base system, which resulted in random success and failure. In the end we figured out that it is neither futex nor inteldrm(4) related, so the only thing that was left is the switch to clang. Now the problem is that we have to figure out what part of the system needs to be build with clang to trigger this issue, so we kept on going and systematically recompiled the base system with gcc until everything was ruled out … and it kept on hanging. We were drunk and angry that now we have to go and check hundreds of ports because gnome is not a small standalone port, so between two bottles of wine a build VM was fired up to do a package build with gcc, because manually building all the dependencies would just take too long and we had spent almost two days on this already. Next day ~200 packages were available to bisect and figure out what's going on. After a couple of tries it turned out that the hang is being caused by the gtk+3 package, which is bad since almost everything is using gtk+3. Now it was time to figure out what file the gtk+3 source being built by clang is causing the issue. (Compiler optimizations were ruled out already at this point.) So another set of bisecting happened, building each subdirectory of gtk+3 with clang and waiting for the hang to manifest … and it did not. What the $f? Okay so something else is going on and maybe the configure script of gtk+3 is doing something weird with different compilers, so I quickly did two configure runs with gcc and clang and simply diff'd the two directories. Snippets from the diff: -GDKHIDDENVISIBILITYCFLAGS = -fvisibility=hidden GDKHIDDENVISIBILITYCFLAGS = -ltcvprogcompilerrttiexceptions=no ltcvprogcompilerrttiexceptions=yes -#define GDKEXTERN attribute((visibility("default"))) extern -ltprogcompilernobuiltinflag=' -fno-builtin' +ltprogcompilernobuiltinflag=' -fno-builtin -fno-rtti -fno-exceptions' Okay, okay that's something, but wait … clang has symbol visibility support so what is going on again? Let's take a peek at config.log: configure:29137: checking for -fvisibility=hidden compiler flag configure:29150: cc -c -fvisibility=hidden -I/usr/local/include -I/usr/X11R6/include conftest.c >&5 conftest.c:82:17: error: function definition is not allowed here int main (void) { return 0; } ^ 1 error generated. Okay that's clearly an error but why exactly? autoconf basically generates a huge shell script that will check for whatever you throw at it by creating a file called conftest.c and putting chunks of code into it and then trying to compile it. In this case the relevant part of the code was: | int | main () | { | int main (void) { return 0; } | ; | return 0; | } That is a nested function declaration which is a GNU extension and it is not supported by clang, but that's okay, the question is why the hell would you use nested functions to check for simple compiler flags. The next step was to go and check what is going on in configure.ac to see how the configure script is generated. In the gtk+3 case the following snippet is used: AC_MSG_CHECKING([for -fvisibility=hidden compiler flag]) ACTRYCOMPILE([], [int main (void) { return 0; }], ACMSGRESULT(yes) enablefvisibilityhidden=yes, ACMSGRESULT(no) enablefvisibilityhidden=no) According to the autoconf manual the ACTRYCOMPILE macro accepts the following parameters: That clearly states that a function body has to be specified because the function definition is already provided automatically, so doing ACTRYCOMPILE([], [int main (void) { return 0;}], instead of ACTRYCOMPILE([],[] will result in a nested function declaration, which will work just fine with gcc, even though the autoconf usage is wrong. After fixing the autoconf macro in gtk+3 and rebuilding the complete port from scratch with clang, the hang completely went away as the proper CFLAGS and LDFLAGS were picked up by autoconf for the build. At this point we realized that most of the ports tree uses autoconf so this issue might be a lot bigger than we thought, so I asked sthen@ to do a grep on the ports object directory and just search for "function definition is not allowed here", which resulted in about ~60 additional ports affected. Out of the list of ports there were only two false positive matches. These were actually trying to test whether the compiler supports nested functions. The rest were a combination of several autoconf macros used in a wrong way, e.g: ACTRYCOMPILE, ACTRYLINK. Most of them were fixable by just removing the extra function declaration or by switching to other autoconf macros like ACLANGSOURCE where you can actually declare your own functions if need be. The conclusion is that this issue was a combination of people not reading documentation and just copy/pasting autoconf snippets, instead of reading their documentation and using the macros in the way they were intended, and the fact that switching to a new compiler is never easy and bugs or undefined behaviour are always lurking in the dark. Thanks to everyone who helped fixing all the ports up this quickly! Hopefully all of the changes can be merged upstream, so that others can benefit as well. Interview - David Carlier - @devnexen (https://twitter.com/devnexen) Software Engineer at Afilias *** News Roundup Setting up OpenBSD's LDAP Server (ldapd) with StartTLS and SASL (http://blog.databasepatterns.com/2017/08/setting-up-openbsds-ldap-server-ldapd.html) A tutorial on setting up OpenBSD's native LDAP server with TLS encryption and SASL authentication OpenBSD has its own LDAP server, ldapd. Here's how to configure it for use with StartTLS and SASL authentication Create a certificate (acme-client anyone?) Create a basic config file listen on em0 tls certificate ldapserver This will listen on the em0 interface with tls using the certificate called ldapserver.crt / ldapserver.key Validate the configuration: /usr/sbin/ldapd -n Enable and start the service: rcctl enable ldapd rcctl start ldapd On the client machine: pkg_add openldap-client Copy the certificate to /etc/ssl/trusted.crt Add this line to /etc/openldap/ldap.conf TLS_CACERT /etc/ssl/trusted.crt Enable and start the service rcctl enable saslauthd rcctl start saslauthd Connect to ldapd (-ZZ means force TLS, use -H to specify URI): ldapsearch -H ldap://ldapserver -ZZ FreeBSD Picks Up Support for ZFS Channel Programs in -current (https://svnweb.freebsd.org/base?view=revision&revision=324163) ZFS channel programs (ZCP) adds support for performing compound ZFS administrative actions via Lua scripts in a sandboxed environment (with time and memory limits). This initial commit includes both base support for running ZCP scripts, and a small initial library of API calls which support getting properties and listing, destroying, and promoting datasets. Testing: in addition to the included unit tests, channel programs have been in use at Delphix for several months for batch destroying filesystems. Take a simple task as an example: Create a snapshot, then set a property on that snapshot. In the traditional system for this, when you issue the snapshot command, that closes the currently open transaction group (say #100), and opens a new one, #101. While #100 is being written to disk, other writes are accumulated in #101. Once #100 is flushed to disk, the ‘zfs snapshot' command returns. You can then issue the ‘zfs set' command. This actually ends up going into transaction group #102. Each administrative action needs to wait for the transaction group to flush, which under heavy loads could take multiple seconds. Now if you want to create AND set, you need to wait for two or three transaction groups. Meanwhile, during transaction group #101, the snapshot existed without the property set, which could cause all kinds of side effects. ZFS Channel programs solves this by allowing you to perform a small scripted set of actions as a single atomic operation. In Delphix's appliance, they often needed to do as many as 15 operations together, which might take multiple minutes. Now with channel programs it is much faster, far safer, and has fewer chances of side effects BSDCan 2017 - Matt Ahrens: Building products based on OpenZFS, using channel programs -- Video Soon (http://www.bsdcan.org/2017/schedule/events/854.en.html) Software Is About Storytelling (http://bravenewgeek.com/software-is-about-storytelling/) Tyler Treat writes on the brave new geek blog: Software engineering is more a practice in archeology than it is in building. As an industry, we undervalue storytelling and focus too much on artifacts and tools and deliverables. How many times have you been left scratching your head while looking at a piece of code, system, or process? It's the story, the legacy left behind by that artifact, that is just as important—if not more—than the artifact itself. And I don't mean what's in the version control history—that's often useless. I mean the real, human story behind something. Artifacts, whether that's code or tools or something else entirely, are not just snapshots in time. They're the result of a series of decisions, discussions, mistakes, corrections, problems, constraints, and so on. They're the product of the engineering process, but the problem is they usually don't capture that process in its entirety. They rarely capture it at all. They commonly end up being nothing but a snapshot in time. It's often the sign of an inexperienced engineer when someone looks at something and says, “this is stupid” or “why are they using X instead of Y?” They're ignoring the context, the fact that circumstances may have been different. There is a story that led up to that point, a reason for why things are the way they are. If you're lucky, the people involved are still around. Unfortunately, this is not typically the case. And so it's not necessarily the poor engineer's fault for wondering these things. Their predecessors haven't done enough to make that story discoverable and share that context. I worked at a company that built a homegrown container PaaS on ECS. Doing that today would be insane with the plethora of container solutions available now. “Why aren't you using Kubernetes?” Well, four years ago when we started, Kubernetes didn't exist. Even Docker was just in its infancy. And it's not exactly a flick of a switch to move multiple production environments to a new container runtime, not to mention the politicking with leadership to convince them it's worth it to not ship any new code for the next quarter as we rearchitect our entire platform. Oh, and now the people behind the original solution are no longer with the company. Good luck! And this is on the timescale of about five years. That's maybe like one generation of engineers at the company at most—nothing compared to the decades or more software usually lives (an interesting observation is that timescale, I think, is proportional to the size of an organization). Don't underestimate momentum, but also don't underestimate changing circumstances, even on a small time horizon. The point is, stop looking at technology in a vacuum. There are many facets to consider. Likewise, decisions are not made in a vacuum. Part of this is just being an empathetic engineer. The corollary to this is you don't need to adopt every bleeding-edge tech that comes out to be successful, but the bigger point is software is about storytelling. The question you should be asking is how does your organization tell those stories? Are you deliberate or is it left to tribal knowledge and hearsay? Is it something you truly value and prioritize or simply a byproduct? Documentation is good, but the trouble with documentation is it's usually haphazard and stagnant. It's also usually documentation of how and not why. Documenting intent can go a long way, and understanding the why is a good way to develop empathy. Code survives us. There's a fantastic talk by Bryan Cantrill on oral tradition in software engineering (https://youtu.be/4PaWFYm0kEw) where he talks about this. People care about intent. Specifically, when you write software, people care what you think. As Bryan puts it, future generations of programmers want to understand your intent so they can abide by it, so we need to tell them what our intent was. We need to broadcast it. Good code comments are an example of this. They give you a narrative of not only what's going on, but why. When we write software, we write it for future generations, and that's the most underestimated thing in all of software. Documenting intent also allows you to document your values, and that allows the people who come after you to continue to uphold them. Storytelling in software is important. Without it, software archeology is simply the study of puzzles created by time and neglect. When an organization doesn't record its history, it's bound to repeat the same mistakes. A company's memory is comprised of its people, but the fact is people churn. Knowing how you got here often helps you with getting to where you want to be. Storytelling is how we transcend generational gaps and the inevitable changing of the old guard to the new guard in a maturing engineering organization. The same is true when we expand that to the entire industry. We're too memoryless—shipping code and not looking back, discovering everything old that is new again, and simply not appreciating our lineage. Beastie Bits 1st BSD Users Stockholm Meetup (https://www.meetup.com/en-US/BSD-Users-Stockholm/) Absolute FreeBSD, 3rd Edition draft completed (https://blather.michaelwlucas.com/archives/3020) Absolute FreeBSD, 3rd Edition Table of Contents (https://blather.michaelwlucas.com/archives/2995) t2k17 Hackathon Report: My first time (Aaron Bieber) (https://undeadly.org/cgi?action=article;sid=20170824193521) The release of pfSense 2.4.0 will be slightly delayed to apply patches for vulnerabilities in 3rd party packages that are part of pfSense (https://www.netgate.com/blog/no-plan-survives-contact-with-the-internet.html) Feedback/Questions Ben writes in that zrepl is in ports now (http://dpaste.com/1XMJYMH#wrap) Peter asks us about Netflix on BSD (http://dpaste.com/334WY4T#wrap) meka writes in about dhclient exiting (http://dpaste.com/3GSGKD3#wrap) ***

We recap EuroBSDcon in Paris, tell the story behind a pf PR, and show you how to do screencasting with OpenBSD. This episode was brought to you by Headlines Recap of EuroBSDcon 2017 in Paris, France (https://2017.eurobsdcon.org) EuroBSDcon was held in Paris, France this year, which drew record numbers this year. With over 300 attendees, it was the largest BSD event I have ever attended, and I was encouraged by the higher than expected number of first time attendees. The FreeBSD Foundation held a board meeting on Wednesday afternoon with the members who were in Paris. Topics included future conferences (including a conference kit we can mail to people who want to represent FreeBSD) and planning for next year. The FreeBSD Devsummit started on Thursday at the beautiful Mozilla Office in Paris. After registering and picking up our conference bag, everyone gathered for a morning coffee with lots of handshaking and greeting. We then gathered in the next room which had a podium with microphone, screens as well as tables and chairs. After developers sat down, Benedict opened the devsummit with a small quiz about France for developers to win a Mogics Power Bagel (https://www.mogics.com/?page_id=3824). 45 developers participated and DES won the item in the end. After introductions and collecting topics of interest from everyone, we started with the Work in Progress (WIP) session. The WIP session had different people present a topic they are working on in 7 minute timeslots. Topics ranged from FreeBSD Forwarding Performance, fast booting options, and a GELI patch under review to attach multiple providers. See their slides on the FreeBSD wiki (https://wiki.freebsd.org/DevSummit/201709). After lunch, the FreeBSD Foundation gave a general update on staff and funding, as well as a more focused presentation about our partnership with Intel. People were interested to hear what was done so far and asked a few questions to the Intel representative Glenn Weinberg. After lunch, developers worked quietly on their own projects. The mic remained open and occasionally, people would step forward and gave a short talk without slides or motivated a discussion of common interest. The day concluded with a dinner at a nice restaurant in Paris, which allowed to continue the discussions of the day. The second day of the devsummit began with a talk about the CAM-based SDIO stack by Ilya Bakulin. His work would allow access to wifi cards/modules on embedded boards like the Raspberry Pi Zero W and similar devices as many of these are using SDIO for data transfers. Next up was a discussion and Q&A session with the FreeBSD core team members who were there (missing only Benno Rice, Kris Moore, John Baldwin, and Baptiste Daroussin, the latter being busy with conference preparations). The new FCP (FreeBSD community proposals) were introduced for those who were not at BSDCan this year and the hows and whys about it. Allan and I were asked to describe our experiences as new members of core and we encouraged people to run for core when the next election happens. After a short break, Scott Long gave an overview of the work that's been started on NUMA (Non-Uniform Memory Architecture), what the goals of the project are and who is working on it. Before lunch, Christian Schwarz presented his work on zrepl, a new ZFS replication solution he developed using Go. This sparked interest in developers, a port was started (https://reviews.freebsd.org/D12462) and people suggested to Christian that he should submit his talk to AsiaBSDcon and BSDCan next year. Benedict had to leave before lunch was done to teach his Ansible tutorial (which was well attended) at the conference venue. There were organized dinners, for those two nights, quite a feat of organization to fit over 100 people into a restaurant and serve them quickly. On Saturday, there was a social event, a river cruise down the Seine. This took the form of a ‘standing' dinner, with a wide selection of appetizer type dishes, designed to get people to walk around and converse with many different people, rather than sit at a table with the same 6-8 people. I talked to a much larger group of people than I had managed to at the other dinners. I like having both dinner formats. We would also like to thank all of the BSDNow viewers who attended the conference and made the point of introducing themselves to us. It was nice to meet you all. The recordings of the live video stream from the conference are available immediately, so you can watch the raw versions of the talks now: Auditorium Keynote 1: Software Development in the Age of Heroes (https://youtu.be/4iR8g9-39LM?t=179) by Thomas Pornin (https://twitter.com/BearSSLnews) Tuning FreeBSD for routing and firewalling (https://youtu.be/4iR8g9-39LM?t=1660) by Olivier Cochard-Labbé (https://twitter.com/ocochardlabbe) My BSD sucks less than yours, Act I (https://youtu.be/4iR8g9-39LM?t=7040) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_) My BSD sucks less than yours, Act II (https://youtu.be/4iR8g9-39LM?t=14254) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_) Reproducible builds on NetBSD (https://youtu.be/4iR8g9-39LM?t=23351) by Christos Zoulas Your scheduler is not the problem (https://youtu.be/4iR8g9-39LM?t=26845) by Martin Pieuchot Keynote 2: A French story on cybercrime (https://youtu.be/4iR8g9-39LM?t=30540) by Éric Freyssinet (https://twitter.com/ericfreyss) Case studies of sandboxing base system with Capsicum (https://youtu.be/jqdHYEH_BQY?t=731) by Mariusz Zaborski (https://twitter.com/oshogbovx) OpenBSD's small steps towards DTrace (a tale about DDB and CTF) (https://youtu.be/jqdHYEH_BQY?t=6030) by Jasper Lievisse Adriaanse The Realities of DTrace on FreeBSD (https://youtu.be/jqdHYEH_BQY?t=13096) by George Neville-Neil (https://twitter.com/gvnn3) OpenSMTPD, current state of affairs (https://youtu.be/jqdHYEH_BQY?t=16818) by Gilles Chehade (https://twitter.com/PoolpOrg) Hoisting: lessons learned integrating pledge into 500 programs (https://youtu.be/jqdHYEH_BQY?t=21764) by Theo de Raadt Keynote 3: System Performance Analysis Methodologies (https://youtu.be/jqdHYEH_BQY?t=25463) by Brendan Gregg (https://twitter.com/brendangregg) Closing Session (https://youtu.be/jqdHYEH_BQY?t=29355) Karnak “Is it done yet ?” The never ending story of pkg tools (https://youtu.be/1hjzleqGRYk?t=71) by Marc Espie (https://twitter.com/espie_openbsd) A Tale of six motherboards, three BSDs and coreboot (https://youtu.be/1hjzleqGRYk?t=7498) by Piotr Kubaj and Katarzyna Kubaj State of the DragonFly's graphics stack (https://youtu.be/1hjzleqGRYk?t=11475) by François Tigeot From NanoBSD to ZFS and Jails – FreeBSD as a Hosting Platform, Revisited (https://youtu.be/1hjzleqGRYk?t=16227) by Patrick M. Hausen Bacula – nobody ever regretted making a backup (https://youtu.be/1hjzleqGRYk?t=20069) by Dan Langille (https://twitter.com/DLangille) Never Lose a Syslog Message (https://youtu.be/qX0BS4P65cQ?t=325) by Alexander Bluhm Running CloudABI applications on a FreeBSD-based Kubernetes cluster (https://youtu.be/qX0BS4P65cQ?t=5647) by Ed Schouten (https://twitter.com/EdSchouten) The OpenBSD web stack (https://youtu.be/qX0BS4P65cQ?t=13255) by Michael W. Lucas (https://twitter.com/mwlauthor) The LLDB Debugger on NetBSD (https://youtu.be/qX0BS4P65cQ?t=16835) by Kamil Rytarowski What's in store for NetBSD 8.0? (https://youtu.be/qX0BS4P65cQ?t=21583) by Alistair Crooks Louxor A Modern Replacement for BSD spell(1) (https://youtu.be/6Nen6a1Xl7I?t=156) by Abhinav Upadhyay (https://twitter.com/abhi9u) Portable Hotplugging: NetBSD's uvm_hotplug(9) API development (https://youtu.be/6Nen6a1Xl7I?t=5874) by Cherry G. Mathew Hardening pkgsrc (https://youtu.be/6Nen6a1Xl7I?t=9343) by Pierre Pronchery (https://twitter.com/khorben) Discovering OpenBSD on AWS (https://youtu.be/6Nen6a1Xl7I?t=14874) by Laurent Bernaille (https://twitter.com/lbernail) OpenBSD Testing Infrastructure Behind bluhm.genua.de (https://youtu.be/6Nen6a1Xl7I?t=18639) by Jan Klemkow The school of hard knocks – PT1 (https://youtu.be/8wuW8lfsVGc?t=276) by Sevan Janiyan (https://twitter.com/sevanjaniyan) 7 years of maintaining firefox, and still looking ahead (https://youtu.be/8wuW8lfsVGc?t=5321) by Landry Breuil Branch VPN solution based on OpenBSD, OSPF, RDomains and Ansible (https://youtu.be/8wuW8lfsVGc?t=12385) by Remi Locherer Running BSD on AWS (https://youtu.be/8wuW8lfsVGc?t=15983) by Julien Simon and Nicolas David Getting started with OpenBSD device driver development (https://youtu.be/8wuW8lfsVGc?t=21491) by Stefan Sperling A huge thanks to the organizers, program committee, and sponsors of EuroBSDCon. Next year, EuroBSDcon will be in Bucharest, Romania. *** The story of PR 219251 (https://www.sigsegv.be//blog/freebsd/PR219251) The actual story I wanted Kristof to tell, the pf bug he fixed at the Essen Hackathon earlier this summer. As I threatened to do in my previous post, I'm going to talk about PR 219251 for a bit. The bug report dates from only a few months ago, but the first report (that I can remeber) actually came from Shawn Webb on Twitter, of all places Despite there being a stacktrace it took quite a while (nearly 6 months in fact) before I figured this one out. It took Reshad Patuck managing to distill the problem down to a small-ish test script to make real progress on this. His testcase meant that I could get core dumps and experiment. It also provided valuable clues because it could be tweaked to see what elements were required to trigger the panic. This test script starts a (vnet) jail, adds an epair interface to it, sets up pf in the jail, and then reloads the pf rules on the host. Interestingly the panic does not seem to occur if that last step is not included. Obviously not the desired behaviour, but it seems strange. The instances of pf in the jails are supposed to be separate. We try to fetch a counter value here, but instead we dereference a bad pointer. There's two here, so already we need more information. Inspection of the core dump reveals that the state pointer is valid, and contains sane information. The rule pointer (rule.ptr) points to a sensible location, but the data is mostly 0xdeadc0de. This is the memory allocator being helpful (in debug mode) and writing garbage over freed memory, to make use-after-free bugs like this one easier to find. In other words: the rule has been free()d while there was still a state pointing to it. Somehow we have a state (describing a connection pf knows about) which points to a rule which no longer exists. The core dump also shows that the problem always occurs with states and rules in the default vnet (i.e. the host pf instance), not one of the pf instances in one of the vnet jails. That matches with the observation that the test script does not trigger the panic unless we also reload the rules on the host. Great, we know what's wrong, but now we need to work out how we can get into this state. At this point we're going to have to learn something about how rules and states get cleaned up in pf. Don't worry if you had no idea, because before this bug I didn't either. The states keep a pointer to the rule they match, so when rules are changed (or removed) we can't just delete them. States get cleaned up when connections are closed or they time out. This means we have to keep old rules around until the states that use them expire. When rules are removed pfunlinkrule() adds then to the Vpfunlinkedrules list (more on that funny V prefix later). From time to time the pf purge thread will run over all states and mark the rules that are used by a state. Once that's done for all states we know that all rules that are not marked as in-use can be removed (because none of the states use it). That can be a lot of work if we've got a lot of states, so pfpurgethread() breaks that up into smaller chuncks, iterating only part of the state table on every run. We iterate over all of our virtual pf instances (VNETFOREACH()), check if it's active (for FreeBSD-EN-17.08, where we've seen this code before) and then check the expired states with pfpurgeexpiredstates(). We start at state 'idx' and only process a certain number (determined by the PFTMINTERVAL setting) states. The pfpurgeexpiredstates() function returns a new idx value to tell us how far we got. So, remember when I mentioned the odd V_ prefix? Those are per-vnet variables. They work a bit like thread-local variables. Each vnet (virtual network stack) keeps its state separate from the others, and the V_ variables use a pointer that's changed whenever we change the currently active vnet (say with CURVNETSET() or CURVNETRESTORE()). That's tracked in the 'curvnet' variable. In other words: there are as many Vpfvnetactive variables as there are vnets: number of vnet jails plus one (for the host system). Why is that relevant here? Note that idx is not a per-vnet variable, but we handle multiple pf instances here. We run through all of them in fact. That means that we end up checking the first X states in the first vnet, then check the second X states in the second vnet, the third X states in the third and so on and so on. That of course means that we think we've run through all of the states in a vnet while we really only checked some of them. So when pfpurgeunlinkedrules() runs it can end up free()ing rules that actually are still in use because pfpurgethread() skipped over the state(s) that actually used the rule. The problem only happened if we reloaded rules in the host, because the active ruleset is never free()d, even if there are no states pointing to the rule. That explains the panic, and the fix is actually quite straightforward: idx needs to be a per-vnet variable, Vpfpurge_idx, and then the problem is gone. As is often the case, the solution to a fairly hard problem turns out to be really simple. As you might expect, finding the problem takes a lot more work that fixing it Thanks to Kristof for writing up this detailed post explaining how the problem was found, and what caused it. *** vBSDcon 2017: BSD at Work (https://www.ixsystems.com/blog/vbsdcon-2017-dexter/) The third biennial vBSDcon hosted by Verisign took place September 7th through 9th with the FreeBSD Developer Summit taking place the first day. vBSDcon and iXsystems' MeetBSD event have been alternating between the East and West coasts of the U.S.A. and these two events play vital roles in reaching Washington, DC-area and Bay Area/Silicon Valley audiences. Where MeetBSD serves many BSD Vendors, vBSDcon attracts a unique government and security industry demographic that isn't found anywhere else. Conference time and travel budgets are always limited and bringing these events to their attendees is a much-appreciated service provided by their hosts. The vBSDcon FreeBSD DevSummit had a strong focus on OpenZFS, the build system and networking with the FreeBSD 12 wish list of features in mind. How to best incorporate the steady flow of new OpenZFS features into FreeBSD such as dataset-level encryption was of particular interest. This feature from a GNU/Linux-based storage vendor is tribute to the growth of the OpenZFS community which is vital in light of the recent “Death of Solaris and ZFS” at Oracle. There has never been more demand for OpenZFS on FreeBSD and the Oracle news further confirms our collective responsibility to meet that demand. The official conference opened with my talk on “Isolated BSD Build Environments” in which I explained how the bhyve hypervisor can be used to effortlessly tour FreeBSD 5.0-onward and build specific source releases on demand to trace regressions to their offending commit. I was followed by a FreeNAS user who made the good point that FreeNAS is an exemplary “entry vector” into Unix and Enterprise Storage fundamentals, given that many of the vectors our generation had are gone. Where many of us discovered Unix and the Internet via console terminals at school or work, smart phones are only delivering the Internet without the Unix. With some irony, both iOS and Android are Unix-based yet offer few opportunities for their users to learn and leverage their Unix environments. The next two talks were The History and Future of Core Dumps in FreeBSD by Sam Gwydir and Using pkgsrc for multi-platform deployments in heterogeneous environments by G. Clifford Williams. I strongly recommend that anyone wanting to speak at AsiaBSDCon read Sam's accompanying paper on core dumps because I consider it the perfect AsiaBSDCon topic and his execution is excellent. Core dumps are one of those things you rarely think about until they are a DROP EVERYTHING! priority. G. Clifford's talk was about what I consider a near-perfect BSD project: pkgsrc, the portable BSD package manager. I put it up there with OpenSSH and mandoc as projects that have provided significant value to other Open Source operating systems. G. Clifford's real-world experiences are perfectly inline with vBSDcon's goal to be more production-oriented than other BSDCons. Of the other talks, any and all Dtrace talks are always appreciated and George Neville-Neil's did not disappoint. He based it on his experiences with the Teach BSD project which is bringing FreeBSD-based computer science education to schools around the world. The security-related talks by John-Mark Gurney, Dean Freeman and Michael Shirk also represented vBSDcon's consideration of the local community and made a convincing point that the BSDs should make concerted efforts to qualify for Common Criteria, FIPS, and other Government security requirements. While some security experts will scoff at these, they are critical to the adoption of BSD-based products by government agencies. BSD Now hosts Allan Jude and Benedict Reuschling hosted an OpenZFS BoF and Ansible talk respectively and I hosted a bhyve hypervisor BoF. The Hallway Track and food at vBSDcon were excellent and both culminated with an after-dinner dramatic reading of Michael W. Lucas' latest book that raised money for the FreeBSD Foundation. A great time was had by all and it was wonderful to see everyone! News Roundup FreeBSD 10.4-RC2 Available (https://lists.freebsd.org/pipermail/freebsd-stable/2017-September/087848.html) FreeBSD 10.4 will be released soon, this is the last chance to find bugs before the official release is cut. Noteworthy Changes Since 10.4-RC1: Given that the amd64 disc1 image was overflowing, more of the base components installed into the disc1 (live) file systems had to be disabled. Most notably, this removed the compiler toolchain from the disc1 images. All disabled tools are still available with the dvd1 images, though. The aesni(4) driver now no longer shares a single FPU context across multiple sessions in multiple threads, addressing problems seen when employing aesni(4) for ipsec(4). Support for netmap(4) by the ixgbe(4) driver has been brought into line with the netmap(4) API present in stable/10. Also, ixgbe(4) now correctly handles VFs in its netmap(4) support again instead of treating these as PFs. During the creation of amd64 and i386 VM images, etcupdate(8) and mergemaster(8) databases now are bootstrapped, akin to what happens along the extraction of base.txz as part of a new installation via bsdinstall(8). This change allows for both of these tools to work out-of-box on the VM images and avoids errors seen when upgrading these images via freebsd-update(8). If you are still on the stable/10 branch, you should test upgrading to 10.4, and make sure there are no problems with your workload Additional testing specifically of the features that have changed since 10.4-BETA1 would also be most helpful This will be the last release from the stable/10 branch *** OpenBSD changes of note 628 (https://www.tedunangst.com/flak/post/openbsd-changes-of-note-628) EuroBSDCon in two weeks. Be sure to attend early and often. Many and various documentation improvements for libcrypto. New man pages, rewrites, expanded bugs sections, and more. Only allow upward migration in vmd. There's a README for the syspatch build system if you want to run your own. Move the kernel relinking code from /etc/rc into a seperate script usable by syspatch. Kernel patches can now be reduced to just the necessary files. Make the callers of sogetopt() responsible for allocating memory. Now allocation and free occur in the same place. Use waitpid() instead of wait() in most programs to avoid accidentally collecting the wrong child. Have cu call isatty() before making assumptions. Switch mandoc rendering of mathematical symbols and greek letters from trying to imitate the characters' graphical shapes, which resulted in unintelligible renderings in many cases, to transliterations conveying the characters' meanings. Update libexpat to 2.2.4. Fix copying partial UTF-8 characters. Sigh, here we go again. Work around bug in F5's handling of the supported elliptic curves extension. RFC 4492 only defines elliptic_curves for ClientHello. However, F5 is sending it in ServerHello. We need to skip over it since our TLS extension parsing code is now more strict. After a first install, run syspatch -c to check for patches. If SMAP is present, clear PSL_AC on kernel entry and interrupt so that only the code in copy{in,out}* that need it run with it set. Panic if it's set on entry to trap() or syscall(). Prompted by Maxime Villard's NetBSD work. Errata. New drivers for arm: rktemp, mvpinctrl, mvmpic, mvneta, mvmdio, mvpxa, rkiic, rkpmic. No need to exec rm from within mandoc. We know there's exactly one file and directory to remove. Similarly with running cmp. Revert to Mesa 13.0.6 to hopefully address rendering issues a handful of people have reported with xpdf/fvwm on ivy bridge with modesetting driver. Rewrite ALPN extension using CBB/CBS and the new extension framework. Rewrite SRTP extension using CBB/CBS and the new extension framework. Revisit 2q queue sizes. Limit the hot queue to 1/20th the cache size up to a max of 4096 pages. Limit the warm and cold queues to half the cache. This allows us to more effectively notice re-interest in buffers instead of losing it in a large hot queue. Add glass console support for arm64. Probably not yet for your machine, though. Replace heaps of hand-written syscall stubs in ld.so with a simpler framework. 65535 is a valid port to listen on. When xinit starts an X server that listens only on UNIX socket, prefer DISPLAY=unix:0 rather than DISPLAY=:0. This will prevent applications from ever falling back to TCP if the UNIX socket connection fails (such as when the X server crashes). Reverted. Add -z and -Z options to apmd to auto suspend or hibernate when low on battery. Remove the original (pre-IETF) chacha20-poly1305 cipher suites. Add urng(4) which supports various USB RNG devices. Instead of adding one driver per device, start bundling them into a single driver. Remove old deactivated pledge path code. A replacement mechanism is being brewed. Fix a bug from the extension parsing rewrite. Always parse ALPN even if no callback has been installed to prevent leaving unprocessed data which leads to a decode error. Clarify what is meant by syslog priorities being ordered, since the numbers and priorities are backwards. Remove a stray setlocale() from ksh, eliminating a lot of extra statically linked code. Unremove some NPN symbols from libssl because ports software thinks they should be there for reasons. Fix saved stack location after resume. Somehow clang changed it. Resume works again on i386. Improve error messages in vmd and vmctl to be more informative. Stop building the miniroot installer for OMAP3 Beagleboards. It hasn't worked in over a year and nobody noticed. Have the callers of sosetopt() free the mbuf for symmetry. On octeon, let the kernel use the hardware FPU even if emulation is compiled in. It's faster. Fix support for 486DX CPUs by not calling cpuid. I used to own a 486. Now I don't. Merge some drm fixes from linux. Defer probing of floppy drives, eliminating delays during boot. Better handling of probes and beacons and timeouts and scans in wifi stack to avoid disconnects. Move mutex, condvar, and thread-specific data routes, pthreadonce, and pthreadexit from libpthread to libc, along with low-level bits to support them. Let's thread aware (but not actually threaded) code work with just libc. New POSIX xlocale implementation. Complete as long as you only use ASCII and UTF-8, as you should. Round and round it goes; when 6.2 stops, nobody knows. A peak at the future? *** Screencasting with OpenBSD (http://eradman.com/posts/screencasting.html) USB Audio Any USB microphone should appear as a new audio device. Here is the dmesg for my mic by ART: uaudio0 at uhub0 port 2 configuration 1 interface 0 "M-One USB" rev 1.10/0.01 addr 2 uaudio0: audio rev 1.00, 8 mixer controls audio1 at uaudio0 audioctl can read off all of the specific characterisitcs of this device $ audioctl -f /dev/audio1 | grep record mode=play,record record.rate=48000 record.channels=1 record.precision=16 record.bps=2 record.msb=1 record.encoding=slinear_le record.pause=0 record.active=0 record.block_size=1960 record.bytes=0 record.errors=0 Now test the recording from the second audio device using aucat(1) aucat -f rsnd/1 -o file.wav If the device also has a headset audio can be played through the same device. aucat -f rsnd/1 -i file.wav Screen Capture using Xvfb The rate at which a framebuffer for your video card is a feature of the hardware and software your using, and it's often very slow. x11vnc will print an estimate of the banwidth for the system your running. x11vnc ... 09/05/2012 22:23:45 fb read rate: 7 MB/sec This is about 4fps. We can do much better by using a virtual framebuffer. Here I'm setting up a new screen, setting the background color, starting cwm and an instance of xterm Xvfb :1 -screen 0 720x540x16 & DISPLAY=:1 xsetroot -solid steelblue & DISPLAY=:1 cwm & DISPLAY=:1 xterm +sb -fa Hermit -fs 14 & Much better! Now we're up around 20fps. x11vnc -display :1 & ... 11/05/2012 18:04:07 fb read rate: 168 MB/sec Make a connection to this virtual screen using raw encoding to eliminate time wasted on compression. vncviewer localhost -encodings raw A test recording with sound then looks like this ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 ~/out.avi Note: always stop the recording and playback using q, not Ctrl-C so that audio inputs are shut down properly. Screen Capture using Xephyr Xephyr is perhaps the easiest way to run X with a shadow framebuffer. This solution also avoids reading from the video card's RAM, so it's reasonably fast. Xephyr -ac -br -noreset -screen 800x600 :1 & DISPLAY=:1 xsetroot -solid steelblue & DISPLAY=:1 cwm & DISPLAY=:1 xrdb -load ~/.Xdefaults & DISPLAY=:1 xterm +sb -fa "Hermit" -fs 14 & Capture works in exactally the same way. This command tries to maintain 12fps. ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi To capture keyboard and mouse input press Ctrl then Shift. This is very handy for using navigating a window manager in the nested X session. Arranging Windows I have sometimes found it helpful to launch applications and arrange them in a specific way. This will open up a web browser listing the current directory and position windows using xdotool DISPLAY=:1 midori "file:///pwd" & sleep 2 DISPLAY=:1 xdotool search --name "xterm" windowmove 0 0 DISPLAY=:1 xdotool search --class "midori" windowmove 400 0 DISPLAY=:1 xdotool search --class "midori" windowsize 400 576 This will position the window precisely so that it appears to be in a tmux window on the right. Audio/Video Sync If you find that the audio is way out of sync with the video, you can ajust the start using the -ss before the audio input to specify the number of seconds to delay. My final recording command line, that delays the audio by 0.5 seconds, writing 12fps ffmpeg -ss 0.5 -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi Sharing a Terminal with tmux If you're trying to record a terminal session, tmux is able to share a session. In this way a recording of an X framebuffer can be taken without even using the screen. Start by creating the session. tmux -2 -S /tmp/tmux0 Then on the remote side connect on the same socket tmux -2 -S /tmp/tmux0 attach Taking Screenshots Grabbing a screenshots on Xvfb server is easily accomplished with ImageMagick's import command DISPLAY=:1 import -window root screenshot.png Audio Processing and Video Transcoding The first step is to ensure that the clip begins and ends where you'd like it to. The following will make a copy of the recording starting at time 00:00 and ending at 09:45 ffmpeg -i interactive-sql.avi -vcodec copy -acodec copy -ss 00:00:00 -t 00:09:45 interactive-sql-trimmed.avi mv interactive-sql-trimmed.avi interactive-sql.avi Setting the gain correctly is very important with an analog mixer, but if you're using a USB mic there may not be a gain option; simply record using it's built-in settings and then adjust the levels afterwards using a utility such as normalize. First extact the audio as a raw PCM file and then run normalize ffmpeg -i interactive-sql.avi -c:a copy -vn audio.wav normalize audio.wav Next merge the audio back in again ffmpeg -i interactive-sql.avi -i audio.wav -map 0:0 -map 1:0 -c copy interactive-sql-normalized.avi The final step is to compress the screencast for distribution. Encoding to VP8/Vorbis is easy: ffmpeg -i interactive-sql-normalized.avi -c:v libvpx -b:v 1M -c:a libvorbis -q:a 6 interactive-sql.webm H.264/AAC is tricky. For most video players the color space needs to be set to yuv420p. The -movflags puts the index data at the beginning of the file to enable streaming/partial content requests over HTTP: ffmpeg -y -i interactive-sql-normalized.avi -c:v libx264 -preset slow -crf 14 -pix_fmt yuv420p -movflags +faststart -c:a aac -q:a 6 interactive-sql.mp4 TrueOS @ Ohio Linuxfest '17! (https://www.trueos.org/blog/trueos-ohio-linuxfest-17/) Dru Lavigne and Ken Moore are both giving presentations on Saturday the 30th. Sit in and hear about new developments for the Lumina and FreeNAS projects. Ken is offering Lumina Rising: Challenging Desktop Orthodoxy at 10:15 am in Franklin A. Hear his thoughts about the ideas propelling desktop environment development and how Lumina, especially Lumina 2, is seeking to offer a new model of desktop architecture. Elements discussed include session security, application dependencies, message handling, and operating system integration. Dru is talking about What's New in FreeNAS 11 at 2:00 pm in Franklin D. She'll be providing an overview of some of the new features added in FreeNAS 11.0, including: Alert Services Starting specific services at boot time AD Monitoring to ensure the AD service restarts if disconnected A preview of the new user interface support for S3-compatible storage and the bhyve hypervisor She's also giving a sneak peek of FreeNAS 11.1, which has some neat features: A complete rewrite of the Jails/Plugins system as FreeNAS moves from warden to iocage Writing new plugins with just a few lines of code A brand new asynchronous middleware API Who's going? Attending this year are: Dru Lavigne (dlavigne): Dru leads the technical documentation team at iX, and contributes heavily to open source documentation projects like FreeBSD, FreeNAS, and TrueOS. Ken Moore (beanpole134): Ken is the lead developer of Lumina and a core contributor to TrueOS. He also works on a number of other Qt5 projects for iXsystems. J.T. Pennington (q5sys): Some of you may be familiar with his work on BSDNow, but J.T. also contributes to the TrueOS, Lumina, and SysAdm projects, helping out with development and general bug squashing. *** Beastie Bits Lumina Development Preview: Theme Engine (https://www.trueos.org/blog/lumina-development-preview-theme-engine/) It's happening! Official retro Thinkpad lappy spotted in the wild (https://www.theregister.co.uk/2017/09/04/retro_thinkpad_spotted_in_the_wild/) LLVM libFuzzer and SafeStack ported to NetBSD (https://blog.netbsd.org/tnf/entry/llvm_libfuzzer_and_safestack_ported) Remaining 2017 FreeBSD Events (https://www.freebsdfoundation.org/news-and-events/event-calendar/2017-openzfs-developer-summit/) *** Feedback/Questions Andrew - BSD Teaching Material (http://dpaste.com/0YTT0VP) Seth - Switching to Tarsnap after Crashplan becomes no more (http://dpaste.com/1SK92ZX#wrap) Thomas - Native encryption in ZFS (http://dpaste.com/02KD5FX#wrap) Coding Cowboy - Coding Cowboy - Passwords and clipboards (http://dpaste.com/31K0E40#wrap) ***

We recap vBSDcon, give you the story behind a PF EN, reminisce in Solaris memories, and show you how to configure different DEs on FreeBSD. This episode was brought to you by Headlines [vBSDCon] vBSDCon was held September 7 - 9th. We recorded this only a few days after getting home from this great event. Things started on Wednesday night, as attendees of the thursday developer summit arrived and broke into smallish groups for disorganized dinner and drinks. We then held an unofficial hacker lounge in a medium sized seating area, working and talking until we all decided that the developer summit started awfully early tomorrow. The developer summit started with a light breakfast and then then we dove right in Ed Maste started us off, and then Glen Barber gave a presentation about lessons learned from the 11.1-RELEASE cycle, and comparing it to previous releases. 11.1 was released on time, and was one of the best releases so far. The slides are linked on the DevSummit wiki page (https://wiki.freebsd.org/DevSummit/20170907). The group then jumped into hackmd.io a collaborative note taking application, and listed of various works in progress and upstreaming efforts. Then we listed wants and needs for the 12.0 release. After lunch we broke into pairs of working groups, with additional space for smaller meetings. The first pair were, ZFS and Toolchain, followed by a break and then a discussion of IFLIB and network drivers in general. After another break, the last groups of the day met, pkgbase and secure boot. Then it was time for the vBSDCon reception dinner. This standing dinner was a great way to meet new people, and for attendees to mingle and socialize. The official hacking lounge Thursday night was busy, and included some great storytelling, along with a bunch of work getting done. It was very encouraging to watch a struggling new developer getting help from a seasoned veteran. Watching the new developers eyes light up as the new information filled in gaps and they now understood so much more than just a few minutes before, and they raced off to continue working, was inspirational, and reminded me why these conferences are so important. The hacker lounge shut down relatively early by BSD conference standards, but, the conference proper started at 8:45 sharp the next morning, so it made sense. Friday saw a string of good presentations, I think my favourite was Jonathan Anderson's talk on Oblivious sandboxing. Jonathan is a very energetic speaker, and was able to keep everyone focused even during relatively complicated explanations. Friday night I went for dinner at ‘Big Bowl', a stir-fry bar, with a largish group of developers and users of both FreeBSD and OpenBSD. The discussions were interesting and varied, and the food was excellent. Benedict had dinner with JT and some other folks from iXsystems. Friday night the hacker lounge was so large we took over a bigger room (it had better WiFi too). Saturday featured more great talks. The talk I was most interested in was from Eric McCorkle, who did the EFI version of my GELIBoot work. I had reviewed some of the work, but it was interesting to hear the story of how it happened, and to see the parallels with my own story. My favourite speaker was Paul Vixie, who gave a very interesting talk about the gets() function in libc. gets() was declared unsafe before the FreeBSD project even started. The original import of the CSRG code into FreeBSD includes the compile time, and run-time warnings against using gets(). OpenBSD removed gets() in version 5.6, in 2014. Following Paul's presentation, various patches were raised, to either cause use of gets() to crash the program, or to remove gets() entirely, causing such programs to fail to link. The last talk before the closing was Benedict's BSD Systems Management with Ansible (https://people.freebsd.org/~bcr/talks/vBSDcon2017_Ansible.pdf). Shortly after, Allan won a MacBook Pro by correctly guessing the number of components in a jar that was standing next to the registration desk (Benedict was way off, but had a good laugh about the unlikely future Apple user). Saturday night ended with the Conference Social, and excellent dinner with more great conversations On Sunday morning, a number of us went to the Smithsonian Air and Space Museum site near the airport, and saw a Concorde, an SR-71, and the space shuttle Discovery, among many other exhibits. Check out the full photo album by JT (https://t.co/KRmSNzUSus), our producer. Thanks to all the sponsors for vBSDcon and all the organizers from Verisign, who made it such a great event. *** The story behind FreeBSD-EN-17.08.pf (https://www.sigsegv.be//blog/freebsd/FreeBSD-EN-17.08.pf) After our previous deep dive on a bug in episode 209, Kristof Provost, the maintainer of pf on FreeBSD (he is going to hate me for saying that) has written the story behind a recent ERRATA notice for FreeBSD First things first, so I have to point out that I think Allan misremembered things. The heroic debugging story is PR 219251, which I'll try to write about later. FreeBSD-EN-17:08.pf is an issue that affected some FreeBSD 11.x systems, where FreeBSD would panic at startup. There were no reports for CURRENT. There's very little to go on here, but we do know the cause of the panic ("integer divide fault"), and that the current process was "pf purge". The pf purge thread is part of the pf housekeeping infrastructure. It's a housekeeping kernel thread which cleans up things like old states and expired fragments. The lack of mention of pf functions in the backtrace is a hint unto itself. It suggests that the error is probably directly in pfpurgethread(). It might also be in one of the static functions it calls, because compilers often just inline those so they don't generate stack frames. Remember that the problem is an "integer divide fault". How can integer divisions be a problem? Well, you can try to divide by zero. The most obvious suspect for this is this code: idx = pfpurgeexpiredstates(idx, pfhashmask / (Vpfdefaultrule.timeout[PFTMINTERVAL] * 10)); However, this variable is both correctly initialised (in pfattachvnet()) and can only be modified through the DIOCSETTIMEOUT ioctl() call and that one checks for zero. At that point I had no idea how this could happen, but because the problem did not affect CURRENT I looked at the commit history and found this commit from Luiz Otavio O Souza: Do not run the pf purge thread while the VNET variables are not initialized, this can cause a divide by zero (if the VNET initialization takes to long to complete). Obtained from: pfSense Sponsored by: Rubicon Communications, LLC (Netgate) That sounds very familiar, and indeed, applying the patch fixed the problem. Luiz explained it well: it's possible to use Vpfdefaultrule.timeout before it's initialised, which caused this panic. To me, this reaffirms the importance of writing good commit messages: because Luiz mentioned both the pf purge thread and the division by zero I was easily able to find the relevant commit. If I hadn't found it this fix would have taken a lot longer. Next week we'll look at the more interesting story I was interested in, which I managed to nag Kristof into writing *** The sudden death and eternal life of Solaris (http://dtrace.org/blogs/bmc/2017/09/04/the-sudden-death-and-eternal-life-of-solaris/) A blog post from Bryan Cantrill about the death of Solaris As had been rumored for a while, Oracle effectively killed Solaris. When I first saw this, I had assumed that this was merely a deep cut, but in talking to Solaris engineers still at Oracle, it is clearly much more than that. It is a cut so deep as to be fatal: the core Solaris engineering organization lost on the order of 90% of its people, including essentially all management. Of note, among the engineers I have spoken with, I heard two things repeatedly: “this is the end” and (from those who managed to survive Friday) “I wish I had been laid off.” Gone is any of the optimism (however tepid) that I have heard over the years — and embarrassed apologies for Oracle's behavior have been replaced with dismay about the clumsiness, ineptitude and callousness with which this final cut was handled. In particular, that employees who had given their careers to the company were told of their termination via a pre-recorded call — “robo-RIF'd” in the words of one employee — is both despicable and cowardly. To their credit, the engineers affected saw themselves as Sun to the end: they stayed to solve hard, interesting problems and out of allegiance to one another — not out of any loyalty to the broader Oracle. Oracle didn't deserve them and now it doesn't have them — they have been liberated, if in a depraved act of corporate violence. Assuming that this is indeed the end of Solaris (and it certainly looks that way), it offers a time for reflection. Certainly, the demise of Solaris is at one level not surprising, but on the other hand, its very suddenness highlights the degree to which proprietary software can suffer by the vicissitudes of corporate capriciousness. Vulnerable to executive whims, shareholder demands, and a fickle public, organizations can simply change direction by fiat. And because — in the words of the late, great Roger Faulkner — “it is easier to destroy than to create,” these changes in direction can have lasting effect when they mean stopping (or even suspending!) work on a project. Indeed, any engineer in any domain with sufficient longevity will have one (or many!) stories of exciting projects being cancelled by foolhardy and myopic management. For software, though, these cancellations can be particularly gutting because (in the proprietary world, anyway) so many of the details of software are carefully hidden from the users of the product — and much of the innovation of a cancelled software project will likely die with the project, living only in the oral tradition of the engineers who knew it. Worse, in the long run — to paraphrase Keynes — proprietary software projects are all dead. However ubiquitous at their height, this lonely fate awaits all proprietary software. There is, of course, another way — and befitting its idiosyncratic life and death, Solaris shows us this path too: software can be open source. In stark contrast to proprietary software, open source does not — cannot, even — die. Yes, it can be disused or rusty or fusty, but as long as anyone is interested in it at all, it lives and breathes. Even should the interest wane to nothing, open source software survives still: its life as machine may be suspended, but it becomes as literature, waiting to be discovered by a future generation. That is, while proprietary software can die in an instant, open source software perpetually endures by its nature — and thrives by the strength of its communities. Just as the existence of proprietary software can be surprisingly brittle, open source communities can be crazily robust: they can survive neglect, derision, dissent — even sabotage. In this regard, I speak from experience: from when Solaris was open sourced in 2005, the OpenSolaris community survived all of these things. By the time Oracle bought Sun five years later in 2010, the community had decided that it needed true independence — illumos was born. And, it turns out, illumos was born at exactly the right moment: shortly after illumos was announced, Oracle — in what remains to me a singularly loathsome and cowardly act — silently re-proprietarized Solaris on August 13, 2010. We in illumos were indisputably on our own, and while many outsiders gave us no chance of survival, we ourselves had reason for confidence: after all, open source communities are robust because they are often united not only by circumstance, but by values, and in our case, we as a community never lost our belief in ZFS, Zones, DTrace and myriad other technologies like MDB, FMA and Crossbow. Indeed, since 2010, illumos has thrived; illumos is not only the repository of record for technologies that have become cross-platform like OpenZFS, but we have also advanced our core technologies considerably, while still maintaining highest standards of quality. Learning some of the mistakes of OpenSolaris, we have a model that allows for downstream innovation, experimentation and differentiation. For example, Joyent's SmartOS has always been focused on our need for a cloud hypervisor (causing us to develop big features like hardware virtualization and Linux binary compatibility), and it is now at the heart of a massive buildout for Samsung (who acquired Joyent a little over a year ago). For us at Joyent, the Solaris/illumos/SmartOS saga has been formative in that we have seen both the ill effects of proprietary software and the amazing resilience of open source software — and it very much informed our decision to open source our entire stack in 2014. Judging merely by its tombstone, the life of Solaris can be viewed as tragic: born out of wedlock between Sun and AT&T and dying at the hands of a remorseless corporate sociopath a quarter century later. And even that may be overstating its longevity: Solaris may not have been truly born until it was made open source, and — certainly to me, anyway — it died the moment it was again made proprietary. But in that shorter life, Solaris achieved the singular: immortality for its revolutionary technologies. So while we can mourn the loss of the proprietary embodiment of Solaris (and we can certainly lament the coarse way in which its technologists were treated!), we can rejoice in the eternal life of its technologies — in illumos and beyond! News Roundup OpenBSD on the Lenovo Thinkpad X1 Carbon (5th Gen) (https://jcs.org/2017/09/01/thinkpad_x1c) Joshua Stein writes about his experiences running OpenBSD on the 5th generation Lenovo Thinkpad X1 Carbon: ThinkPads have sort of a cult following among OpenBSD developers and users because the hardware is basic and well supported, and the keyboards are great to type on. While no stranger to ThinkPads myself, most of my OpenBSD laptops in recent years have been from various vendors with brand new hardware components that OpenBSD does not yet support. As satisfying as it is to write new kernel drivers or extend existing ones to make that hardware work, it usually leaves me with a laptop that doesn't work very well for a period of months. After exhausting efforts trying to debug the I2C touchpad interrupts on the Huawei MateBook X (and other 100-Series Intel chipset laptops), I decided to take a break and use something with better OpenBSD support out of the box: the fifth generation Lenovo ThinkPad X1 Carbon. Hardware Like most ThinkPads, the X1 Carbon is available in a myriad of different internal configurations. I went with the non-vPro Core i7-7500U (it was the same price as the Core i5 that I normally opt for), 16Gb of RAM, a 256Gb NVMe SSD, and a WQHD display. This generation of X1 Carbon finally brings a thinner screen bezel, allowing the entire footprint of the laptop to be smaller which is welcome on something with a 14" screen. The X1 now measures 12.7" wide, 8.5" deep, and 0.6" thick, and weighs just 2.6 pounds. While not available at initial launch, Lenovo is now offering a WQHD IPS screen option giving a resolution of 2560x1440. Perhaps more importantly, this display also has much better brightness than the FHD version, something ThinkPads have always struggled with. On the left side of the laptop are two USB-C ports, a USB-A port, a full-size HDMI port, and a port for the ethernet dongle which, despite some reviews stating otherwise, is not included with the laptop. On the right side is another USB-A port and a headphone jack, along with a fan exhaust grille. On the back is a tray for the micro-SIM card for the optional WWAN device, which also covers the Realtek microSD card reader. The tray requires a paperclip to eject which makes it inconvenient to remove, so I think this microSD card slot is designed to house a card semi-permanently as a backup disk or something. On the bottom are the two speakers towards the front and an exhaust grille near the center. The four rubber feet are rather plastic feeling, which allows the laptop to slide around on a desk a bit too much for my liking. I wish they were a bit softer to be stickier. Charging can be done via either of the two USB-C ports on the left, though I wish more vendors would do as Google did on the Chromebook Pixel and provide a port on both sides. This makes it much more convenient to charge when not at one's desk, rather than having to route a cable around to one specific side. The X1 Carbon includes a 65W USB-C PD with a fixed USB-C cable and removable country-specific power cable, which is not very convenient due to its large footprint. I am using an Apple 61W USB-C charger and an Anker cable which charge the X1 fine (unlike HP laptops which only work with HP USB-C chargers). Wireless connectivity is provided by a removable Intel 8265 802.11a/b/g/n/ac WiFi and Bluetooth 4.1 card. An Intel I219-V chip provides ethernet connectivity and requires an external dongle for the physical cable connection. The screen hinge is rather tight, making it difficult to open with one hand. The tradeoff is that the screen does not wobble in the least bit when typing. The fan is silent at idle, and there is no coil whine even under heavy load. During a make -j4 build, the fan noise is reasonable and medium-pitched, rather than a high-pitched whine like on some laptops. The palm rest and keyboard area remain cool during high CPU utilization. The full-sized keyboard is backlit and offers two levels of adjustment. The keys have a soft surface and a somewhat clicky feel, providing very quiet typing except for certain keys like Enter, Backspace, and Escape. The keyboard has a reported key travel of 1.5mm and there are dedicated Page Up and Page Down keys above the Left and Right arrow keys. Dedicated Home, End, Insert, and Delete keys are along the top row. The Fn key is placed to the left of Control, which some people hate (although Lenovo does provide a BIOS option to swap it), but it's in the same position on Apple keyboards so I'm used to it. However, since there are dedicated Page Up, Page Down, Home, and End keys, I don't really have a use for the Fn key anyway. Firmware The X1 Carbon has a very detailed BIOS/firmware menu which can be entered with the F1 key at boot. F12 can be used to temporarily select a different boot device. A neat feature of the Lenovo BIOS is that it supports showing a custom boot logo instead of the big red Lenovo logo. From Windows, download the latest BIOS Update Utility for the X1 Carbon (my model was 20HR). Run it and it'll extract everything to C:driversflash(some random string). Drop a logo.gif file in that directory and run winuptp.exe. If a logo file is present, it'll ask whether to use it and then write the new BIOS to its staging area, then reboot to actually flash it. + OpenBSD support Secure Boot has to be disabled in the BIOS menu, and the "CSM Support" option must be enabled, even when "UEFI/Legacy Boot" is left on "UEFI Only". Otherwise the screen will just go black after the OpenBSD kernel loads into memory. Based on this component list, it seems like everything but the fingerprint sensor works fine on OpenBSD. *** Configuring 5 different desktop environments on FreeBSD (https://www.linuxsecrets.com/en/entry/51-freebsd/2017/09/04/2942-configure-5-freebsd-x-environments) This fairly quick tutorial over at LinuxSecrets.com is a great start if you are new to FreeBSD, especially if you are coming from Linux and miss your favourite desktop environment It just goes to show how easy it is to build the desktop you want on modern FreeBSD The tutorial covers: GNOME, KDE, Xfce, Mate, and Cinnamon The instructions for each boil down to some variation of: Install the desktop environment and a login manager if it is not included: > sudo pkg install gnome3 Enable the login manager, and usually dbus and hald: > sudo sysrc dbusenable="YES" haldenable="YES" gdmenable="YES" gnomeenable="YES"? If using a generic login manager, add the DE startup command to your .xinitrc: > echo "exec cinnamon" > ~/.xinitrc And that is about it. The tutorial goes into more detail on other configuration you can do to get your desktop just the way you like it. To install Lumina: > sudo pkg install lumina pcbsd-utils-qt5 This will install Lumina and the pcbsd utilities package which includes pcdm, the login manager. In the near future we hear the login manager and some of the other utilities will be split into separate packages, making it easier to use them on vanilla FreeBSD. > sudo sysrc pcdmenable=”YES” dbusenable="YES" hald_enable="YES" Reboot, and you should be greeted with the graphical login screen *** A return-oriented programming defense from OpenBSD (https://lwn.net/Articles/732201/) We talked a bit about RETGUARD last week, presenting Theo's email announcing the new feature Linux Weekly News has a nice breakdown on just how it works Stack-smashing attacks have a long history; they featured, for example, as a core part of the Morris worm back in 1988. Restrictions on executing code on the stack have, to a great extent, put an end to such simple attacks, but that does not mean that stack-smashing attacks are no longer a threat. Return-oriented programming (ROP) has become a common technique for compromising systems via a stack-smashing vulnerability. There are various schemes out there for defeating ROP attacks, but a mechanism called "RETGUARD" that is being implemented in OpenBSD is notable for its relative simplicity. In a classic stack-smashing attack, the attack code would be written directly to the stack and executed there. Most modern systems do not allow execution of on-stack code, though, so this kind of attack will be ineffective. The stack does affect code execution, though, in that the call chain is stored there; when a function executes a "return" instruction, the address to return to is taken from the stack. An attacker who can overwrite the stack can, thus, force a function to "return" to an arbitrary location. That alone can be enough to carry out some types of attacks, but ROP adds another level of sophistication. A search through a body of binary code will turn up a great many short sequences of instructions ending in a return instruction. These sequences are termed "gadgets"; a large program contains enough gadgets to carry out almost any desired task — if they can be strung together into a chain. ROP works by locating these gadgets, then building a series of stack frames so that each gadget "returns" to the next. There is, of course, a significant limitation here: a ROP chain made up of exclusively polymorphic gadgets will still work, since those gadgets were not (intentionally) created by the compiler and do not contain the return-address-mangling code. De Raadt acknowledged this limitation, but said: "we believe once standard-RET is solved those concerns become easier to address separately in the future. In any case a substantial reduction of gadgets is powerful". Using the compiler to insert the hardening code greatly eases the task of applying RETGUARD to both the OpenBSD kernel and its user-space code. At least, that is true for code written in a high-level language. Any code written in assembly must be changed by hand, though, which is a fair amount of work. De Raadt and company have done that work; he reports that: "We are at the point where userland and base are fully working without regressions, and the remaining impacts are in a few larger ports which directly access the return address (for a variety of reasons)". It can be expected that, once these final issues are dealt with, OpenBSD will ship with this hardening enabled. The article wonders about applying the same to Linux, but notes it would be difficult because the Linux kernel cannot currently be compiled using LLVM If any benchmarks have been run to determine the cost of using RETGUARD, they have not been publicly posted. The extra code will make the kernel a little bigger, and the extra overhead on every function is likely to add up in the end. But if this technique can make the kernel that much harder to exploit, it may well justify the extra execution overhead that it brings with it. All that's needed is somebody to actually do the work and try it out. Videos from BSDCan have started to appear! (https://www.youtube.com/playlist?list=PLeF8ZihVdpFfVEsCxNWGDmcATJfRZacHv) Henning Brauer: tcp synfloods - BSDCan 2017 (https://www.youtube.com/watch?v=KuHepyI0_KY) Benno Rice: The Trouble with FreeBSD - BSDCan 2017 (https://www.youtube.com/watch?v=1DM5SwoXWSU) Li-Wen Hsu: Continuous Integration of The FreeBSD Project - BSDCan 2017 (https://www.youtube.com/watch?v=SCLfKWaUGa8) Andrew Turner: GENERIC ARM - BSDCan 2017 (https://www.youtube.com/watch?v=gkYjvrFvPJ0) Bjoern A. Zeeb: From the outside - BSDCan 2017 (https://www.youtube.com/watch?v=sYmW_H6FrWo) Rodney W. Grimes: FreeBSD as a Service - BSDCan 2017 (https://www.youtube.com/watch?v=Zf9tDJhoVbA) Reyk Floeter: The OpenBSD virtual machine daemon - BSDCan 2017 (https://www.youtube.com/watch?v=Os9L_sOiTH0) Brian Kidney: The Realities of DTrace on FreeBSD - BSDCan 2017 (https://www.youtube.com/watch?v=NMUf6VGK2fI) The rest will continue to trickle out, likely not until after EuroBSDCon *** Beastie Bits Oracle has killed sun (https://meshedinsights.com/2017/09/03/oracle-finally-killed-sun/) Configure Thunderbird to send patch friendly (http://nanxiao.me/en/configure-thunderbird-to-send-patch-friendly/) FreeBSD 10.4-BETA4 Available (https://www.freebsd.org/news/newsflash.html#event20170909:01) iXsystems looking to hire kernel and zfs developers (especially Sun/Oracle Refugees) (https://www.facebook.com/ixsystems/posts/10155403417921508) Speaking of job postings, UnitedBSD.com has few job postings related to BSD (https://unitedbsd.com/) Call for papers USENIX FAST ‘18 - February 12-15, 2018, Due: September 28 2017 (https://www.freebsdfoundation.org/news-and-events/call-for-papers/usenix-fast-18-call-for-papers/) Scale 16x - March 8-11, 2018, Due: October 31, 2017 (https://www.freebsdfoundation.org/news-and-events/call-for-papers/scale-16x-call-for-participation/) FOSDEM ‘18 - February 3-4, 2018, Due: November 3 2017 (https://www.freebsdfoundation.org/news-and-events/call-for-papers/fosdem-18-call-for-participation/) Feedback/Questions Jason asks about cheap router hardware (http://dpaste.com/340KRHG) Prashant asks about latest kernels with freebsd-update (http://dpaste.com/2J7DQQ6) Matt wants know about VM Performance & CPU Steal Time (http://dpaste.com/1H5SZ81) John has config questions regarding Dell precision 7720, FreeBSD, NVME, and ZFS (http://dpaste.com/0X770SY) ***

We explore whether a BSD can replicate Cisco router performance; RETGUARD, OpenBSDs new exploit mitigation technology, Dragonfly's HAMMER2 filesystem implementation & more! This episode was brought to you by Headlines Can a BSD system replicate the performance of a Cisco router? (https://www.reddit.com/r/networking/comments/6upchy/can_a_bsd_system_replicate_the_performance_of/) Short Answer: No, but it might be good enough for what you need Traditionally routers were built with a tightly coupled data plane and control plane. Back in the 80s and 90s the data plane was running in software on commodity CPUs with proprietary software. As the needs and desires for more speeds and feeds grew, the data plane had to be implemented in ASICs and FPGAs with custom memories and TCAMs. While these were still programmable in a sense, they certainly weren't programmable by anyone but a small handful of people who developed the hardware platform. The data plane was often layered, where features not handled by the hardware data plane were punted to a software only data path running on a more general CPU. The performance difference between the two were typically an order or two of magnitude. source (https://fd.io/wp-content/uploads/sites/34/2017/07/FDioVPPwhitepaperJuly2017.pdf) Except for encryption (e.g. IPsec) or IDS/IPS, the true measure of router performance is packets forwarded per unit time. This is normally expressed as Packets-per-second, or PPS. To 'line-rate' forward on a 1gbps interface, you must be able to forward packets at 1.488 million pps (Mpps). To forward at "line-rate" between 10Gbps interfaces, you must be able to forward at 14.88Mpps. Even on large hardware, kernel-forwarding is limited to speeds that top out below 2Mpps. George Neville-Neil and I did a couple papers on this back in 2014/2015. You can read the papers (https://github.com/freebsd-net/netperf/blob/master/Documentation/Papers/ABSDCon2015Paper.pdf) for the results. However, once you export the code from the kernel, things start to improve. There are a few open source code bases that show the potential of kernel-bypass networking for building a software-based router. The first of these is netmap-fwd which is the FreeBSD ip_forward() code hosted on top of netmap, a kernel-bypass technology present in FreeBSD (and available for linux). Full-disclosure, netmap-fwd was done at my company, Netgate. netmap-fwd will l3 forward around 5 Mpps per core. slides (https://github.com/Netgate/netmap-fwd/blob/master/netmap-fwd.pdf) The first of these is netmap-fwd (https://github.com/Netgate/netmap-fwd) which is the FreeBSD ip_forward() code hosted on top of netmap (https://github.com/luigirizzo/netmap), a kernel-bypass technology present in FreeBSD (and available for linux). Full-disclosure, netmap-fwd was done at my company, Netgate. (And by "my company" I mean that I co-own it with my spouse.). netmap-fwd will l3 forward around 5 Mpps per core. slides (https://github.com/Netgate/netmap-fwd/blob/master/netmap-fwd.pdf) Nanako Momiyama of the Keio Univ Tokuda Lab presented on IP Forwarding Fastpath (https://www.bsdcan.org/2017/schedule/events/823.en.html) at BSDCan this past May. She got about 5.6Mpps (roughly 10% faster than netmap-fwd) using a similar approach where the ip_foward() function was rewritten as a module for VALE (the netmap-based in-kernel switch). Slides (https://2016.eurobsdcon.org/PresentationSlides/NanakoMomiyama_TowardsFastIPForwarding.pdf) from her previous talk at EuroBSDCon 2016 are available. (Speed at the time was 2.8Mpps.). Also a paper (https://www.ht.sfc.keio.ac.jp/~nanako/conext17-sw.pdf) from that effort, if you want to read it. Of note: They were showing around 1.6Mpps even after replacing the in-kernel routing lookup algorithm with DXR. (DXR was written by Luigi Rizzo, who is also the primary author of netmap.) Not too long after netmap-fwd was open sourced, Ghandi announced packet-journey, an application based on drivers and libraries and from DPDK. Packet-journey is also an L3 router. The GitHub page for packet-journey lists performance as 21,773.47 mbps (so 21.77Gbps) for 64-byte UDP frames with 50 ACLs and 500,000 routes. Since they're using 64-byte frames, this translates to roughly 32.4Mpps. Finally, there is recent work in FreeBSD (which is part of 11.1-RELEASE) that gets performance up to 2x the level of netmap-fwd or the work by Nanako Momiyama. 10 million PPS: Here (http://blog.cochard.me/2015/09/receipt-for-building-10mpps-freebsd.html) is a decent introduction. But of course, even as FreeBSD gets up to being able to do 10gbps at line-rate, 40 and 100 gigabits are not uncommon now Even with the fastest modern CPUs, this is very little time to do any kind of meaningful packet processing. At 10Gbps, your total budget per packet, to receive (Rx) the packet, process the packet, and transmit (Tx) the packet is 67.2 ns. Complicating the task is the simple fact that main memory (RAM) is 70 ns away. The simple conclusion here is that, even at 10Gbps, if you have to hit RAM, you can't generate the PPS required for line-rate forwarding. There is some detail about design tradeoffs in the Ryzen architecture and how that might impact using those machines as routers Anyway... those are all interesting, but the natural winner here is FD.io's Vector Packet Processing (VPP). Read this (http://blogs.cisco.com/sp/a-bigger-helping-of-internet-please) VPP is an efficient, flexible open source data plane. It consists of a set of forwarding nodes arranged in a directed graph and a supporting framework. The framework has all the basic data structures, timers, drivers (and interfaces to both DPDK and netmap), a scheduler which allocates the CPU time between the graph nodes, performance and debugging tools, like counters and built-in packet trace. The latter allows you to capture the paths taken by the packets within the graph with high timestamp granularity, giving full insight into the processing on a per-packet level. The net result here is that Cisco (again, Cisco) has shown the ability to route packets at 1 Tb/s using VPP on a four socket Purley system There is also much discussion of the future of pfSense, as they transition to using VPP This is a very lengthy write up which deserves a full read, plus there are some comments from other people *** RETGUARD, the OpenBSD next level in exploit mitigation, is about to debut (https://marc.info/?l=openbsd-tech&m=150317547021396&w=2) This year I went to BSDCAN in Ottawa. I spent much of it in the 'hallway track', and had an extended conversation with various people regarding our existing security mitigations and hopes for new ones in the future. I spoke a lot with Todd Mortimer. Apparently I told him that I felt return-address protection was impossible, so a few weeks later he sent a clang diff to address that issue... The first diff is for amd64 and i386 only -- in theory RISC architectures can follow this approach soon. The mechanism is like a userland 'stackghost' in the function prologue and epilogue. The preamble XOR's the return address at top of stack with the stack pointer value itself. This perturbs by introducing bits from ASLR. The function epilogue undoes the transform immediately before the RET instruction. ROP attack methods are impacted because existing gadgets are transformed to consist of " RET". That pivots the return sequence off the ROP chain in a highly unpredictable and inconvenient fashion. The compiler diff handles this for all the C code, but the assembly functions have to be done by hand. I did this work first for amd64, and more recently for i386. I've fixed most of the functions and only a handful of complex ones remain. For those who know about polymorphism and pop/jmp or JOP, we believe once standard-RET is solved those concerns become easier to address seperately in the future. In any case a substantial reduction of gadgets is powerful. For those worried about introducing worse polymorphism with these "xor; ret" epilogues themselves, the nested gadgets for 64bit and 32bit variations are +1 "xor %esp,(%rsp); ret", +2 "and $0x24,%al; ret" and +3 "and $0xc3,%al; int3". Not bad. Over the last two weeks, we have received help and advice to ensure debuggers (gdb, egdb, ddb, lldb) can still handle these transformed callframes. Also in the kernel, we discovered we must use a smaller XOR, because otherwise userland addresses are generated, and cannot rely on SMEP as it is really new feature of the architecture. There were also issues with pthreads and dlsym, which leads to a series of uplifts around _builtinreturn_address and DWARF CFI. Application of this diff doesn't require anything special, a system can simply be built twice. Or shortcut by building & installing gnu/usr.bin/clang first, then a full build. We are at the point where userland and base are fully working without regressions, and the remaining impacts are in a few larger ports which directly access the return address (for a variety of reasons). So work needs to continue with handling the RET-addr swizzle in those ports, and then we can move forward. You can find the full message with the diff here (https://marc.info/?l=openbsd-tech&m=150317547021396&w=2) *** Interview - Ed Maste, Charlie & Siva - @ed_maste (https://twitter.com/ed_maste), @yzgyyang (https://twitter.com/yzgyyang) & @svmhdvn (https://twitter.com/svmhdvn) Co-op Students for the FreeBSD Foundation *** News Roundup Next DFly release will have an initial HAMMER2 implementation (http://lists.dragonflybsd.org/pipermail/users/2017-August/313558.html) The next DragonFly release (probably in September some time) will have an initial HAMMER2 implementation. It WILL be considered experimental and won't be an installer option yet. This initial release will only have single-image support operational plus basic features. It will have live dedup (for cp's), compression, fast recovery, snapshot, and boot support out of the gate. This first H2 release will not have clustering or multi-volume support, so don't expect those features to work. I may be able to get bulk dedup and basic mirroring operational by release time, but it won't be very efficient. Also, right now, sync operations are fairly expensive and will stall modifying operations to some degree during the flush, and there is no reblocking (yet). The allocator has a 16KB granularity (on HAMMER1 it was 2MB), so for testing purposes it will still work fairly well even without reblocking. The design is in a good place. I'm quite happy with how the physical layout turned out. Allocations down to 1KB are supported. The freemap has a 16KB granularity with a linear counter (one counter per 512KB) for packing smaller allocations. INodes are 1KB and can directly embed 512 bytes of file data for files 512 bytes. The freemap is also zoned by type for I/O locality. The blockrefs are 'fat' at 128 bytes but enormously powerful. That will allow us to ultimately support up to a 512-bit crypto hash and blind dedup using said hash. Not on release, but that's the plan. I came up with an excellent solution for directory entries. The 1KB allocation granularity was a bit high but I didn't want to reduce it. However, because blockrefs are now 128 byte entities, and directory entries are hashed just like in H1, I was able to code them such that a directory entry is embedded in the blockref itself and does not require a separate data reference or allocation beyond that. Filenames up to 64 bytes long can be accomodated in the blockref using the check-code area of the blockref. Longer filenames will use an additional data reference hanging off the blockref to accomodate up to 255 char filenames. Of course, a minimum of 1KB will have to be allocated in that case, but filenames are

In this episode, we take a look at the reimplementation of NetBSD using a Microkernel, check out what makes DHCP faster, and see what high-process count support for DragonflyBSD has to offer, and we answer the questions you've always wanted to ask us. This episode was brought to you by Headlines A Reimplementation Of Netbsd Using a Microkernel (http://theembeddedboard.review/a-reimplementation-of-netbsd-using-a-microkernel-part-1-of-2/) Minix author Andy Tanenbaum writes in Part 1 of a-reimplementation-of-netbsd-using-a-microkernel (http://theembeddedboard.review/a-reimplementation-of-netbsd-using-a-microkernel-part-1-of-2/) Based on the MINIX 3 microkernel, we have constructed a system that to the user looks a great deal like NetBSD. It uses pkgsrc, NetBSD headers and libraries, and passes over 80% of the KYUA tests). However, inside, the system is completely different. At the bottom is a small (about 13,000 lines of code) microkernel that handles interrupts, message passing, low-level scheduling, and hardware related details. Nearly all of the actual operating system, including memory management, the file system(s), paging, and all the device drivers run as user-mode processes protected by the MMU. As a consequence, failures or security issues in one component cannot spread to other ones. In some cases a failed component can be replaced automatically and on the fly, while the system is running, and without user processes noticing it. The talk will discuss the history, goals, technology, and status of the project. Research at the Vrije Universiteit has resulted in a reimplementation of NetBSD using a microkernel instead of the traditional monolithic kernel. To the user, the system looks a great deal like NetBSD (it passes over 80% of the KYUA tests). However, inside, the system is completely different. At the bottom is a small (about 13,000 lines of code) microkernel that handles interrupts, message passing, low-level scheduling, and hardware related details. Nearly all of the actual operating system, including memory management, the file system(s), paging, and all the device drivers run as user-mode processes protected by the MMU. As a consequence, failures or security issues in one component cannot spread to other ones. In some cases a failed component can be replaced automatically and on the fly, while the system is running. The latest work has been adding live update, making it possible to upgrade to a new version of the operating system WITHOUT a reboot and without running processes even noticing. No other operating system can do this. The system is built on MINIX 3, a derivative of the original MINIX system, which was intended for education. However, after the original author, Andrew Tanenbaum, received a 2 million euro grant from the Royal Netherlands Academy of Arts and Sciences and a 2.5 million euro grant from the European Research Council, the focus changed to building a highly reliable, secure, fault tolerant operating system, with an emphasis on embedded systems. The code is open source and can be downloaded from www.minix3.org. It runs on the x86 and ARM Cortex V8 (e.g., BeagleBones). Since 2007, the Website has been visited over 3 million times and the bootable image file has been downloaded over 600,000 times. The talk will discuss the history, goals, technology, and status of the project. Part 2 (http://theembeddedboard.review/a-reimplementation-of-netbsd-using-a-microkernel-part-2-of-2/) is also available. *** Rapid DHCP: Or, how do Macs get on the network so fast? (https://cafbit.com/post/rapid_dhcp_or_how_do/) One of life's minor annoyances is having to wait on my devices to connect to the network after I wake them from sleep. All too often, I'll open the lid on my EeePC netbook, enter a web address, and get the dreaded "This webpage is not available" message because the machine is still working on connecting to my Wi-Fi network. On some occasions, I have to twiddle my thumbs for as long as 10-15 seconds before the network is ready to be used. The frustrating thing is that I know it doesn't have to be this way. I know this because I have a Mac. When I open the lid of my MacBook Pro, it connects to the network nearly instantaneously. In fact, no matter how fast I am, the network comes up before I can even try to load a web page. My curiosity got the better of me, and I set out to investigate how Macs are able to connect to the network so quickly, and how the network connect time in other operating systems could be improved. I figure there are three main categories of time-consuming activities that occur during network initialization: Link establishment. This is the activity of establishing communication with the network's link layer. In the case of Wi-Fi, the radio must be powered on, the access point detected, and the optional encryption layer (e.g. WPA) established. After link establishment, the device is able to send and receive Ethernet frames on the network. Dynamic Host Configuration Protocol (DHCP). Through DHCP handshaking, the device negotiates an IP address for its use on the local IP network. A DHCP server is responsible for managing the IP addresses available for use on the network. Miscellaneous overhead. The operating system may perform any number of mundane tasks during the process of network initialization, including running scripts, looking up preconfigured network settings in a local database, launching programs, etc. My investigation thus far is primarily concerned with the DHCP phase, although the other two categories would be interesting to study in the future. I set up a packet capture environment with a spare wireless access point, and observed the network activity of a number of devices as they initialized their network connection. For a worst-case scenario, let's look at the network activity captured while an Android tablet is connecting: This tablet, presumably in the interest of "optimization", is initially skipping the DHCP discovery phase and immediately requesting its previous IP address. The only problem is this is a different network, so the DHCP server ignores these requests. After about 4.5 seconds, the tablet stubbornly tries again to request its old IP address. After another 4.5 seconds, it resigns itself to starting from scratch, and performs the DHCP discovery needed to obtain an IP address on the new network. In all fairness, this delay wouldn't be so bad if the device was connecting to the same network as it was previously using. However, notice that the tablet waits a full 1.13 seconds after link establishment to even think about starting the DHCP process. Engineering snappiness usually means finding lots of small opportunities to save a few milliseconds here and there, and someone definitely dropped the ball here. In contrast, let's look at the packet dump from the machine with the lightning-fast network initialization, and see if we can uncover the magic that is happening under the hood: The key to understanding the magic is the first three unicast ARP requests. It looks like Mac OS remembers certain information about not only the last connected network, but the last several networks. In particular, it must at least persist the following tuple for each of these networks: > 1. The Ethernet address of the DHCP server > 2. The IP address of the DHCP server > 3. Its own IP address, as assigned by the DHCP server During network initialization, the Mac transmits carefully crafted unicast ARP requests with this stored information. For each network in its memory, it attempts to send a request to the specific Ethernet address of the DHCP server for that network, in which it asks about the server's IP address, and requests that the server reply to the IP address which the Mac was formerly using on that network. Unless network hosts have been radically shuffled around, at most only one of these ARP requests will result in a response—the request corresponding to the current network, if the current network happens to be one of the remembered networks. This network recognition technique allows the Mac to very rapidly discover if it is connected to a known network. If the network is recognized (and presumably if the Mac knows that the DHCP lease is still active), it immediately and presumptuously configures its IP interface with the address it knows is good for this network. (Well, it does perform a self-ARP for good measure, but doesn't seem to wait more than 13ms for a response.) The DHCP handshaking process begins in the background by sending a DHCP request for its assumed IP address, but the network interface is available for use during the handshaking process. If the network was not recognized, I assume the Mac would know to begin the DHCP discovery phase, instead of sending blind requests for a former IP address as the Galaxy Tab does. The Mac's rapid network initialization can be credited to more than just the network recognition scheme. Judging by the use of ARP (which can be problematic to deal with in user-space) and the unusually regular transmission intervals (a reliable 1.0ms delay between each packet sent), I'm guessing that the Mac's DHCP client system is entirely implemented as tight kernel-mode code. The Mac began the IP interface initialization process a mere 10ms after link establishment, which is far faster than any other device I tested. Android devices such as the Galaxy Tab rely on the user-mode dhclient system (part of the dhcpcd package) dhcpcd program, which no doubt brings a lot of additional overhead such as loading the program, context switching, and perhaps even running scripts. The next step for some daring kernel hacker is to implement a similarly aggressive DHCP client system in the Linux kernel, so that I can enjoy fast sign-on speeds on my Android tablet, Android phone, and Ubuntu netbook. There already exists a minimal DHCP client implementation in the Linux kernel, but it lacks certain features such as configuring the DNS nameservers. Perhaps it wouldn't be too much work to extend this code to support network recognition and interface with a user-mode daemon to handle such auxillary configuration information received via DHCP. If I ever get a few spare cycles, maybe I'll even take a stab at it. You can also find other ways of optimizing the dhclient program and how it works in the dhclient tutorial on Calomel.org (https://calomel.org/dhclient.html). *** BSDCam Trip Report (https://www.freebsdfoundation.org/blog/bsdcam-2017-trip-report-michael-lucas/) Over the decades, FreeBSD development and coordination has shifted from being purely on-line to involving more and more in-person coordination and cooperation. The FreeBSD Foundation sponsors a devsummit right before BSDCan, EuroBSDCon, and AsiaBSDCon, so that developers traveling to the con can leverage their airfare and hammer out some problems. Yes, the Internet is great for coordination, but nothing beats a group of developers spending ten minutes together to sketch on a whiteboard and figuring out exactly how to make something bulletproof. In addition to the coordination efforts, though, conference devsummits are hierarchical. There's a rigid schedule, with topics decided in advance. Someone leads the session. Sessions can be highly informative, passionate arguments, or anything in between. BSDCam is… a little different. It's an invaluable part of the FreeBSD ecosystem. However, it's something that I wouldn't normally attend. But right now, is not normal. I'm writing a new edition of Absolute FreeBSD. To my astonishment, people have come to rely on this book when planning their deployments and operations. While I find this satisfying, it also increases the pressure on me to get things correct. When I wrote my first FreeBSD book back in 2000, a dozen mailing lists provided authoritative information on FreeBSD development. One person could read every one of those lists. Today, that's not possible—and the mailing lists are only one narrow aspect of the FreeBSD social system. Don't get me wrong—it's pretty easy to find out what people are doing and how the system works. But it's not that easy to find out what people will be doing and how the system will work. If this book is going to be future-proof, I needed to leave my cozy nest and venture into the wilds of Cambridge, England. Sadly, the BSDCam chair agreed with my logic, so I boarded an aluminum deathtrap—sorry, a “commercial airliner”—and found myself hurtled from Detroit to Heathrow. And one Wednesday morning, I made it to the William Gates building of Cambridge University, consciousness nailed to my body by a thankfully infinite stream of proper British tea. BSDCam attendance is invitation only, and the facilities can only handle fifty folks or so. You need to be actively working on FreeBSD to wrangle an invite. Developers attend from all over the world. Yet, there's no agenda. Robert Watson is the chair, but he doesn't decide on the conference topics. He goes around the room and asks everyone to introduce themselves, say what they're working on, and declare what they want to discuss during the conference. The topics of interest are tallied. The most popular topics get assigned time slots and one of the two big rooms. Folks interested in less popular topics are invited to claim one of the small breakout rooms. Then the real fun begins. I started by eavesdropping in the virtualization workshop. For two hours, people discussed FreeBSD's virtualization needs, strengths, and weaknesses. What needs help? What should this interface look like? What compatibility is important, and what isn't? By the end of the session, the couple dozen people had developed a reasonable consensus and, most importantly, some folks had added items to their to-do lists. Repeat for a dozen more topics. I got a good grip on what's really happening with security mitigation techniques, FreeBSD's cloud support, TCP/IP improvements, advances in teaching FreeBSD, and more. A BSDCan devsummit presentation on packaging the base system is informative, but eavesdropping on two dozen highly educated engineers arguing about how to nail down the final tidbits needed to make that a real thing is far more educational. To my surprise, I was able to provide useful feedback for some sessions. I speak at a lot of events outside of the FreeBSD world, and was able to share much of what I hear at Linux conferences. A tool that works well for an experienced developer doesn't necessarily work well for everyone. Every year, I leave BSDCan tired. I left BSDCam entirely exhausted. These intense, focused discussions stretched my brain. But, I have a really good idea where key parts of FreeBSD development are actually headed. This should help future-proof the new Absolute FreeBSD, as much as any computer book can be future-proof. Plus, BSDCam throws the most glorious conference dinner I've ever seen. I want to thank Robert Watson for his kind invitation, and the FreeBSD Foundation for helping defray the cost of this trip Interview - The BSDNow Crew As a kid, what did you dream of to become as an adult? JT: An Astronaut BR: I wanted to be a private detective, because of all the crime novels that I read back then. I didn't get far with it. However, I think the structured analysis skills (who did what, when, and such) help me in debugging and sysadmin work. AJ: Didn't think about it much How do you manage to stay organized day to day with so much things you're actively doing each day? (Day job, wife/girlfriend, conferences, hobbies, friends, etc.) JT: Who said I was organized? BR: A lot of stuff in my calendar as reminders, open browser tabs as “to read later” list. A few things like task switching when getting stuck helps. Also, focus on a single goal for the day, even though there will be distractions. Slowly, but steadily chip away at the things you're working on. Rather than to procrastinate and put things back to review later, get started early with easy things for a big task and then tackle the hard part. Often, things look totally chaotic and unmanageable, until you start working on them. AJ: I barely manage. Lots of Google Calendar reminders, and the entire wall of my office is covered in whiteboard sheet todo lists. I use pinboard.in to deal with finding and organizing bookmarks. Write things down, don't trust your memory. What hobbies outside of IT do you have? JT: I love photography, but I do that Professional part time, so I'm not sure if that counts as a hobby anymore. I guess it'd have to be working in the garage on my cars. BR: I do Tai Chi to relax once a week in a group, but can also do it alone, pretty much everywhere. Way too much Youtube watching and browsing the web. I did play some games before studying at the university and I'm still proud that I could control it to the bare minimum not to impact my studies. A few “lapses” from time to time, revisiting the old classics since the newer stuff won't run on my machines anyway. Holiday time is pretty much spent for BSD conferences and events, this is where I can relax and talk with like-minded people from around the world, which is fascinating. Plus, it gets me to various places and countries I never would have dared to visit on my own. AJ: I play a few video games, and I like to ski, although I don't go very often as most of my vacation time is spent hanging out with my BSD friends at various conferences How do you relax? JT: What is this word ‘relax' and what does it mean? BR: My Tai Chi plays a big part in it I guess. I really calms you and the constant stream of thoughts for a while. It also gives you better clarity of what's important in life. Watching movies, sleeping long. AJ: Usually watching TV or Movies. Although I have taken to doing most of my TV watching on my exercise bike now, but it is still mentally relaxing If FreeBSD didn't exist, which BSD flavour would you use? Why? JT: I use TrueOS, but if FreeBSD didn't exist, that project might not either… so… My other choice would be HardenedBSD, but since it's also based on FreeBSD I'm in the same dillema. BR: I once installed NetBSD to see what It can do. If FreeBSD wouldn't exist, I would probably try my luck with it. OpenBSD is also appealing, but I've never installed it. AJ: When I started using FreeBSD in 2000, the only other BSD I had heard of at the time was OpenBSD. If FreeBSD wasn't around, I don't think the world would look like it does, so it is hard to speculate. If any of the BSD's weren't around and you had to use Linux, which camp would belong to? (Redhat, SUSE, Debian, Ubuntu, Gentoo?) JT: I learned Linux in the mid 90s using Slackware, which I used consistently up until the mid 2000s, when I joined the PuppyLinux community and eventually became a developer (FYI, Puppy was/is/can be based on Slackware -- its complicated). So I'd go back to using either Slackware or PuppyLinux. BR: I tried various Linux distributions until I landed at Debian. I used is pretty extensively as my desktop OS at home, building custom kernels and packages to install them until I discovered FreeBSD. I ran both side by side for a few months for learning until one day I figured out that I had not booted Debian in a while, so I switched completely. AJ: The first Linux I played with was Slackware, and it is the most BSD like, but the bits of Linux I learned in school were Redhat and so I can somewhat wrap my head around it, although now that they are changing everything to systemd, all of that old knowledge is more harmful than useful. Are you still finding yourself in need to use Windows/Mac OS? Why? JT: I work part time as a professional Photographer, so I do use Windows for my photography work. While I can do everything I need to do in Linux, it comes down to being pragmatic about my time. What takes me several hours to accomplish in Linux I can accomplish in 20 minutes on Windows. BR: I was a long time Windows-only user before my Unix days. But back when Vista was about to come out and I needed a new laptop, my choice was basically learning to cope with Vistas awful features or learn MacOS X. I did the latter, it increased my productivity since it's really a good Unix desktop experience (at least, back then). I only have to use Windows at work from time to time as I manage our Windows Terminal server, which keeps the exposure low enough and I only connect to it to use a certain app not available for the Mac or the BSDs. AJ: I still use Windows to play games, for a lot of video conferencing, and to produce BSD Now. Some of it could be done on BSD but not as easily. I have promised myself that I will switch to 100% BSD rather than upgrade to Windows 10, so we'll see how that goes. Please describe your home networking setup. Router type, router OS, router hardware, network segmentation, wifi apparatus(es), other devices connected, and anything else that might be interesting about your home network. BR: Very simple and boring: Apple Airport Express base station and an AVM FritzBox for DNS, DHCP, and the link to my provider. A long network cable to my desktop machine. That I use less and less often. I just bought an RPI 3 for some home use in the future to replace it. Mostly my brother's and my Macbook Pro's are connected, our phones and the iPad of my mother. AJ: I have a E3-1220 v3 (dual 3.1ghz + HT) with 8 GB of ram, and 4x Intel gigabit server NICs as my router, and it runs vanilla FreeBSD (usually some snapshot of -current). I have 4 different VLANs, Home, Office, DMZ, and Guest WiFi. WiFi is served via a tiny USB powered device I bought in Tokyo years ago, it serves 3 different SSIDs, one for each VLAN except the DMZ. There are ethernet jacks in every room wired for 10 gigabit, although the only machines with 10 gigabit are my main workstation, file server, and some machines in the server rack. There are 3 switches, one for the house (in the laundry room), one for the rack, and one for 10gig stuff. There is a rack in the basement spare bedroom, it has 7 servers in it, mostly storage for live replicas of customer data for my company. How do guys manage to get your work done on FreeBSD desktops? What do you do when you need to a Linux or Windows app that isn't ported, or working? I've made several attempts to switch to FreeBSD, but each attempt failed because of tools not being available (e.g. Zoom, Dropbox, TeamViewer, Crashplan) or broken (e.g. VirtualBox). BR: I use VIrtualBox for everything that is not natively available or Windows-only. Unfortunately, that means no modern games. I mostly do work in the shell when I'm on FreeBSD and when it has to be a graphical application, then I use Fluxbox as the DE. I want to get work done, not look at fancy eye-candy that get's boring after a while. Deactivated the same stuff on my mac due to the same reason. I look for alternative software online, but my needs are relatively easy to satisfy as I'm not doing video editing/rendering and such. AJ: I generally find that I don't need these apps. I use Firefox, Thunderbird, OpenSSH, Quassel, KomodoEdit, and a few other apps, so my needs are not very demanding. It is annoying when packages are broken, but I usually work around this with boot environments, and being able to just roll back to a version that worked for a few days until the problem is solved. I do still have access to a windows machine for the odd time I need specific VPN software or access to Dell/HP etc out-of-band management tools. Which desktop environments are your favorite, and why? For example, I like i3, Xfce, and I'm drawn to Lumina's ethos, but so far always seem to end up back on Xfc because of its ease of use, flexibility, and dashing good looks. JT: As a Lumina Desktop developer, I think my preference is obvious. ;) I am also a long timeOpenBox user, so I have a soft place in my heart for that as well. BR: I use Fluxbox when I need to work with a lot of windows or an application demands X11. KDE and others are too memory heavy for me and I rarely use even 20% of the features they provide. AJ: I was a long time KDE user, but I have adopted Lumina. I find it fast, and that it gets out of my way and lets me do what I want. It had some annoyances early on, but I've nagged the developers into making it work for me. Which command-line shells do you prefer, why, and how (if at all) have you customised the environment or prompt? BR: I use zsh, but without all the fancy stuff you can find online. It might make you more productive, yes. But again, I try to keep things simple. I'm slowly learning tmux and want to work more in it in the future. I sometimes look at other BSD people's laptops and am amazed at what they do with window-management in tmux. My prompt looks like this: bcr@Voyager:~> 20:20 17-08-17 Put this in your .zshrc to get the same result: PROMPT='%n@%m:%~>' RPROMPT='%T %D' AJ: I started using tcsh early on, because it was the shell on the first box I had access to, and because one of the first things I read in “BSD Hacks” was how to enable ‘typo correction”, which made my life a lot better especially on dial up in the early days. My shell prompt looks like this: allan@CA-TOR1-02:/usr/home/allan% What is one thing (or more) missing in FreeBSD you would import from another project or community? Could be tech, process, etc. JT: AUFS from Linux BR: Nohup from Illumos where you can detach an already running process and put it in the background. I often forget that and I'm not in tmux when that happens, so I can see myself use that feature a lot. AJ: Zones (more complete Jails) from IllumOS how do you manage your time to learn about and work on FreeBSD? Does your work/employment enable what you do, or are your contributions mainly done in private time? JT: These days I'm mostly learning things I need for work, so it just falls into something I'm doing while working on work projects. BR: We have a lot of time during the semester holidays to learn on our own, it's part of the idea of being in a university to keep yourself updated, at least for me. Especially in the fast moving world of IT. I also read a lot in my free time. My interests can shift sometimes, but then I devour everything I can find on the topic. Can be a bit excessive, but has gotten me where I am now and I still need a lot to learn (and want to). Since I work with FreeBSD at work (my owndoing), I can try out many things there. AJ: My work means a spend a lot of time working with FreeBSD, but not that much time working ON it. My contributions are mostly done outside of work, but as I own the company I do get more flexibility to take time off for conferences and other FreeBSD related stuff. we know we can bribe Michael W Lucas with gelato (good gelato that is), but what can we use to bribe you guys? Like when I want to have Allan to work on fixing a bug which prevents me from running ZFS on this fancy rock64 board? BR: Desserts of various kinds. AJ: I am probably not the right person to look at your rock64 board. Most people in the project have taken to bribing me with chocolate. In general, my todo list is so long, the best way is a trade, you take this task and I'll take that task. Is your daily mobile device iOS, Android, Windows Mobile, or other? Why? JT: These days I'm using Android on my Blackberry Priv, but until recently I was still a heavy user of Sailfish OS. I would use SailfishOS everyday, if I could find a phone with a keyboard that I could run it on. BR: iOS on the iPhone 7 currently. Never used an Android phone, saw it on other people's devices and what they can do with it (much more). But the infrequent security updates (if any at all) keep me away from it. AJ: I have a Google Nexus 6 (Android 7.1). I wanted the ‘pure' Android experience, and I had been happy with my previous Nexus S. I don't run a custom OS/ROM or anything because I use the phone to verify that video streams work on an ‘average users device'. I am displeased that support for my device will end soon. I am not sure what device I will get next, but it definitely won't be an iPhone. News Roundup Beta Update - Request for (more) Testing (http://undeadly.org/cgi?action=article&sid=20170808065718&mode=flat&count=30) https://beta.undeadly.org/ has received an update. The most significant changes include: The site has been given a less antiquated "look". (As the topic icons have been eliminated, we are no longer seeking help with those graphics.) The site now uses a moderate amount of semantic HTML5. Several bugs in the HTML fragment validator (used for submissions and comments) have been fixed. To avoid generating invalid HTML, submission content which fails validation is no longer displayed in submission/comment previews. Plain text submissions are converted to HTML in a more useful fashion. (Instead of just converting each EOL to , the converter now generates proper paragraphs and interprets two or more consecutive EOLs as indicating a paragraph break.) The redevelopment remains a work-in-progress. Many thanks to those who have contributed! As before, constructive feedback would be appreciated. Of particular interest are reports of bugs in behaviour (for example, in the HTML validator or in authentication) that would preclude the adoption of the current code for the main site. High-process-count support added to master (http://lists.dragonflybsd.org/pipermail/users/2017-August/313552.html) We've fixed a number of bottlenecks that can develop when the number of user processes runs into the tens of thousands or higher. One thing led to another and I said to myself, "gee, we have a 6-digit PID, might as well make it work to a million!". With the commits made today, master can support at least 900,000 processes with just a kern.maxproc setting in /boot/loader.conf, assuming the machine has the memory to handle it. And, in fact, as today's machines start to ratchet up there in both memory capacity and core count, with fast storage (NVMe) and fast networking (10GigE and higher), even in consumer boxes, this is actually something that one might want to do. With AMD's threadripper and EPYC chips now out, the IntelAMD cpu wars are back on! Boasting up to 32 cores (64 threads) per socket and two sockets on EPYC, terabytes of ram, and motherboards with dual 10GigE built-in, the reality is that these numbers are already achievable in a useful manner. In anycase, I've tested these changes on a dual-socket xeon. I can in-fact start 900,000 processes. They don't get a whole lot of cpu and running 'ps' would be painful, but it works and the system is still responsive from the shell with all of that going on. xeon126# uptime 1:42PM up 9 mins, 3 users, load averages: 890407.00, 549381.40, 254199.55 In fact, judging from the memory use, these minimal test processes only eat around 60KB each. 900,000 of them ate only 55GB on a 128GB machine. So even a million processes is not out of the question, depending on the cpu requirements for those processes. Today's modern machines can be stuffed with enormous amounts of memory. Of course, our PIDs are currently limited to 6 digits, so a million is kinda the upper limit in terms of discrete user processes (verses pthreads which are less restricted). I'd rather not go to 7 digits (yet). CFT: Driver for generic MS Windows 7/8/10 - compatible USB HID multi-touch touchscreens (https://lists.freebsd.org/pipermail/freebsd-current/2017-August/066783.html) Following patch [1] adds support for generic MS Windows 7/8/10 - compatible USB HID multi-touch touchscreens via evdev protocol. It is intended to be a native replacement of hid-multitouch.c driver found in Linux distributions and multimedia/webcamd port. Patch is made for 12-CURRENT and most probably can be applied to recent 11-STABLE and 11.1-RELEASE (not tested) How to test" 1. Apply patch [1] 2. To compile this driver into the kernel, place the following lines into your kernel configuration file: device wmt device usb device evdev Alternatively, to load the driver as a module at boot time, place the following line in loader.conf(5): wmt_load="YES" 3. Install x11-drivers/xf86-input-evdev or x11-drivers/xf86-input-libinput port 4. Tell XOrg to use evdev or libinput driver for the device: ``` Section "ServerLayout" InputDevice "TouchScreen0" "SendCoreEvents" EndSection Section "InputDevice" Identifier "TouchScreen0" Driver "evdev" # Driver "libinput" Option "Device" "/dev/input/eventXXX" EndSection ``` Exact value of "/dev/input/eventXXX" can be obtained with evemu-record utility from devel/evemu. Note1: Currently, driver does not support pens or touchpads. Note2: wmt.ko should be kld-loaded before uhid driver to take precedence over it! Otherwise uhid can be kld-unloaded after loading of wmt. wmt review: https://reviews.freebsd.org/D12017 Raw diff: https://reviews.freebsd.org/D12017.diff *** Beastie Bits BSDMag Programing Languages Infographic (https://bsdmag.org/programm_history/) t2k17 Hackathon Report: Bob Beck on buffer cache tweaks, libressl and pledge progress (http://undeadly.org/cgi?action=article&sid=20170815171854) New FreeBSD Journal (https://www.freebsdfoundation.org/past-issues/resource-control/) NetBSD machines at Open Source Conference 2017 Kyoto (http://mail-index.netbsd.org/netbsd-advocacy/2017/08/10/msg000744.html) *** Feedback/Questions Dan - HDD question (http://dpaste.com/3H6TDJV) Benjamin - scrub of death (http://dpaste.com/10F086V) Jason - Router Opinion (http://dpaste.com/2D9102K) Sohrab - Thanks (http://dpaste.com/1XYYTWF) ***

We look at an OpenBSD setup on a new laptop, revel in BSDCan trip reports, and visit daemons and friendly ninjas. This episode was brought to you by Headlines OpenBSD and the modern laptop (http://bsdly.blogspot.de/2017/07/openbsd-and-modern-laptop.html) Peter Hansteen has a new blog post about OpenBSD (http://www.openbsd.org/) on laptops: Did you think that OpenBSD is suitable only for firewalls and high-security servers? Think again. Here are my steps to transform a modern mid to high range laptop into a useful Unix workstation with OpenBSD. One thing that never ceases to amaze me is that whenever I'm out and about with my primary laptop at conferences and elsewhere geeks gather, a significant subset of the people I meet have a hard time believing that my laptop runs OpenBSD, and that it's the only system installed. and then it takes a bit of demonstrating that yes, the graphics runs with the best available resolution the hardware can offer, the wireless network is functional, suspend and resume does work, and so forth. And of course, yes, I do use that system when writing books and articles too. Apparently heavy users of other free operating systems do not always run them on their primary workstations. Peter goes on to describe the laptops he's had over the years (all running OpenBSD) and after BSDCan 2017, he needed a new one due to cracks in the display. So the time came to shop around for a replacement. After a bit of shopping around I came back to Multicom, a small computers and parts supplier outfit in rural Åmli in southern Norway, the same place I had sourced the previous one. One of the things that attracted me to that particular shop and their own-branded offerings is that they will let you buy those computers with no operating system installed. That is of course what you want to do when you source your operating system separately, as we OpenBSD users tend to do. The last time around I had gone for a "Thin and lightweight" 14 inch model (Thickness 20mm, weight 2.0kg) with 16GB RAM, 240GB SSD for system disk and 1TB HD for /home (since swapped out for a same-size SSD, as the dmesg will show). Three years later, the rough equivalent with some added oomph for me to stay comfortable for some years to come ended me with a 13.3 inch model, 18mm and advertised as 1.3kg (but actually weighing in at 1.5kg, possibly due to extra components), 32GB RAM, 512GB SSD and 2TB harddisk. For now the specification can be viewed online here (https://www.multicom.no/systemconfigurator.aspx?q=st:10637291;c:100559;fl:0#4091-10500502-1;4086-10637290-1;4087-8562157-2;4088-9101982-1;4089-9101991-1) (the site language is Norwegian, but product names and units of measure are not in fact different). The OpenBSD installer is a wonder of straightforward, no-nonsense simplicity that simply gets the job done. Even so, if you are not yet familiar with OpenBSD, it is worth spending some time reading the OpenBSD FAQ's installation guidelines and the INSTALL.platform file (in our case, INSTALL.amd64) to familiarize yourself with the procedure. If you're following this article to the letter and will be installing a snapshot, it is worth reading the notes on following -current too. The main hurdle back when I was installing the 2014-vintage 14" model was getting the system to consider the SSD which showed up as sd1 the automatic choice for booting (I solved that by removing the MBR, setting the size of the MBR on the hard drive that showed up as sd0 to 0 and enlarging the OpenBSD part to fill the entire drive). + He goes on to explain the choices he made in the installer and settings made after the reboot to set up his work environment. Peter closes with: If you have any questions on running OpenBSD as a primary working environment, I'm generally happy to answer but in almost all cases I would prefer that you use the mailing lists such as misc@openbsd.org or the OpenBSD Facebook (https://www.facebook.com/groups/2210554563/) group so the question and hopefully useful answers become available to the general public. Browsing the slides for my recent OpenBSD and you (https://home.nuug.no/~peter/openbsd_and_you/) user group talk might be beneficial if you're not yet familiar with the system. And of course, comments on this article are welcome. BSDCan 2017 Trip Report: Roller Angel (https://www.freebsdfoundation.org/blog/2017-bsdcan-trip-report-roller-angel/) We could put this into next week's show, because we have another trip report already that's quite long. After dropping off my luggage, I headed straight over to the Goat BoF which took place at The Royal Oak. There were already a number of people there engaged in conversation with food and drink. I sat down at a table and was delighted that the people sitting with me were also into the BSD's and were happy to talk about it the whole time. I felt right at home from the start as people were very nice to me, and were interested in what I was working on. I honestly didn't know that I would fit in so well. I had a preconceived notion that people may be a bit hard to approach as they are famous and so technically advanced. At first, people seemed to only be working in smaller circles. Once you get more familiar with the faces, you realize that these circles don't always contain the same people and that they are just people talking about specific topics. I found that it was easy to participate in the conversation and also found out that people are happy to get your feedback on the subject as well. I was actually surprised how easily I got along with everyone and how included I felt in the activities. I volunteered to help wherever possible and got to work on the video crew that recorded the audio and slides of the talks. The people at BSDCan are incredibly easy to talk to, are actually interested in what you're doing with BSD, and what they can do to help. It's nice to feel welcome in the community. It's like going home. Dan mentioned in his welcome on the first day of BSDCan that the conference is like home for many in the community. The trip report is very detailed and chronicles the two days of the developer summit, and the two days of the conference There was some discussion about a new code of conduct by Benno Rice who mentioned that people are welcome to join a body of people that is forming that helps work out issues related to code of conduct and forwards their recommendations on to core. Next, Allan introduced the idea of creating a process for formally discussing big project changes or similar discussions that is going to be known as FCP or FreeBSD Community Proposal. In Python we have the Python Enhancement Proposal or PEP which is very similar to the idea of FCP. I thought this idea is a great step for FreeBSD to be implementing as it has been a great thing for Python to have. There was some discussion about taking non-code contributions from people and how to recognize those people in the project. There was a suggestion to have a FreeBSD Member status created that can be given to people whose non-code contributions are valuable to the project. This idea seemed to be on a lot of people's minds as something that should be in place soon. The junior jobs on the FreeBSD Wiki were also brought up as a great place to look for ideas on how to get involved in contributing to FreeBSD. Roller wasted no time, and started contributing to EdgeBSD at the conference. On the first day of BSDCan I arrived at the conference early to coordinate with the team that records the talks. We selected the rooms that each of us would be in to do the recording and set up a group chat via WhatsApp for coordination. Thanks to Roller, Patrick McAvoy, Calvin Hendryx-Parker, and all of the others who volunteered their time to run the video and streaming production at BSDCan, as well as all others who volunteered, even if it was just to carry a box. BSDCan couldn't happen without the army of volunteers. After the doc lounge, I visited the Hacker Lounge. There were already several tables full of people talking and working on various projects. In fact, there was a larger group of people who were collaborating on the new libtrue library that seemed to be having a great time. I did a little socializing and then got on my laptop and did some more work on the documentation using my new skills. I really enjoyed having a hacker lounge to go to at night. I want to give a big thank you to the FreeBSD Foundation for approving my travel grant. It was a great experience to meet the community and participate in discussions. I'm very grateful that I was able to attend my first BSDCan. After visiting the doc lounge a few times, I managed to get comfortable using the tools required to edit the documentation. By the end of the conference, I had submitted two documentation patches to the FreeBSD Bugzilla with several patches still in progress. Prior to the conference I expected that I would be spending a lot of time working on my Onion Omega and Edge Router Lite projects that I had with me, but I actually found that there was always something fun going on that I would rather do or work on. I can always work on those projects at home anyway. I had a good time working with the FreeBSD community and will continue working with them by editing the documentation and working with Bugzilla. One of the things I enjoy about these trip reports is when they help convince other people to make the trip to their first conference. Hopefully by sharing their experience, it will convince you to come to the next conference: vBSDCon in Virginia, USA: Sept 7-9 EuroBSDCon in Paris, France: Sept 21-24 BSDTW in Taipei, Taiwan: November 11-12 (CFP ends July 31st) *** BSDCan 2017 - Trip report double-p (http://undeadly.org/cgi?action=article&sid=20170629150641) Prologue Most overheard in Tokyo was "see you in Ottawaaaaah", so with additional "personal item" being Groff I returned home to plan the trip to BSDCan. Dan was very helpful with getting all the preparations (immigration handling), thanks for that. Before I could start, I had to fix something: the handling of the goat. With a nicely created harness, I could just hang it along my backpack. Done that it went to the airport of Hamburg and check-in for an itinerary of HAM-MUC-YUL. While the feeder leg was a common thing, boarding to YUL was great - cabin-crew likes Groff :) Arriving in Montreal was like entering a Monsoon zone or something, sad! After the night the weather was still rain-ish but improving and i shuttled to Dorval VIARail station to take me to Ottawa (ever avoid AirCanada, right?). Train was late, but the conductor (or so) was nice to talk to - and wanted to know about Groff's facebook page :-P. Picking a cab in Ottawa to take me to "Residence" was easy at first - just that it was the wrong one. Actually my fault and so I had a "nice, short" walk to the actual one in the rain with wrong directions. Eventually I made it and after unpacking, refreshment it was time to hit the Goat BOF! Day 1 Since this was my first BSDCan I didnt exactly knew what to expect from this BOF. But it was like, we (Keeper, Dan, Allan, ..) would talk about "who's next" and things like that. How mistaken I was :). Besides the sheer amount of BSD people entering the not-so-yuuge Oak some Dexter sneaked in camouflage. The name-giver got a proper position to oversee the mess and I was glad I did not leave him behind after almost too many Creemores. Day 2 Something happened it's crystal blue on the "roof" and sun is trying its best to wake me up. To start the day, I pick breakfast at 'Father+Sons' - I can really recommend that. Very nice home made fries (almost hashbrowns) and fast delivery! Stuffed up I trott along to get to phessler's tutorial about BGP-for-sysadmins-and-developers. Peter did a great job, but the "lab" couldn't happen, since - oh surprise - the wifi was sluggish as hell. Must love the first day on a conference every time. Went to Hackroom in U90 afterwards, just to fix stuff "at home". IPsec giving pains again. Time to pick food+beer afterwards and since it's so easy to reach, we went to the Oak again. Having a nice backyard patio experience it was about time to meet new people. Cheers to Tom, Aaron, Nick, Philip and some more, we'd an awesome night there. I also invited some not-really-computer local I know by other means who was completly overwhelmed by what kind of "nerds" gather around BSD. He planned to stay "a beer" - and it was rather some more and six hours. Looks like "we" made some impression on him :). Day 3 Easy day, no tutorials at hand, so first picking up breakfast at F+S again and moving to hackroom in U90. Since I promised phessler to help with an localized lab-setup, I started to hack on a quick vagrant/ansible setup to mimic his BGP-lab and went quickly through most of it. Plus some more IPsec debugging and finally fixing it, we went early in the general direction of the Red Lion to pick our registration pack. But before that could happen it was called to have shawarma at 3brothers along. Given a tight hangover it wasn't the brightest idea to order a poutine m-(. Might be great the other day, it wasn't for me at the very time and had to throw away most of it :(. Eventually passing on to the Red Lion I made the next failure with just running into the pub - please stay at the front desk until "seated". I never get used to this concept. So after being "properly" seated, we take our beers and the registration can commence after we had half of it. So I register myself; btw it's a great idea to grant "not needed" stuff to charity. So dont pick "just because", think about it if you really need this or that gadget. Then I register Groff - he really needs badges - just to have Dru coming back to me some minutes later one to hand me the badge for Henning. That's just "amazing"; I dont know IF i want to break this vicious circle the other day, since it's so funny. Talked to Theo about the ongoing IPsec problems and he taught me about utrace(2) which looks "complicated" but might be an end of the story the other day. Also had a nice talk to Peter (H.) about some other ideas along books. BTW, did I pay for ongoing beers? I think Tom did - what a guy :). Arriving at the Residence, I had to find my bathroom door locked (special thing).. crazy thing is they dont have a master key at the venue, but to have to call in one from elsewhere. Short night shortened by another 30minutes :(. Day 4 Weather is improving into beach+sun levels - and it's Conference Day! The opening keynote from Geist was very interesting ("citation needed"). Afterwards I went to zfs-over-ssh, nothing really new (sorry Allan). But then Jason had a super interesting talk on how about to apply BSD for the health-care system in Australia. I hope I can help him with the last bits (rdomain!) in the end. While lunch I tried to recall my memories about utrace(2) while talking to Theo. Then it was about to present my talk and I think it was well perceipted. One "not so good" feedback was about not taking the audience more into account. I think I was asking every other five slides or so - but, well. The general feedback (in spoken terms) was quite good. I was a bit "confused" and I did likely a better job in Tokyo, but well. Happened we ended up in the Oak again.. thanks to mwl, shirkdog, sng, pitrh, kurtm for having me there :) Day 5 While the weather had to decide "what next", I rushed to the venue just to gather Reyk's talk about vmd(8). Afterwards it was MSTP from Paeps which was very interesting and we (OpenBSD) should look into it. Then happened BUG BOF and I invite all "coastal Germans" to cbug.de :) I had to run off for other reasons and came back to Dave's talk which was AWESOME. Following was Rod's talk.. well. While I see his case, that was very poor. The auction into closing was awesome again, and I spend $50 on a Tshirt. :) + Epilogue I totally got the exit dates wrong. So first cancel a booking of an Hotel and then rebook the train to YUL. So I have plenty of time "in the morning" to get breakfast with the local guy. After that he drives me to VIARail station and I dig into "business" cussions. Well, see you in Ottawa - or how about Paris, Taipei? Bind Broker (http://www.tedunangst.com/flak/post/bind-broker) Ted Unangst writes about an interesting idea he has He has a single big server, and lots of users who would like to share it, many want to run web servers. This would be great, but alas, archaic decisions made long ago mean that network sockets aren't really files and there's this weird concept of privileged ports. Maybe we could assign each user a virtual machine and let them do whatever they want, but that seems wasteful. Think of the megabytes! Maybe we could setup nginx.conf to proxy all incoming connections to a process of the user's choosing, but that only works for web sites and we want to be protocol neutral. Maybe we could use iptables, but nobody wants to do that. What we need is a bind broker. At some level, there needs to be some kind of broker that assigns IPs to users and resolves conflicts. It should be possible to build something of this nature given just the existing unix tools we have, instead of changing system design. Then we can deploy our broker to existing systems without upgrading or disrupting their ongoing operation. The bind broker watches a directory for the creation, by users, of unix domain sockets. Then it binds to the TCP port of the same name, and transfers traffic between them. A more complete problem specification is as follows. A top level directory, which contains subdirectories named after IP addresses. Each user is assigned a subdirectory, which they have write permission to. Inside each subdirectory, the user may create unix sockets named according to the port they wish to bind to. We might assign user alice the IP 10.0.0.5 and the user bob the IP 10.0.0.10. Then alice could run a webserver by binding to net/10.0.0.5/80 and bob could run a mail server by binding to net/10.0.0.10/25. This maps IP ownership (which doesn't really exist in unix) to the filesystem namespace (which does have working permissions). So this will be a bit different than jails. The idea is to use filesystem permissions to control which users can bind to which IP addresses and ports The broker is responsible for watching each directory. As new sockets are created, it should respond by binding to the appropriate port. When a socket is deleted, the network side socket should be closed as well. Whenever a connection is accepted on the network side, a matching connection is made on the unix side, and then traffic is copied across. A full set of example code is provided There's no completely portable way to watch a directory for changes. I'm using a kevent extension. Otherwise we might consider a timeout and polling with fstat, or another system specific interface (or an abstraction layer over such an interface). Otherwise, if one of our mappings is ready to read (accept), we have a new connection to handle. The first half is straightforward. We accept the connection and make a matching connect call to the unix side. Then I broke out the big cheat stick and just spliced the sockets together. In reality, we'd have to set up a read/copy/write loop for each end to copy traffic between them. That's not very interesting to read though. The full code, below, comes in at 232 lines according to wc. Minus includes, blank lines, and lines consisting of nothing but braces, it's 148 lines of stuff that actually gets executed by the computer. Add some error handling, and working read/write code, and 200 lines seems about right. A very interesting idea. I wonder about creating a virtual file system that would implement this and maybe do a bit more to fully flesh out this idea. What do you think? *** News Roundup Daemons and friendly Ninjas (https://euroquis.nl/bobulate/?p=1600) There's quite a lot of software that uses CMake as a (meta-)buildsystem. A quick count in the FreeBSD ports tree shows me 1110 ports (over a thousand) that use it. CMake generates buildsystem files which then direct the actual build — it doesn't do building itself. There are multiple buildsystem-backends available: in regular usage, CMake generates Makefiles (and does a reasonable job of producing Makefiles that work for GNU Make and for BSD Make). But it can generate Ninja, or Visual Studio, and other buildsystem files. It's quite flexible in this regard. Recently, the KDE-FreeBSD team has been working on Qt WebEngine, which is horrible. It contains a complete Chromium and who knows what else. Rebuilding it takes forever. But Tobias (KDE-FreeBSD) and Koos (GNOME-FreeBSD) noticed that building things with the Ninja backend was considerably faster for some packages (e.g. Qt WebEngine, and Evolution data-thingy). Tobias wanted to try to extend the build-time improvements to all of the CMake-based ports in FreeBSD, and over the past few days, this has been a success. Ports builds using CMake now default to using Ninja as buildsystem-backend. Here's a bitty table of build-times. These are one-off build times, so hardly scientifically accurate — but suggestive of a slight improvement in build time. Name Size GMake Ninja liblxt 50kB 0:32 0:31 llvm38 1655kB * 19:43 musescore 47590kB 4:00 3:54 webkit2-gtk3 14652kB 44:29 37:40 Or here's a much more thorough table of results from tcberner@, who did 5 builds of each with and without ninja. I've cut out the raw data, here are just the average-of-five results, showing usually a slight improvement in build time with Ninja. Name av make av ninj Delta D/Awo compiler-rt 00:08 00:07 -00:01 -14% openjpeg 00:06 00:07 +00:01 +17% marble 01:57 01:43 -00:14 -11% uhd 01:49 01:34 -00:15 -13% opencacscade 04:08 03:23 -00:45 -18% avidemux 03:01 02:49 -00:12 – 6% kdevelop 01:43 01:33 -00:10 – 9% ring-libclient 00:58 00:53 -00:05 – 8% Not everything builds properly with Ninja. This is usually due to missing dependencies that CMake does not discover; this shows up when foo depends on bar but no rule is generated for it. Depending on build order and speed, bar may be there already by the time foo gets around to being built. Doxygen showed this, where builds on 1 CPU core were all fine, but 8 cores would blow up occasionally. In many cases, we've gone and fixed the missing implicit dependencies in ports and upstreams. But some things are intractable, or just really need GNU Make. For this, the FreeBSD ports infrastructure now has a knob attached to CMake for switching a port build to GNU Make. Normal: USES=cmake Out-of-source: USES=cmake:outsource GNU Make: USES=cmake:noninja gmake OoS, GMake: USES=cmake:outsource,noninja gmake Bad: USES=cmake gmake For the majority of users, this has no effect, but for our package-building clusters, and for KDE-FreeBSD developers who build a lot of CMake-buildsystem software in a day it may add up to an extra coffee break. So I'll raise a shot of espresso to friendship between daemons and ninjas. Announcing the pkgsrc-2017Q2 release (http://mail-index.netbsd.org/pkgsrc-users/2017/07/10/msg025237.html) For the 2017Q2 release we welcome the following notable package additions and changes to the pkgsrc collection: Firefox 54 GCC 7.1 MATE 1.18 Ruby 2.4 Ruby on Rails 4.2 TeX Live 2017 Thunderbird 52.1 Xen 4.8 We say goodbye to: Ruby 1.8 Ruby 2.1 The following infrastructure changes were introduced: Implement optional new pkgtasks and init infrastructure for pkginstall. Various enhancements and fixes for building with ccache. Add support to USE_LANGUAGES for newer C++ standards. Enhanced support for SSP, FORTIFY, and RELRO. The GitHub mirror has migrated to https://github.com/NetBSD/pkgsrc In total, 210 packages were added, 43 packages were removed, and 1,780 package updates were processed since the pkgsrc-2017Q1 release. *** OpenBSD changes of note 624 (http://www.tedunangst.com/flak/post/openbsd-changes-of-note-624) There are a bunch, but here are a few that jump out: Start plugging some leaks. Compile kernels with umask 007. Install them minus read permissions. Pure preprocessor implementation of the roff .ec and .eo requests, though you are warned that very bad things will happen to anybody trying to use these macros in OpenBSD manuals. Random linking for arm64. And octeon. And alpha. And hppa. There's some variation by platform, because every architecture has the kernel loaded with different flavors of initial physical and virtual mappings. And landisk. And loongson. And sgi. And macppc. And a gap file for sparc64, but nobody yet dares split locore. And arm7. Errata for perl File::Path race condition. Some fixes for potential link attacks against cron. Add pledge violations to acct reporting. Take random linking to the next stage. More about KARL - kernel address randomized link. As noted, a few difficulties with hibernate and such, but the plan is coming together. Add a new function reorder_kernel() that relinks and installs the new kernel in the background on system startup. Add support for the bootblocks to detect hibernate and boot the previous kernel. Remove the poorly described “stuff” from ksh. Replace usage of TIOCSTI in csh using a more common IO loop. Kind of like the stuff in ksh, but part of the default command line editing and parsing code, csh would read too many characters, then send the ones it didn't like back into the terminal. Which is weird, right? Also, more importantly, eliminating the code that uses TIOCSTI to inject characters into ttys means that maybe TIOCSTI can be removed. Revamp some of the authentication logging in ssh. Add a verbose flag to rm so you can panic immediately upon seeing it delete the wrong file instead of waiting to discover your mistake after the fact. Update libexpat to version 2.2.1 which has some security fixes. Never trust an expat, that's my motto. Update inteldrm to code based on Linux 4.4.70. This brings us support for Skylake and Cherryview and better support for Broadwell and Valleyview. Also adds MST support. Fun times for people with newish laptops. *** OPNsense 17.1.9 released (https://opnsense.org/opnsense-17-1-9-released/) firewall: move gateway switching from system to firewall advanced settings firewall: keep category selection when changing tabs firewall: do not skip gateway switch parsing too early (contributed by Stephane Lesimple) interfaces: show VLAN description during edit firmware: opnsense-revert can now handle multiple packages at once firmware: opnsense-patch can now handle permission changes from patches dnsmasq: use canned –bogus-priv for noprivatereverse dnsmasq: separate log file, ACL and menu entries dynamic dns: fix update for IPv6 (contributed by Alexander Leisentritt) dynamic dns: remove usage of CURLAUTH_ANY (contributed by Alexander Leisentritt) intrusion detection: suppress “fast mode available” boot warning in PCAP mode openvpn: plugin framework adaption unbound: add local-zone type transparent for PTR zone (contributed by Davide Gerhard) unbound: separate log file, ACL and menu entries wizard: remove HTML from description strings mvc: group relation to something other than uuid if needed mvc: rework “item in” for our Volt templates lang: Czech to 100% translated (contributed by Pavel Borecki) plugins: zabbix-agent 1.1 (contributed by Frank Wall) plugins: haproxy 1.16 (contributed by Frank Wall) plugins: acme-client 1.8 (contributed by Frank Wall) plugins: tinc fix for switch mode (contributed by Johan Grip) plugins: monit 1.3 (contributed by Frank Brendel) src: support dhclient supersede statement for option 54 (contributed by Fabian Kurtz) src: add Intel Atom Cherryview SOC HSUART support src: add the ID for the Huawei ME909S LTE modem src: HardenedBSD Stack Clash mitigations[1] ports: sqlite 3.19.3[2] ports: openvpn 2.4.3[3] ports: sudo 1.8.20p2[4] ports: dnsmasq 2.77[5] ports: openldap 2.4.45[6] ports: php 7.0.20[7] ports: suricata 3.2.2[8] ports: squid 3.5.26[9] ports: carootnss 3.31 ports: bind 9.11.1-P2[10] ports: unbound 1.6.3[11] ports: curl 7.54.1[12] *** Beastie Bits Thinkpad x230 - trying to get TrackPoint / Touchpad working in X (http://lists.dragonflybsd.org/pipermail/users/2017-July/313519.html) FreeBSD deprecates all r-cmds (rcp, rlogin, etc.) (http://marc.info/?l=freebsd-commits-all&m=149918307723723&w=2) Bashfill - art for your terminal (https://max.io/bash.html) Go 1.9 release notes: NetBSD support is broken, please help (https://github.com/golang/go/commit/32002079083e533e11209824bd9e3a797169d1c4) Jest, A ReST api for creating and managing FreeBSD jails written in Go (https://github.com/altsrc-io/Jest) *** Feedback/Questions John - zfs send/receive (http://dpaste.com/3ANETHW#wrap) Callum - laptops (http://dpaste.com/11TV0BJ) & An update (http://dpaste.com/3A14BQ6#wrap) Lars - Snapshot of VM datadisk (http://dpaste.com/0MM37NA#wrap) Daryl - Jail managers (http://dpaste.com/0CDQ9EK#wrap) ***

This episode gives you the full dose of BSDCan 2017 recap as well as a blog post on conference speaking advice. Headlines Pre-conference activities: Goat BoF, FreeBSD Foundation Board Meeting, and FreeBSD Journal Editorial Board Meeting The FreeBSD Foundation has a new President as Justin Gibbs is busy this year with building a house, so George Neville-Neil took up the task to serve as President, with Justin Gibbs as Secretary. Take a look at the updated Board of Directors (https://www.freebsdfoundation.org/about/board-of-directors/). We also have a new staff member (https://www.freebsdfoundation.org/about/staff/): Scott Lamons joined the Foundation team as senior program manager. Scott's work for the Foundation will focus on managing and evangelizing programs for advanced technologies in FreeBSD including preparing project plans, coordinating resources, and facilitating interactions between commercial vendors, the Foundation, and the FreeBSD community. The Foundation also planned various future activities, visits of upcoming conferences, and finding new ways to support and engage the community. The Foundation now has interns in the form of co-op students from the University of Waterloo, Canada. This is described further in the May 2017 Development Projects Update (https://www.freebsdfoundation.org/blog/may-2017-development-projects-update/). Both students (Siva and Charlie) were also the conference, helping out at the Foundation table, demonstrating the tinderbox dashboard. Follow the detailed instructions (https://www.freebsdfoundation.org/news-and-events/blog/blog-post/building-a-physical-freebsd-build-status-dashboard/) to build one of your own. The Foundation put out a call for Project Proposal Solicitation for 2017 (https://www.freebsdfoundation.org/blog/freebsd-foundation-2017-project-proposal-solicitation/). If you think you have a good proposal for work relating to any of the major subsystems or infrastructure for FreeBSD, we'd be happy to review it. Don't miss the deadlines for travel grants to some of the upcoming conferences. You can find the necessary forms and deadlines at the Travel Grant page (https://www.freebsdfoundation.org/what-we-do/travel-grants/travel-grants/) on the Foundation website. Pictures from the Goat BoF can be found on Keltia.net (https://assets.keltia.net/photos/BSDCan-2017/Royal%20Oak/index.html) Overlapping with the GoatBoF, members of the FreeBSD Journal editorial board met in a conference room in the Novotel to plan the upcoming issues. Topics were found, authors identified, and new content was discussed to appeal to even more readers. Check out the FreeBSD Journal website (https://www.freebsdfoundation.org/journal/) and subscribe if you like to support the Foundation in that way. FreeBSD Devsummit Day 1 & 2 (https://wiki.freebsd.org/DevSummit/201706) The first day of the Devsummit began with introductory slides by Gordon Tetlow, who organized the devsummit very well. Benno Rice of the FreeBSD core team presented the work done on the new Code of Conduct, which will become effective soon. A round of Q&A followed, with positive feedback from the other devsummit attendees supporting the new CoC. After that, Allan Jude joined to talk about the new FreeBSD Community Proposal (FCP) (https://github.com/freebsd/fcp) process. Modelled after IETF RFCs, Joyent RFDs, and Python PEP, it is a new way for the project to reach consensus on the design or implementation of new features or processes. The FCP repo contains FCP#0 that describes the process, and a template for writing a proposal. Then, the entire core team (except John Baldwin, who could not make it this year) and core secretary held a core Q&A session, Answering questions, gathering feedback and suggestions. After the coffee break, we had a presentation about Intel's QAT integration in FreeBSD. When the lunch was over, people spread out into working groups about BearSSL, Transport (TCP/IP), and OpenZFS. OpenZFS working group (https://pbs.twimg.com/media/DBu_IMsWAAId2sN.jpg:large): Matt Ahrens lead the group, and spent most of the first session providing a status update about what features have been recently committed, are out for review, on the horizon, or in the design phase. Existing Features Compressed ARC Compressed Send/Recv Recently Upstreamed A recent commit improved RAID-Z write speeds by declaring writes to padding blocks to be optional, and to always write them if they can be aggregated with the next write. Mostly impacts large record sizes. ABD (ARC buffer scatter/gather) Upstreaming In Progress Native Encryption Channel Programs Device Removal (Mirrors and Stripes) Redacted Send/recv Native TRIM Support (FreeBSD has its own, but this is better and applies to all ZFS implementations) Faster (mostly sequential) scrub/resilver DRAID (A great deal of time was spent explaining how this works, with diagrams on the chalk board) vdev metadata classes (store metadata on SSDs with data is on HDDs, or similar setups. Could also be modified to do dedup to SSD) Multi-mount protection (“safe import”, for dual-headed storage shelves) zpool checkpoint (rollback an entire pool, including zfs rename and zfs destroy) Further Out Import improvements Import with missing top-level vdevs (some blocks unreadable, but might let you get some data) Improved allocator performance -- vdev spacemap log ZIL performance Persistent L2ARC ZSTD Compression Day 2 Day two started with the Have/Want/Need session for FreeBSD 12.0. A number of features that various people have or are in the process of building, were discussed with an eye towards upstreaming them. Features we want to have in time for 12.0 (early 2019) were also discussed. After the break was the Vendor summit, which continued the discussion of how FreeBSD and its vendors can work together to make a better operating system, and better products based on it After lunch, the group broke up into various working groups: Testing/CI, Containers, Hardening UFS, and GELI Improvements Allan lead the GELI Improvements session. The main thrust of the discussions was fixing an outstanding bug in GELI when using both key slots with passphrases. To solve this, and make GELI more extensible, the metadata format will be extended to allow it to store more than 512 bytes of data (currently 511 bytes are used). The new format will allow arbitrarily large metadata, defined at creation time by selecting the number of user key slots desired. The new extended metadata format will contain mostly the same fields, except the userkey will no longer be a byte array of IV-key, Data-key, HMAC, but a struct that will contain all data about that key This new format will store the number of pkcs5v2 iterations per key, instead of only having a single location to store this number for all keys (the source of the original bug) A new set of flags per key, to control some aspects of the key (does it require a keyfile, etc), as well as possibly the role of the key. An auxdata field related to the flags, this would allow a specific key with a specific flag set, to boot a different partition, rather than decrypt the main partition. A URI to external key material is also stored per key, allowing GELI to uniquely identify the correct data to load to be able to use a specific decryption key And the three original parts of the key are stored in separate fields now. The HMAC also has a type field, allowing for a different HMAC algorithm to be used in the future. The main metadata is also extended to include a field to store the number of user keys, and to provide an overall HMAC of the metadata, so that it can be verified using the master key (provide any of the user keys) Other topics discussed: Ken Merry presented sedutil, a tool for managing Self Encrypting Drives, as may be required by certain governments and other specific use cases. Creating a deniable version of GELI, where the metadata is also encrypted The work to implemented GELI in the UEFI loader was discussed, and a number of developers volunteered to review and test the code Following the end of the Dev Summit, the “Newcomers orientation and mentorship” session was run by Michael W. Lucas, which attempts to pair up first time attendees with oldtimers, to make sure they always know a few people they can ask if they have questions, or if they need help getting introduced to the right people. News Roundup Conference Day 1 (http://www.bsdcan.org/2017/schedule/day_2017-06-09.en.html) The conference opened with some short remarks from Dan Langille, and then the opening keynote by Dr Michael Geist, a law professor at the University of Ottawa where he holds the Canada Research Chair in Internet and E-commerce Law. The keynote focused on what some of the currently issues are, and how the technical community needs to get involved at all levels. In Canada especially, contacting your representatives is quite effective, and when it does not happen, they only hear the other side of the story, and often end up spouting talking points from lobbyists as if they were facts. The question period for the keynote ran well overtime because of the number of good questions the discussion raised, including how do we fight back against large telcos with teams of lawyers and piles of money. Then the four tracks of talks started up for the day The day wrapped up with the Work In Progress (WIP) session. Allan Jude presented work on ZSTD compression in ZFS Drew Gallatin presented about work at Netflix on larger mbufs, to avoid the need for chaining and to allow more data to be pushed at once. Results in an 8% CPU time reduction when pushing 90 gbps of TLS encrypted traffic Dan Langille presented about letsencrypt (the acme.sh tool specifically), and bacula Samy Al Bahra presented about Concurrency Kit *** Conference Day 2 (http://www.bsdcan.org/2017/schedule/day_2017-06-10.en.html) Because Dan is a merciful soul, BSDCan starts an hour later on the second day Another great round of talks and BoF sessions over lunch The hallway track was great as always, and I spent most of the afternoon just talking with people Then the final set of talks started, and I was torn between all four of them Then there was the auction, and the closing party *** BSDCan 2017 Auction Swag (https://blather.michaelwlucas.com/archives/2962) Groff Fundraiser Pins: During the conference, You could get a unique Groff pin, by donating more than the last person to either the FreeBSD or OpenBSD foundation Michael W. Lucas and his wife Liz donated some interesting home made and local items to the infamous Charity Auction I donated the last remaining copy of the “Canadian Edition” of “FreeBSD Mastery: Advanced ZedFS”, and a Pentium G4400 (Skylake) CPU (Supports ECC or non-ECC) Peter Hessler donated his pen (Have you read “Git Commit Murder” yet?) Theo De Raadt donated his autographed conference badge David Maxwell donated a large print of the group photo from last years FreeBSD Developers Summit, which was purchased by Allan There was also a FreeBSD Dev Summit T-Shirt (with the Slogan: What is Core doing about it?) autographed by all of the attending members of core, with a forged jhb@ signature. Lastly, someone wrote “I

We're at BSDCan, but we have an interview with Michael W. Lucas which you don't want to miss. This episode was brought to you by Headlines We are off to BSDCan but we have an interview and news roundup for you. Interview - Michael W. Lucas - mwlucas@michaelwlucas.com (mailto:mwlucas@michaelwlucas.com) / @mwlauthor (https://twitter.com/mwlauthor) Books, conferences & how these two combine *** News Roundup In The Name Of Sane Email: Setting Up OpenBSD's spamd(8) With Secondary MXes In Play (http://bsdly.blogspot.no/2012/05/in-name-of-sane-email-setting-up-spamd.html) “The Grumpy BSD Guy”, Peter Hansteen is at it again, they have produced an updated version of a full recipe for OpenBSD's spamd for your primary AND secondary mail servers Recipes in our field are all too often offered with little or no commentary to help the user understand the underlying principles of how a specific configuration works. To counter the trend and offer some free advice on a common configuration, here is my recipe for a sane mail setup. Mailing lists can be fun. Most of the time the discussions on lists like openbsd-misc are useful, entertaining or both. But when your battle with spam fighting technology ends up blocking your source of information and entertainment (like in the case of the recent thread titled "spamd greylisting: false positives" - starting with this message), frustration levels can run high, and in the process it emerged that some readers out there place way too much trust in a certain site offering barely commented recipes (named after a rare chemical compound Cl-Hg-Hg-Cl). 4 easy steps: Make sure your MXes (both primary and secondary) are able to receive mail for your domains Set set up content filtering for all MXes, since some spambots actually speak SMTP Set up spamd in front of all MXes Set up synchronization between your spamds These are the basic steps. If you want to go even further, you can supplement your greylisting and publicly available blacklists with your own greytrapping, but greytrapping is by no means required. Once you have made sure that your mail exchangers will accept mail for your domains (checking that secondaries do receive and spool mail when you stop the SMTP service on the primary), it's time to start setting up the content filtering. The post provides links if you need help getting the basic mail server functionality going At this point you will more likely than not discover that any differences in filtering setups between the hosts that accept and deliver mail will let spam through via the weakest link. Tune accordingly, or at least until you are satisfied that you have a fairly functional configuration. As you will have read by now in the various sources I cited earlier, you need to set up rules to redirect traffic to your spamd as appropriate. Now let's take a peek at what I have running at my primary site's gateway. The articles provides a few different sets of rules The setup includes running all outgoing mail through spamd to auto-populate the whitelists, allowing replies to your emails to get through without greylisting At this point, you have seen how to set up two spamds, each running in front of a mail exchanger. You can choose to run with the default spamd.conf, or you can edit in your own customizations. There is also a link to Peter's spamd.conf if you want to use “what works for me” The fourth and final required step for a spamd setup with backup mail exchangers it to set up synchronization between the spamds. The synchronization keeps your greylists in sync and transfers information on any greytrapped entries to the partner spamds. As the spamd man page explains, the synchronization options -y and -Y are command line options to spamd. The articles steps through the process of configuring spamd to listen for synchronization, and to send synchronization messages to its peer With these settings in place, you have more or less completed step four of our recipe. The article also shows you how to configure spamd to log to a separate log file, to make the messages easier to find and consolidate between your mail servers After noting the system load on your content filtering machines, restart your spamds. Then watch the system load values on the content filterers and take a note of them from time to time, say every 30 minutes or so Step 4) is the last required step for building a multi-MX configuration. You may want to just leave the system running for a while and watch any messages that turn up in the spamd logs or the mail exchanger's logs The final embellishment is to set up local greytrapping. The principle is simple: If you have one or more addresses in your domain that you know will never be valid, you add them to your list of trapping addresses any host that tries to deliver mail to noreply@mydomain.nx will be added to the local blacklist spamd-greytrap to be stuttered at for as long as it takes. Greytrapping can be fun, you can search for posts here tagged with the obvious keywords. To get you started, I offer up my published list of trap addresses, built mainly from logs of unsuccessful delivery attempts here, at The BSDly.net traplist page, while the raw list of trap email addresses is available here. If you want to use that list in a similar manner for your site, please do, only remember to replace the domain names with one or more that you will be receiving mail for. Let us know how this affects your inbox *** Beastie Bits Status of FreeBSD's capsicum on Linux (http://www.capsicum-linux.org/) How to build a gateway, from 1979 (http://www.networksorcery.com/enp/ien/ien109.txt) Linux escapee Hamza Sheikh on “Why FreeBSD?” (https://bsdmag.org/why_freebsd/) UNIX is still as relevant as ever (https://blog.opengroup.org/2012/05/17/unix-is-still-as-relevant-as-ever/) Upcoming Summer 2017 FreeBSD Foundation Events (https://www.freebsdfoundation.org/blog/upcoming-summer-2017-freebsd-foundation-events/) ***

A pledge of love to OpenBSD, combating ransomware like WannaCry with OpenZFS, and using PFsense to maximize your non-gigabit Internet connection This episode was brought to you by Headlines ino64 project committed to FreeBSD 12-CURRENT (https://svnweb.freebsd.org/base?view=revision&revision=318736) The ino64 project has been completed and merged into FreeBSD 12-CURRENT Extend the inot, devt, nlinkt types to 64-bit ints. Modify struct dirent layout to add doff, increase the size of dfileno to 64-bits, increase the size of dnamlen to 16-bits, and change the required alignment. Increase struct statfs fmntfromname[] and fmntonname[] array length MNAMELEN to 1024 This means the length of a mount point (MNAMELEN) has been increased from 88 byte to 1024 bytes. This allows longer ZFS dataset names and more nesting, and generally improves the usefulness of nested jails It also allow more than 4 billion files to be stored in a single file system (both UFS and ZFS). It also deals with a number of NFS problems, such as Amazon's EFS (cloud NFS), which uses 64 bit IDs even with small numbers of files. ABI breakage is mitigated by providing compatibility using versioned symbols, ingenious use of the existing padding in structures, and by employing other tricks. Unfortunately, not everything can be fixed, especially outside the base system. For instance, third-party APIs which pass struct stat around are broken in backward and forward incompatible ways. A bug in poudriere that may cause some packages to not rebuild is being fixed. Many packages like perl will need to be rebuilt after this change Update note: strictly follow the instructions in UPDATING. Build and install the new kernel with COMPAT_FREEBSD11 option enabled, then reboot, and only then install new world. So you need the new GENERIC kernel with the COMPAT_FREEBSD11 option, so that your old userland will work with the new kernel, and you need to build, install, and reboot onto the new kernel before attempting to install world. The usual process of installing both and then rebooting will NOT WORK Credits: The 64-bit inode project, also known as ino64, started life many years ago as a project by Gleb Kurtsou (gleb). Kirk McKusick (mckusick) then picked up and updated the patch, and acted as a flag-waver. Feedback, suggestions, and discussions were carried by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles), and Rick Macklem (rmacklem). Kris Moore (kmoore) performed an initial ports investigation followed by an exp-run by Antoine Brodin (antoine). Essential and all-embracing testing was done by Peter Holm (pho). The heavy lifting of coordinating all these efforts and bringing the project to completion were done by Konstantin Belousov (kib). Sponsored by: The FreeBSD Foundation (emaste, kib) Why I love OpenBSD (https://medium.com/@h3artbl33d/why-i-love-openbsd-ca760cf53941) Jeroen Janssen writes: I do love open source software. Oh boy, I really do love open source software. It's extendable, auditable, and customizable. What's not to love? I'm astonished by the idea that tens, hundreds, and sometimes even thousands of enthusiastic, passionate developers collaborate on an idea. Together, they make the world a better place, bit by bit. And this leads me to one of my favorite open source projects: the 22-year-old OpenBSD operating system. The origins of my love affair with OpenBSD From Linux to *BSD The advantages of OpenBSD It's extremely secure It's well documented It's open source > It's neat and clean My take on OpenBSD ** DO ** Combating WannaCry and Other Ransomware with OpenZFS Snapshots (https://www.ixsystems.com/blog/combating-ransomware/) Ransomware attacks that hold your data hostage using unauthorized data encryption are spreading rapidly and are particularly nefarious because they do not require any special access privileges to your data. A ransomware attack may be launched via a sophisticated software exploit as was the case with the recent “WannaCry” ransomware, but there is nothing stopping you from downloading and executing a malicious program that encrypts every file you have access to. If you fail to pay the ransom, the result will be indistinguishable from your simply deleting every file on your system. To make matters worse, ransomware authors are expanding their attacks to include just about any storage you have access to. The list is long, but includes network shares, Cloud services like DropBox, and even “shadow copies” of data that allow you to open previous versions of files. To make matters even worse, there is little that your operating system can do to prevent you or a program you run from encrypting files with ransomware just as it can't prevent you from deleting the files you own. Frequent backups are touted as one of the few effective strategies for recovering from ransomware attacks but it is critical that any backup be isolated from the attack to be immune from the same attack. Simply copying your files to a mounted disk on your computer or in the Cloud makes the backup vulnerable to infection by virtue of the fact that you are backing up using your regular permissions. If you can write to it, the ransomware can encrypt it. Like medical workers wearing hazmat suits for isolation when combating an epidemic, you need to isolate your backups from ransomware. OpenZFS snapshots to the rescue OpenZFS is the powerful file system at the heart of every storage system that iXsystems sells and of its many features, snapshots can provide fast and effective recovery from ransomware attacks at both the individual user and enterprise level as I talked about in 2015. As a copy-on-write file system, OpenZFS provides efficient and consistent snapshots of your data at any given point in time. Each snapshot only includes the precise delta of changes between any two points in time and can be cloned to provide writable copies of any previous state without losing the original copy. Snapshots also provide the basis of OpenZFS replication or backing up of your data to local and remote systems. Because an OpenZFS snapshot takes place at the block level of the file system, it is immune to any file-level encryption by ransomware that occurs over it. A carefully-planned snapshot, replication, retention, and restoration strategy can provide the low-level isolation you need to enable your storage infrastructure to quickly recover from ransomware attacks. OpenZFS snapshots in practice While OpenZFS is available on a number of desktop operating systems such as TrueOS and macOS, the most effective way to bring the benefits of OpenZFS snapshots to the largest number of users is with a network of iXsystems TrueNAS, FreeNAS Certified and FreeNAS Mini unified NAS and SAN storage systems. All of these can provide OpenZFS-backed SMB, NFS, AFP, and iSCSI file and block storage to the smallest workgroups up through the largest enterprises and TrueNAS offers available Fibre Channel for enterprise deployments. By sharing your data to your users using these file and block protocols, you can provide them with a storage infrastructure that can quickly recover from any ransomware attack thrown at it. To mitigate ransomware attacks against individual workstations, TrueNAS and FreeNAS can provide snapshotted storage to your VDI or virtualization solution of choice. Best of all, every iXsystems TrueNAS, FreeNAS Certified, and FreeNAS Mini system includes a consistent user interface and the ability to replicate between one another. This means that any topology of individual offices and campuses can exchange backup data to quickly mitigate ransomware attacks on your organization at all levels. Join us for a free webinar (http://www.onlinemeetingnow.com/register/?id=uegudsbc75) with iXsystems Co-Founder Matt Olander and learn more about why businesses everywhere are replacing their proprietary storage platforms with TrueNAS then email us at info@ixsystems.com or call 1-855-GREP-4-IX (1-855-473-7449), or 1-408-493-4100 (outside the US) to discuss your storage needs with one of our solutions architects. Interview - Michael W. Lucas - mwlucas@michaelwlucas.com (mailto:mwlucas@michaelwlucas.com) / @twitter (https://twitter.com/mwlauthor) Books, conferences, and how these two combine + BR: Welcome back. Tell us what you've been up to since the last time we interviewed you regarding books and such. + AJ: Tell us a little bit about relayd and what it can do. + BR: What other books do you have in the pipeline? + AJ: What are your criteria that qualifies a topic for a mastery book? + BR: Can you tell us a little bit about these writing workshops that you attend and what happens there? + AJ: Without spoiling too much: How did you come up with the idea for git commit murder? + BR: Speaking of BSDCan, can you tell the first timers about what to expect in the http://www.bsdcan.org/2017/schedule/events/890.en.html (Newcomers orientation and mentorship) session on Thursday? + AJ: Tell us about the new WIP session at BSDCan. Who had the idea and how much input did you get thus far? + BR: Have you ever thought about branching off into a new genre like children's books or medieval fantasy novels? + AJ: Is there anything else before we let you go? News Roundup Using LLDP on FreeBSD (https://tetragir.com/freebsd/networking/using-lldp-on-freebsd.html) LLDP, or Link Layer Discovery Protocol allows system administrators to easily map the network, eliminating the need to physically run the cables in a rack. LLDP is a protocol used to send and receive information about a neighboring device connected directly to a networking interface. It is similar to Cisco's CDP, Foundry's FDP, Nortel's SONMP, etc. It is a stateless protocol, meaning that an LLDP-enabled device sends advertisements even if the other side cannot do anything with it. In this guide the installation and configuration of the LLDP daemon on FreeBSD as well as on a Cisco switch will be introduced. If you are already familiar with Cisco's CDP, LLDP won't surprise you. It is built for the same purpose: to exchange device information between peers on a network. While CDP is a proprietary solution and can be used only on Cisco devices, LLDP is a standard: IEEE 802.3AB. Therefore it is implemented on many types of devices, such as switches, routers, various desktop operating systems, etc. LLDP helps a great deal in mapping the network topology, without spending hours in cabling cabinets to figure out which device is connected with which switchport. If LLDP is running on both the networking device and the server, it can show which port is connected where. Besides physical interfaces, LLDP can be used to exchange a lot more information, such as IP Address, hostname, etc. In order to use LLDP on FreeBSD, net-mgmt/lldpd has to be installed. It can be installed from ports using portmaster: #portmaster net-mgmt/lldpd Or from packages: #pkg install net-mgmt/lldpd By default lldpd sends and receives all the information it can gather , so it is advisable to limit what we will communicate with the neighboring device. The configuration file for lldpd is basically a list of commands as it is passed to lldpcli. Create a file named lldpd.conf under /usr/local/etc/ The following configuration gives an example of how lldpd can be configured. For a full list of options, see %man lldpcli To check what is configured locally, run #lldpcli show chassis detail To see the neighbors run #lldpcli show neighbors details Check out the rest of the article about enabling LLDP on a Cisco switch experiments with prepledge (http://www.tedunangst.com/flak/post/experiments-with-prepledge) Ted Unangst takes a crack at a system similar to the one being designed for Capsicum, Oblivious Sandboxing (See the presentation at BSDCan), where the application doesn't even know it is in the sandbox MP3 is officially dead, so I figure I should listen to my collection one last time before it vanishes entirely. The provenance of some of these files is a little suspect however, and since I know one shouldn't open files from strangers, I'd like to take some precautions against malicious malarkey. This would be a good use for pledge, perhaps, if we can get it working. At the same time, an occasional feature request for pledge is the ability to specify restrictions before running a program. Given some untrusted program, wrap its execution in a pledge like environment. There are other system call sandbox mechanisms that can do this (systrace was one), but pledge is quite deliberately designed not to support this. But maybe we can bend it to our will. Our pledge wrapper can't be an external program. This leaves us with the option of injecting the wrapper into the target program via LD_PRELOAD. Before main even runs, we'll initialize what needs initializing, then lock things down with a tight pledge set. Our eventual target will be ffplay, but hopefully the design will permit some flexibility and reuse. So the new code is injected to override the open syscall, and reads a list of files from an environment variable. Those files are opened and the path and file descriptor are put into a linked list, and then pledge is used to restrict further access to the file system. The replacement open call now searches just that linked list, returning the already opened file descriptors. So as long as your application only tries to open files that you have preopened, it can function without modification within the sandbox. Or at least that is the goal... ffplay tries to dlopen() some things, and because of the way dlopen() works, it doesn't go via the libc open() wrapper, so it doesn't get overridden ffplay also tries to call a few ioctl's, not allowed After stubbing both of those out, it still doesn't work and it is just getting worse Ted switches to a new strategy, using ffmpeg to convert the .mp3 to a .wav file and then just cat it to /dev/audio A few more stubs for ffmpeg, including access(), and adding tty access to the list of pledges, and it finally works This point has been made from the early days, but I think this exercise reinforces it, that pledge works best with programs where you understand what the program is doing. A generic pledge wrapper isn't of much use because the program is going to do something unexpected and you're going to have a hard time wrangling it into submission. Software is too complex. What in the world is ffplay doing? Even if I were working with the source, how long would it take to rearrange the program into something that could be pledged? One can try using another program, but I would wager that as far as multiformat media players go, ffplay is actually on the lower end of the complexity spectrum. Most of the trouble comes from using SDL as an abstraction layer, which performs a bunch of console operations. On the flip side, all of this early init code is probably the right design. Once SDL finally gets its screen handle setup, we could apply pledge and sandbox the actual media decoder. That would be the right way to things. Is pledge too limiting? Perhaps, but that's what I want. I could have just kept adding permissions until ffplay had full access to my X socket, but what kind of sandbox is that? I don't want naughty MP3s scraping my screen and spying on my keystrokes. The sandbox I created had all the capabilities one needs to convert an MP3 to audible sound, but the tool I wanted to use wasn't designed to work in that environment. And in its defense, these were new post hoc requirements. Other programs, even sed, suffer from less than ideal pledge sets as well. The best summary might be to say that pledge is designed for tomorrow's programs, not yesterday's (and vice versa). There were a few things I could have done better. In particular, I gave up getting audio to work, even though there's a nice description of how to work with pledge in the sio_open manual. Alas, even going back and with a bit more effort I still haven't succeeded. The requirements to use libsndio are more permissive than I might prefer. How I Maximized the Speed of My Non-Gigabit Internet Connection (https://medium.com/speedtest-by-ookla/engineer-maximizes-internet-speed-story-c3ec0e86f37a) We have a new post from Brennen Smith, who is the Lead Systems Engineer at Ookla, the company that runs Speedtest.net, explaining how he used pfSense to maximize his internet connection I spend my time wrangling servers and internet infrastructure. My daily goals range from designing high performance applications supporting millions of users and testing the fastest internet connections in the world, to squeezing microseconds from our stack —so at home, I strive to make sure that my personal internet performance is running as fast as possible. I live in an area with a DOCSIS ISP that does not provide symmetrical gigabit internet — my download and upload speeds are not equal. Instead, I have an asymmetrical plan with 200 Mbps download and 10 Mbps upload — this nuance considerably impacted my network design because asymmetrical service can more easily lead to bufferbloat. We will cover bufferbloat in a later article, but in a nutshell, it's an issue that arises when an upstream network device's buffers are saturated during an upload. This causes immense network congestion, latency to rise above 2,000 ms., and overall poor quality of internet. The solution is to shape the outbound traffic to a speed just under the sending maximum of the upstream device, so that its buffers don't fill up. My ISP is notorious for having bufferbloat issues due to the low upload performance, and it's an issue prevalent even on their provided routers. They walk through a list of router devices you might consider, and what speeds they are capable of handling, but ultimately ended up using a generic low power x86 machine running pfSense 2.3 In my research and testing, I also evaluated IPCop, VyOS, OPNSense, Sophos UTM, RouterOS, OpenWRT x86, and Alpine Linux to serve as the base operating system, but none were as well supported and full featured as PFSense. The main setting to look at is the traffic shaping of uploads, to keep the pipe from getting saturated and having a large buffer build up in the modem and further upstream. This build up is what increases the latency of the connection As with any experiment, any conclusions need to be backed with data. To validate the network was performing smoothly under heavy load, I performed the following experiment: + Ran a ping6 against speedtest.net to measure latency. + Turned off QoS to simulate a “normal router”. + Started multiple simultaneous outbound TCP and UDP streams to saturate my outbound link. + Turned on QoS to the above settings and repeated steps 2 and 3. As you can see from the plot below, without QoS, my connection latency increased by ~1,235%. However with QoS enabled, the connection stayed stable during the upload and I wasn't able to determine a statistically significant delta. That's how I maximized the speed on my non-gigabit internet connection. What have you done with your network? FreeBSD on 11″ MacBook Air (https://www.geeklan.co.uk/?p=2214) Sevan Janiyan writes in his tech blog about his experiences running FreeBSD on an 11'' MacBook Air This tiny machine has been with me for a few years now, It has mostly run OS X though I have tried OpenBSD on it (https://www.geeklan.co.uk/?p=1283). Besides the screen resolution I'm still really happy with it, hardware wise. Software wise, not so much. I use an external disk containing a zpool with my data on it. Among this data are several source trees. CVS on a ZFS filesystem on OS X is painfully slow. I dislike that builds running inside Terminal.app are slow at the expense of a responsive UI. The system seems fragile, at the slightest push the machine will either hang or become unresponsive. Buggy serial drivers which do not implement the break signal and cause instability are frustrating. Last week whilst working on Rump kernel (http://rumpkernel.org/) builds I introduced some new build issues in the process of fixing others, I needed to pick up new changes from CVS by updating my copy of the source tree and run builds to test if issues were still present. I was let down on both counts, it took ages to update source and in the process of cross compiling a NetBSD/evbmips64-el release, the system locked hard. That was it, time to look what was possible elsewhere. While I have been using OS X for many years, I'm not tied to anything exclusive on it, maybe tweetbot, perhaps, but that's it. On the BSDnow podcast they've been covering changes coming in to TrueOS (formerly PC-BSD – a desktop focused distro based on FreeBSD), their experiments seemed interesting, the project now tracks FreeBSD-CURRENT, they've replaced rcng with OpenRC as the init system and it comes with a pre-configured desktop environment, using their own window manager (Lumina). Booting the USB flash image it made it to X11 without any issue. The dock has a widget which states the detected features, no wifi (Broadcom), sound card detected and screen resolution set to 1366×768. I planned to give it a try on the weekend. Friday, I made backups and wiped the system. TrueOS installed without issue, after a short while I had a working desktop, resuming from sleep worked out of the box. I didn't spend long testing TrueOS, switching out NetBSD-HEAD only to realise that I really need ZFS so while I was testing things out, might as well give stock FreeBSD 11-STABLE a try (TrueOS was based on -CURRENT). Turns out sleep doesn't work yet but sound does work out of the box and with a few invocations of pkg(8) I had xorg, dwm, firefox, CVS and virtuabox-ose installed from binary packages. VirtualBox seems to cause the system to panic (bug 219276) but I should be able to survive without my virtual machines over the next few days as I settle in. I'm considering ditching VirtualBox and converting the vdi files to raw images so that they can be written to a new zvol for use with bhyve. As my default keyboard layout is Dvorak, OS X set the EFI settings to this layout. The first time I installed FreeBSD 11-STABLE, I opted for full disk encryption but ran into this odd issue where on boot the keyboard layout was Dvorak and password was accepted, the system would boot and as it went to mount the various filesystems it would switch back to QWERTY. I tried entering my password with both layout but wasn't able to progress any further, no bug report yet as I haven't ruled myself out as the problem. Thunderbolt gigabit adapter –bge(4) (https://www.freebsd.org/cgi/man.cgi?query=bge) and DVI adapter both worked on FreeBSD though the gigabit adapter needs to be plugged in at boot to be detected. The trackpad bind to wsp(4) (https://www.freebsd.org/cgi/man.cgi?query=wsp), left, right and middle clicks are available through single, double and tripple finger tap. Sound card binds to snd_hda(4) (https://www.freebsd.org/cgi/man.cgi?query=snd_hda) and works out of the box. For wifi I'm using a urtw(4) (https://www.freebsd.org/cgi/man.cgi?query=urtw) Alfa adapter which is a bit on the large side but works very reliably. A copy of the dmesg (https://www.geeklan.co.uk/files/macbookair/freebsd-dmesg.txt) is here. Beastie Bits OPNsense - call-for-testing for SafeStack (https://forum.opnsense.org/index.php?topic=5200.0) BSD 4.4: cat (https://www.rewritinghistorycasts.com/screencasts/bsd-4.4:-cat) Continuous Unix commit history from 1970 until today (https://github.com/dspinellis/unix-history-repo) Update on Unix Architecture Evolution Diagrams (https://www.spinellis.gr/blog/20170510/) “Relayd and Httpd Mastery” is out! (https://blather.michaelwlucas.com/archives/2951) Triangle BSD User Group Meeting -- libxo (https://www.meetup.com/Triangle-BSD-Users-Group/events/240247251/) *** Feedback/Questions Carlos - ASUS Tinkerboard (http://dpaste.com/1GJHPNY#wrap) James - Firewall question (http://dpaste.com/0QCW933#wrap) Adam - ZFS books (http://dpaste.com/0GMG5M2#wrap) David - Managing zvols (http://dpaste.com/2GP8H1E#wrap) ***

This week on the show, we've got some great stories to bring you, a look at the odder side of UNIX history This episode was brought to you by Headlines syspatch in testing state (http://marc.info/?l=openbsd-tech&m=148058309126053&w=2) Antoine Jacoutot ajacoutot@ openbsd has posted a call for testing for OpenBSD's new syspatch tool “syspatch(8), a "binary" patch system for -release is now ready for early testing. This does not use binary diffing to update the system, but regular signed tarballs containing the updated files (ala installer).” “I would appreciate feedback on the tool. But please send it directly to me, there's no need to pollute the list. This is obviously WIP and the tool may or may not change in drastic ways.” “These test binary patches are not endorsed by the OpenBSD project and should not be trusted, I am only providing them to get early feedback on the tool. If all goes as planned, I am hoping that syspatch will make it into the 6.1 release; but for it to happen, I need to know how it breaks your systems :-)” Instructions (http://syspatch.openbsd.org/pub/OpenBSD/6.0/syspatch/amd64/README.txt) If you test it, report back and let us know how it went *** Weston working (https://lists.freebsd.org/pipermail/freebsd-current/2016-December/064198.html) Over the past few years we've had some user-interest in the state of Wayland / Weston on FreeBSD. In the past day or so, Johannes Lundberg has sent in a progress report to the FreeBSD mailing lists. Without further ADO: We had some progress with Wayland that we'd like to share. Wayland (v1.12.0) Working Weston (v1.12.0) Working (Porting WIP) Weston-clients (installed with wayland/weston port) Working XWayland (run X11 apps in Wayland compositor) Works (maximized window only) if started manually but not when launching X11 app from Weston. Most likely problem with Weston IPC. Sway (i3-compatible Wayland compositor) Working SDL20 (Wayland backend) games/stonesoup-sdl briefly tested. https://twitter.com/johalun/status/811334203358867456 GDM (with Wayland) Halted - depends on logind. GTK3 gtk3-demo runs fine on Weston (might have to set GDK_BACKEND=wayland first. GTK3 apps working (gedit, gnumeric, xfce4-terminal tested, xfce desktop (4.12) does not yet support GTK3)“ Johannes goes on to give instructions on how / where you can fetch their WiP and do your own testing. At the moment you'll need Matt Macy's newer Intel video work, as well as their ports tree which includes all the necessary software bits. Before anybody asks, yes we are watching this for TrueOS! *** Where the rubber meets the road (part two) (https://functionallyparanoid.com/2016/12/15/where-the-rubber-meets-the-road-part-two/) Continuing with our story from Brian Everly from a week ago, we have an update today on the process to dual-boot OpenBSD with Arch Linux. As we last left off, Arch was up and running on the laptop, but some quirks in the hardware meant OpenBSD would take a bit longer. With those issues resolved and the HD seen again, the next issue that reared its head was OpenBSD not seeing the partition tables on the disk. After much frustration, it was time to nuke and pave, starting with OpenBSD first this time. After a successful GPT partitioning and install of OpenBSD, he went back to installing Arch, and then the story got more interesting. “I installed Arch as I detailed in my last post; however, when I fired up gdisk I got a weird error message: “Warning! Disk size is smaller than the main header indicates! Loading secondary header from the last sector of the disk! You should use ‘v' to verify disk integrity, and perhaps options on the expert's menu to repair the disk.” Immediately after this, I saw a second warning: “Caution: Invalid backup GPT header, but valid main header; regenerating backup header from main header.” And, not to be outdone, there was a third: “Warning! Main and backup partition tables differ! Use the ‘c' and ‘e' options on the recovery & transformation menu to examine the two tables.” Finally (not kidding), there was a fourth: “Warning! One or more CRCs don't match. You should repair the disk!” Given all of that, I thought to myself, “This is probably why I couldn't see the disk properly when I partitioned it under Linux on the OpenBSD side. I'll let it repair things and I should be good to go.” I then followed the recommendation and repaired things, using the primary GPT table to recreate the backup one. I then installed Arch and figured I was good to go.“ After confirming through several additional re-installs that the behavior was reproducible, he then decided to go full on crazy,and partition with MBR. That in and of itself was a challenge, since as he mentions, not many people dual-boot OpenBSD with Linux on MBR, especially using luks and lvm! If you want to see the details on how that was done, check it out. The story ends in success though! And better yet: “Now that I have everything working, I'll restore my config and data to Arch, configure OpenBSD the way I like it and get moving. I'll take some time and drop a note on the tech@ mailing list for OpenBSD to see if they can figure out what the GPT problem was I was running into. Hopefully it will make that part of the code stronger to get an edge-case bug report like this.” Take note here, if you run into issues like this with any OS, be sure to document in detail what happened so developers can explore solutions to the issue. *** FreeBSD and ZFS as a time capsule for OS X (https://blog.feld.me/posts/2016/12/using-freebsd-as-a-time-capsule-for-osx/) Do you have any Apple users in your life? Perhaps you run FreeBSD for ZFS somewhere else in the house or office. Well today we have a blog post from Mark Felder which shows how you can use FreeBSD as a time-capsule for your OSX systems. The setup is quite simple, to get started you'll need packages for netatalk3 and avahi-app for service discovery. Next up will be your AFP configuration. He helpfully provides a nice example that you should be able to just cut-n-paste. Be sure to check the hosts allow lines and adjust to fit your network. Also of note will be the backup location and valid users to adjust. A little easier should be the avahi setup, which can be a straight copy-n-paste from the site, which will perform the service advertisements. The final piece is just enabling specific services in /etc/rc.conf and either starting them by hand, or rebooting. At this point your OSX systems should be able to discover the new time-capsule provider on the network and DTRT. *** News Roundup netbenches - FreeBSD network forwarding performance benchmark results (https://github.com/ocochard/netbenches) Olivier Cochard-Labbé, original creator of FreeNAS, and leader of the BSD Router Project, has a github repo of network benchmarks There are many interesting results, and all of the scripts, documentation, and configuration files to run the tests yourself IPSec Performance on an Atom C2558, 12-head vs IPSec Performance Branch (https://github.com/ocochard/netbenches/tree/master/Atom_C2558_4Cores-Intel_i350/ipsec/results/fbsd12.projects-ipsec.equilibrium) Compared to: Xeon L5630 2.13GHz (https://github.com/ocochard/netbenches/tree/2f3bb1b3c51e454736f1fcc650c3328071834f8d/Xeon_L5630-4Cores-Intel_82599EB/ipsec/results/fbsd11.0) and IPSec with Authentication (https://github.com/ocochard/netbenches/tree/305235114ba8a3748ad9681c629333f87f82613a/Atom_C2558_4Cores-Intel_i350/ipsec.ah/results/fbsd12.projects-ipsec.equilibrium) I look forward to seeing tests on even more hardware, as people with access to different configurations try out these benchmarks *** A tcpdump Tutorial and Primer with Examples (https://danielmiessler.com/study/tcpdump/) Most users will be familiar with the basics of using tcpdump, but this tutorial/primer is likely to fill in a lot of blanks, and advance many users understanding of tcpdump “tcpdump is the premier network analysis tool for information security professionals. Having a solid grasp of this über-powerful application is mandatory for anyone desiring a thorough understanding of TCP/IP. Many prefer to use higher level analysis tools such as Wireshark, but I believe this to usually be a mistake.” tcpdump is an important tool for any system or network administrator, it is not just for security. It is often the best way to figure out why the network is not behaving as expected. “In a discipline so dependent on a true understanding of concepts vs. rote learning, it's important to stay fluent in the underlying mechanics of the TCP/IP suite. A thorough grasp of these protocols allows one to troubleshoot at a level far beyond the average analyst, but mastery of the protocols is only possible through continued exposure to them.” Not just that, but TCP/IP is a very interesting protocol, considering how little it has changed in its 40+ year history “First off, I like to add a few options to the tcpdump command itself, depending on what I'm looking at. The first of these is -n, which requests that names are not resolved, resulting in the IPs themselves always being displayed. The second is -X, which displays both hex and ascii content within the packet.” “It's also important to note that tcpdump only takes the first 96 bytes of data from a packet by default. If you would like to look at more, add the -s number option to the mix, where number is the number of bytes you want to capture. I recommend using 0 (zero) for a snaplength, which gets everything.” The page has a nice table of the most useful options It also has a great primer on doing basic filtering If you are relatively new to using tcpdump, I highly recommend you spend a few minutes reading through this article *** How Unix made it to the top (http://minnie.tuhs.org/pipermail/tuhs/2016-December/007519.html) Doug McIlroy gives us a nice background post on how “Unix made it to the top” It's fairly short / concise, so I felt it would be good to read in its entirety. “It has often been told how the Bell Labs law department became the first non-research department to use Unix, displacing a newly acquired stand-alone word-processing system that fell short of the department's hopes because it couldn't number the lines on patent applications, as USPTO required. When Joe Ossanna heard of this, he told them about roff and promised to give it line-numbering capability the next day. They tried it and were hooked. Patent secretaries became remote members of the fellowship of the Unix lab. In due time the law department got its own machine. Less well known is how Unix made it into the head office of AT&T. It seems that the CEO, Charlie Brown, did not like to be seen wearing glasses when he read speeches. Somehow his PR assistant learned of the CAT phototypesetter in the Unix lab and asked whether it might be possible to use it to produce scripts in large type. Of course it was. As connections to the top never hurt, the CEO's office was welcomed as another ouside user. The cost--occasionally having to develop film for the final copy of a speech--was not onerous. Having teethed on speeches, the head office realized that Unix could also be useful for things that didn't need phototypesetting. Other documents began to accumulate in their directory. By the time we became aware of it, the hoard came to include minutes of AT&T board meetings. It didn't seem like a very good idea for us to be keeping records from the inner sanctum of the corporation on a computer where most everybody had super-user privileges. A call to the PR guy convinced him of the wisdom of keeping such things on their own premises. And so the CEO's office bought a Unix system. Just as one hears of cars chosen for their cupholders, so were theseusers converted to Unix for trivial reasons: line numbers and vanity.“ Odd Comments and Strange Doings in Unix (http://orkinos.cmpe.boun.edu.tr/~kosar/odd.html) Everybody loves easter-eggs, and today we have some fun odd ones from the history throughout UNIX told by Dennis Ritchie. First up, was a fun one where the “mv” command could sometimes print the following “values of b may give rise to dom!” “Like most of the messages recorded in these compilations, this one was produced in some situation that we considered unlikely or as result of abuse; the details don't matter. I'm recording why the phrase was selected. The very first use of Unix in the "real business" of Bell Labs was to type and produce patent applications, and for a while in the early 1970s we had three typists busily typing away in the grotty lab on the sixth floor. One day someone came in and observed on the paper sticking out of one of the Teletypes, displayed in magnificent isolation, this ominous phrase: values of b may give rise to dom! It was of course obvious that the typist had interrupted a printout (generating the "!" from the ed editor) and moved up the paper, and that the context must have been something like "varying values of beta may give rise to domain wall movement" or some other fragment of a physically plausible patent application.But the phrase itself was just so striking! Utterly meaningless, but it looks like what... a warning? What is "dom?" At the same time, we were experimenting with text-to-voice software by Doug McIlroy and others, and of course the phrase was tried out with it. For whatever reason, its rendition of "give rise to dom!" accented the last word in a way that emphasized the phonetic similarity between "doom" and the first syllable of "dominance." It pronounced "beta" in the British style, "beeta." The entire occurrence became a small, shared treasure.The phrase had to be recorded somewhere, and it was, in the v6 source. Most likely it was Bob Morris who did the deed, but it could just as easily have been Ken. I hope that your browser reproduces the b as a Greek beta.“ Next up is one you might have heard before: /* You are not expected to understand this */> Every now and then on Usenet or elsewhere I run across a reference to a certain comment in the source code of the Sixth Edition Unix operating system. I've even been given two sweatshirts that quote it. Most probably just heard about it, but those who saw it in the flesh either had Sixth Edition Unix (ca. 1975) or read the annotated version of this system by John Lions (which was republished in 1996: ISBN 1-57298-013-7, Peer-to-Peer Communications).It's often quoted as a slur on the quantity or quality of the comments in the Bell Labs research releases of Unix. Not an unfair observation in general, I fear, but in this case unjustified. So we tried to explain what was going on. "You are not expected to understand this" was intended as a remark in the spirit of "This won't be on the exam," rather than as an impudent challenge. There's a few other interesting stories as well, if the odd/fun side of UNIX history at all interests you, I would recommend checking it out. Beastie Bits With patches in review the #FreeBSD base system builds 100% reproducibly (https://twitter.com/ed_maste/status/811289279611682816) BSDCan 2017 Call for Participation (https://www.freebsdfoundation.org/news-and-events/call-for-papers/bsdcan-2017/) ioCell 2.0 released (https://github.com/bartekrutkowski/iocell/releases) who even calls link_ntoa? (http://www.tedunangst.com/flak/post/who-even-calls-link-ntoa) Booting Androidx86 under bhyve (https://twitter.com/pr1ntf/status/809528845673996288) Feedback/Questions Chris - VNET (http://pastebin.com/016BfvU9) Brian - Package Base (http://pastebin.com/8JJeHuRT) Wim - TrueOS Desktop All-n-one (http://pastebin.com/VC0DPQUF) Daniel - Long Boots (http://pastebin.com/q7pFu7pR) Bryan - ZFS / FreeNAS (http://pastebin.com/xgUnbzr7) Bryan - FreeNAS Security (http://pastebin.com/qqCvVTLB) ***

On this episode of BSDNow, we will be talking to Glen Barber and Peter Wemm of the FreeBSD RE and Cluster Admin teams! That plus our This episode was brought to you by Headlines 2016 FreeBSD Community Survey (https://www.surveymonkey.com/r/freebsd2016) We often get comments from our listeners, “I'm not a developer, how can I help out”? Well today is your chance to do something. The FreeBSD Foundation has its 2016 Community Survey online, where they are asking for feedback from you! I just did the survey, it'll take you about 5 minutes, but gives you a chance to provide valuable feedback to the foundation about things that are important to you. Be sure to answer in as much detail as possible and the foundation will review and use this feedback for its operations going forward. *** ART, OpenBSDs new routing table, single thread performances (http://www.grenadille.net/post/2016/06/17/ART-single-thread-performances) OpenBSD has changed the way routes are looked up in the kernel as part of their path to an SMP networking stack The “Allotment Routing Table” (ART) is a performance tradeoff, where more memory is used to store the routing table, in exchange for faster lookups With this new arrangement, a full BGP routing table will grow from 130MB to 180MB of memory “ART is a free multibit trie based routing table. To keep it simple, it can be seen as using more memory for fewer CPU cycles. In other words, we get a faster lookup by wasting memory. The original paper (http://www.hariguchi.org/art/art.pdf) presents some performance comparisons between two ART configurations and the BSD Radix. But how does this apply to OpenBSD?” “I asked Hrvoje Popovski to run his packet forwarding test on his Xeon box (E5-2620 v2 @ 2.10GHz, 2400.34 MHz) with ix(4) (82599) interfaces. The test setup consist of three machines with the OpenBSD box in the middle” “The simulations have been performed with an OpenBSD -current from June 9th. The machine is configured with pf(4) disabled in order to force a single route lookup for every IPv4 packet. Based on the result of the lookup the kernel decide if it should forward, deliver or drop the packet” *** BSDCan 2016 Playlist (https://www.youtube.com/playlist?list=PLeF8ZihVdpFfoEV67dBSrKfA8ifpUr6qC) The complete set of videos from BSDCan is online and ready to be consumed Remember the good-ole days where we would wait months (or years) to get videos posted from conferences? Well, who are we kidding, some conferences STILL do that, but we can't count BSDCan among them. Only two weeks out from this years exciting BSDCan, and all the videos have now landed on YouTube. Granted, this is no substitute for actually being at the conference, but even if you attended you probably missed quite a few of the talks. There are no videos of the hallway track, which is the best part of the conference Except the dinner discussion of course. and don't forget the hacker lounge *** Should you be scared of Unix signals? (http://jvns.ca/blog/2016/06/13/should-you-be-scared-of-signals/) Do you know much about UNIX Signals? Are you afraid of their complexity? Do you know there are signals other than SIGKILL? This article talks about the practical implications of signals from a programming perspective The things you need to consider when dealing with signals Basically, you register a “signal handler”, the function that will be run when a signal arrives As you program is running, if a signal arrives, your program will be interrupted. Its current state will be saved and any system calls in progress will return EINTR (Error, Interrupted), then your signal handler will be run. Once the signal handler is complete, the state of your application will be restored, and execution will resume As long as your program properly handles this interruption, and errors that might result from it (getting EINTR from a read() call, instead of the data you expected), then everything should be fine. Of course, you need to be careful what you do inside your signal handler, as if you modify any variables or state in your application, it might be very confused when it resumes. *** Interview - Glen and Peter- News Roundup Unik - The Unikernel Compilation and Deployment Platform (uses NetBSD's Rump) (https://github.com/emc-advanced-dev/unik) We've talked a bit about NetBSD's RUMP (unikernel) in the past, including articles on how to deploy services using it. Now we have an interesting project which makes the process super-easy, and dare-we-say almost “Docker-Like?” The Unik project has a fairly complete walkthrough right on their GitHub project page, including details on installation and creating your own unikernel containers. In addition, it provides instructions on boot-strapping your own Go/Node.js/Python/Java applications, and supports out of Box VCenter / AWS / Qemu / VirtualBox providers. *** PkgSrc 50th Release Highlights () pkgsrc is celebrating its 50th release, and to highlight this, they have posted a series of interviews from people who have been active in the project pkgsrc 50th release interviews - Jonathan Perkin (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interviews_jonathan) pkgsrc 50th release interviews - Ryo ONODERA (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interviews_ryo) pkgsrc 50th release interviews - Joerg Sonnenberg (http://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interview_with) pkgsrc 50th release interviews - Sevan Janiyan (https://blog.netbsd.org/tnf/entry/pkgsrc_50th_release_interviews_sevan) *** Migrating to FreeBSD from Solaris 11 (http://justinholcomb.me/blog/2016/02/28/migration-to-freebsd-part1.html) Part 2 (http://justinholcomb.me/blog/2016/03/12/migration-to-freebsd-part2.html) Part 3 (http://justinholcomb.me/blog/2016/03/19/migration-to-freebsd-part3.html) Part 4 (http://justinholcomb.me/blog/2016/03/26/migration-to-freebsd-part4.html) Part 5 (http://justinholcomb.me/blog/2016/04/03/migration-to-freebsd-part5.html) *** How to chroot www/firefox on NetBSD (https://github.com/alnsn/localpkgsrc/tree/master/firefox-chroot) Looking for a jail-like method of running FireFox on NetBSD? (Or possibly other BSDs?) We have a github repo with details on how to setup and run FireFox using a chroot using a “webuser” account for safety. Think of this as a jail alternative, may be useful on systems with no jail support. Of interest is the method used to do X forwarding. It uses Xorg TCP listen option (which is often off by default for security reasons). Perhaps SSH X forwarding would be a better alternative. (Or nullfs mounts of /tmp) *** Beastie Bits Tredly - V1 Release Candidate (https://github.com/tredly/tredly/releases/tag/v1.0.0-rc.1) Call for Testing - ypldap testing against OpenLDAP and Microsoft Active Directory (http://lists.freebsd.org/pipermail/freebsd-current/2016-June/061775.html) BSD Magazine, June 2016 Out Now (https://bsdmag.org/) Hammer2 - Add xxhash to H2 and throw in debug stuff for performance testing (http://lists.dragonflybsd.org/pipermail/commits/2016-June/500610.html) chyves pre-announcement (http://justinholcomb.me/blog/2016/06/14/chyves-project-preannouncement.html) *** Feedback/Questions Michael - Versioning (http://pastebin.com/1hpGrmuL) Michael - Removing Encryption (http://pastebin.com/2PkrMGGx) Bostjan - PC-BSD Questions (http://pastebin.com/q5VdmNxG) Fong - ZFS Rollback (http://pastebin.com/2aedLV7d) Jochen - Docker on FBSD (http://pastebin.com/dneVZkXc) ***

Kris is on vacation this week, so allan flies solo, provides a recap of BSDCan & cover's a boatload of news including Microsoft This episode was brought to you by Headlines BSDCan Recap and Live Stream Videos (http://www.bsdcan.org/2016/) OpenBSD BSDCan 2016 papers now available (http://www.openbsd.org/papers) Allan's slides (http://allanjude.com/bsd/BSDCan2016_-_GELIBoot.pdf) and Paper (http://allanjude.com/bsd/AsiaBSDCon2016_geliboot_pdf1a.pdf) Michael W Lucas presents Allan with a gift (https://www.youtube.com/watch?v=LFgxAHkrSTg) “FreeBSD Mastery: Advanced ZedFS” (http://blather.michaelwlucas.com/archives/2698) Highlighted Tweets: Groff Arrives at BSDCan (https://twitter.com/Keltounet/status/740344735194320896) FreeBSD Foundation recognizes the contributions of Bryan Drewery, Rod Grimes, Warren Block, & Gleb Smirnoff (https://twitter.com/freebsdfndation/status/742456950676393984) A moment of silence and shots in memory in Benjamin Perrault @creepingfur (https://twitter.com/__briancallahan/status/741854476340858880) @gvnn3 sells the FreeBSD Foundation shirt off of his back for Charity (https://twitter.com/Keltounet/status/741763867471155201) Michael W. Lucas asks Matt Ahrens how to pronounce ZFS, “You can pronounce ZFS however you like, but if you pronounce it 'reiserfs', people might be confused.” (https://twitter.com/cperciva/status/741375414967410688) Sysadmin T-Shirt (https://twitter.com/BSDCan/status/741420633007874050) FreeBSD Dev Summit ran out of room on the chalkboards listing accomplishments of 11.0 (https://twitter.com/SeanChittenden/status/740904105388978176) List of things people have or want for FreeBSD 12 (https://twitter.com/Keltounet/status/740928627471159296) Matt Ahrens signing Allan's ZFS book (https://twitter.com/kprovst/status/741322268480049152?cn=bWVudGlvbg%3D%3D&refsrc=email) FreeBSD's new marketing strategy (https://twitter.com/cperciva/status/741707948469157889) Charity Auction: systemd whoopie cushion (https://twitter.com/HippyWizard/status/741768670704066560) Embarass OpenBSD's @HenningBrauer by donating $10 to charity for a selfie with him wearing a Linux t-shirt (https://twitter.com/juliefriday/status/741948048788586496) @GroffTheBSDGoat changes handlers, from @HenningBrauer to @GavinAtkinson (https://twitter.com/GroffTheBSDGoat/status/742415390798716928) Day 1 Video (https://www.youtube.com/watch?v=AOidjSS7Hsg) Day 2 Video (https://www.youtube.com/watch?v=z7pDnBO5wSM) Allan's GELIBoot talk (day 2) (https://www.youtube.com/watch?v=z7pDnBO5wSM&feature=youtu.be&list=PLeF8ZihVdpFfoEV67dBSrKfA8ifpUr6qC&t=4440) *** Media Coverage of Microsoft + FreeBSD story (https://azure.microsoft.com/en-us/blog/freebsd-now-available-in-azure-marketplace/) Microsoft has released their own custom image of FreeBSD 10.3 for the Azure Cloud “This means that not only can you quickly bring-up a FreeBSD VM in Azure, but also that in the event you need technical support, Microsoft support engineers can assist.” “Microsoft is the publisher of the FreeBSD image in the marketplace rather than the FreeBSD Foundation. The FreeBSD Foundation is supported by donations from the FreeBSD community, including companies that build their solutions on FreeBSD. They are not a solution provider or an ISV with a support organization but rather rely on a very active community that support one another. In order to ensure our customers have an enterprise SLA for their FreeBSD VMs running in Azure, we took on the work of building, testing, releasing and maintaining the image in order to remove that burden from the Foundation. We will continue to partner closely with the Foundation as we make further investments in FreeBSD on Hyper-V and in Azure.” "It's quite a significant milestone for FreeBSD community and for Microsoft to publish a supported FreeBSD image on Azure Marketplace. We really appreciate Microsoft's commitment and investment in FreeBSD project". - Justin T. Gibbs, President of FreeBSD Foundation Microsoft took a FreeBSD 10.3-RELEASE image and added additional patches, most of which they have upstreamed but that were too late for the regular 10.3 release cycle. Rather than requiring users to use a snapshot of the stable/10 branch, which would complicate the user experience, and complicate the job of the Microsoft support engineers, they created their own “certified” release This allows Microsoft to selectively deploy errata fixes to the image as well It is not clear how this affects update mechanisms like freebsd-update(8) The Register (http://www.theregister.co.uk/2016/06/09/microsoft_freebsd/) The Inquirer (http://www.theinquirer.net/inquirer/news/2461070/microsoft-creates-own-distribution-of-freebsd-for-azure-developers) Infoworld (http://www.infoworld.com/article/3082090/open-source-tools/is-microsoft-publishing-its-own-freebsd-yes-and-no.html) The Hacker News (http://thehackernews.com/2016/06/microsoft-azure-freebsd.html) Windows Report (http://windowsreport.com/microsoft-freebsd-10-3-ready-made-vm-image-azure/) Windows Club (http://news.thewindowsclub.com/microsoft-freebsd-operating-system-84375/) *** Select works poorly (http://www.tedunangst.com/flak/post/select-works-poorly) “At the bottom of the OpenBSD man page for select is a little note. “Internally to the kernel, select() and pselect() work poorly if multiple processes wait on the same file descriptor.” There's a similar warning in the poll man page. Where does this warning come from and what does it mean?” Ted found that at first glance, OpenBSD's select() appears to be quite bad: “whenever some data gets written, we call wakeup(&selwait);. Based on what we've seen so far, one can conclude that this is likely to be inefficient. Every time any socket has some data available, we wake up every selecting process in the system. Works poorly indeed.” After further investigation, it turns out to not be quite as bad When the select() is first setup, the PID of the process that cares about the FD is recorded in the selinfo struct If a second process runs select() on the same FD, the SI_COLL (Select Collision) flag is set on the selinfo struct When selwakeup() is called, if SI_COLL is set, all select()ing processes are woken up, and the sysctl kern.nselcoll is incremented. If the flag is not set, and only a single PID is waiting for activity on that FD, only that process is woken up “This is not an intractable problem. kevent avoids it entirely. Other implementations may too. But practically, does it need to be solved? My laptop says it's happened 43 times. A server with substantially more uptime says 0. Doesn't seem so bad.” *** Interview - Hans Petter Selasky - hps@freebsd.org (mailto:hps@freebsd.org) / @twitter (https://twitter.com/user) Designing FreeBSD's USB drivers, hooking up a piano to FreeBSD & more! *** News Roundup Timeline of libexpat random vulnerability (http://www.tedunangst.com/flak/post/timeline-of-libexpat-random-vulnerability) Do you use FreeBSD as web server? Why or why not? (https://news.ycombinator.com/item?id=11804565) 20 years of NetBSD code Bloat (http://kristerw.blogspot.sg/2016/05/20-years-of-netbsd-code-bloat.html) HP Chromebook 13 now booting OpenBSD (https://jcs.org/statuses/2016/06/08/740606952149942272/) UNIX for Poets (https://web.stanford.edu/class/cs124/lec/124-UnixForPoets.pdf) Comparing live version upgrade methods (https://distrowatch.com/weekly.php?issue=20160530#upgrades) My life with FreeBSD on a Thinkpad X220 (https://www.reddit.com/r/BSD/comments/4n3flx/my_life_with_freebsd_on_a_thinkpad_x220/)