POPULARITY
10 episodes with asiabsdcon
2 episodes with asiabsdcon
4 episodes with asiabsdcon
2 episodes with asiabsdcon
Key Considerations for Benchmarking Network Storage Performance, OpenZFS 2.3.0 available, Updates on AsiaBSDcon, GhostBSD Desktop Conference, Recovering from external zroot, Create a new issue in a Github repository with Ansible, Stories I refuse to believe, date limit in UFS1 filesystem extended, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines Key Considerations for Benchmarking Network Storage Performance (https://klarasystems.com/articles/considerations-benchmarking-network-storage-performance/) OpenZFS 2.3.0 available (https://github.com/openzfs/zfs/releases/tag/zfs-2.3.0) News Roundup Updates on AsiaBSDCon 2025 - Cancelled - (https://lists.asiabsdcon.org/pipermail/announce/2025-January/000046.html) GhostBSD Desktop Conference (https://www.phoronix.com/news/BSD-Desktop-Conference-GhostBSD) Recovering from external zroot (https://adventurist.me/posts/00350) Create a new issue in a Github repository with Ansible (https://jpmens.net/2025/01/25/create-a-new-issue-in-a-github-repository/) Stories I refuse to believe (https://flak.tedunangst.com/post/stories-i-refuse-to-believe) Defer the January 19, 2038 date limit in UFS1 filesystems to February 7, 2106 (https://cgit.freebsd.org/src/commit/?id=1111a44301da39d7b7459c784230e1405e8980f8) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Feedback - Nelson - Ada/GCC (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/598/feedback/Nelson%20Feedback.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
GPL 3: The Controversial Licensing Model and Potential Solutions, The Geeks way of checking what the outside weather is like, Alpine on a FreeBSD Jail, DragonFly BSD on a Thinkpad T480s, Dealing with USB Storage devices on OmniOS, Creating a Time Capsule instance using Samba, FreeBSD, and ZFS NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines GPL 3: The Controversial Licensing Model and Potential Solutions (https://klarasystems.com/articles/gpl-3-the-controversial-licensing-model-and-potential-solutions/) The Geeks way of checking what the outside wheather is like (https://blog.netbsd.org/tnf/entry/the_geeks_way_of_checking) News Roundup Alpine on a FreeBSD Jail (https://it-notes.dragas.net/2024/01/18/installing-alpine-linux-on-a-freebsd-jail/) DragonFly BSD on a Thinkpad T480s (https://git.sr.ht/~tomh/dragonflybsd-on-a-laptop/tree/master/item/README.md) Dealing with USB Storage devices on OmniOS (https://www.tumfatig.net/2024/dealing-with-usb-storage-devices-on-omnios/) Creating a Time Capsule instance using Samba, FreeBSD, and ZFS (https://dan.langille.org/2024/01/06/creating-a-time-capsule-instance-using-samba-freebsd-and-zfs-2/) Conferences FOSDEM (https://fosdem.org/2024/) AsiaBSDCon (https://2024.asiabsdcon.org/program.html) BSDCan (https://www.bsdcan.org/2024/papers.php) EuroBSDcon (https://2024.eurobsdcon.org/) Southeast Linuxfest (https://southeastlinuxfest.org/2024/01/self-2024-call-for-participation/) Dont let the name fool you, SELF is BSD friendly and they'd love to have BSD/Unix Talks if you're in the area. JT is staff at SELF, so he can put in a good word for you. ;) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
AsiaBSDCon 2023 Trip Report, Converting My X201 ThinkPad into a Slabtop, Stream your OpenBSD desktop audio to other devices, The Gnome and Its "Secret Place", ttyload, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines AsiaBSDCon 2023 Trip Report (https://freebsdfoundation.org/blog/asiabsdcon-2023-trip-report/) Converting My X201 ThinkPad into a Slabtop (https://bt.ht/slabtop/) News Roundup Stream your OpenBSD desktop audio to other devices (https://dataswamp.org/~solene/2023-05-05-openbsd-sound-streaming.html) The Gnome and Its "Secret Place" (https://www.tuhs.org/pipermail/tuhs/2023-May/028363.html) ttyload - Linux/Unix color-coded graphical tracking tool for load average in a terminal (https://www.cyberciti.biz/open-source/command-line-hacks/ttyload-color-coded-graphical-tracking-tool-for-unixlinux-load-average-in-a-terminal/) Beastie Bits • [OpenIndiana with a Sun Microsystems 22" LCD monitor. Running on a 1.8GHz quad core AMD Phenom 9100e processor, 4Gb RAM, nVidia GEForce GT630.](https://www.reddit.com/r/unix/comments/13otjnt/openindiana_with_a_sun_microsystems_22_lcd/) • [cron(8) now supports random ranges with steps](https://www.undeadly.org/cgi?action=article;sid=20230507122935&utm_source=bsdweekly) • [BSDCan 2024 Reorganization](https://mwl.io/archives/22799) • [Depenguin me](https://depenguin.me/) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
5 Key Reasons to Consider Open Source Storage, OpenBSD Minimalist Desktop, BSD XFCE, Alpine Linux VM on bhyve - with root on ZFS, FreeBSD Jail Quick Setup with Networking, and more. NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines 5 Key Reasons to Consider Open Source Storage Over Commercial Offerings (https://klarasystems.com/articles/open-source-storage-over-commercial-offerings/) OpenBSD Minimalist Desktop (https://nechtan.io/articles/openbsd_minimalist_desktop.html) News Roundup BSD-XFCE (https://github.com/Wamphyre/BSD-XFCE) Creating an Alpine Linux VM on bhyve - with root on ZFS (optionally encrypted) (https://it-notes.dragas.net/2022/11/01/creating-an-alpine-vm-on-bhyve-with-root-on-zfs-optionally-encrypted/) FreeBSD Jail Quick Setup with Networking (2022) (https://www.shaka.today/freebsd-jail-quick-setup-with-networking-2022/) Beastie Bits EuroBSDcon videos are now up (https://www.youtube.com/c/EuroBSDcon/videos) LibreSSL 3.6.1 released (https://undeadly.org/cgi?action=article;sid=20221104064712) Raspberry Pi 4 with FreeBSD 13-RELEASE: A Perfect Miniature Homelab (https://www.coreystephan.com/pi4-freebsd/) AsiaBSDcon 2023 CfP (https://2023.asiabsdcon.org/cfp.html.en) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Feedback/Questions John - Allan's meetup (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/482/feedback/John%20-%20Allan's%20meetup.md) Matthew - atime and a question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/482/feedback/Matthew%20-%20atime%20and%20a%20question.md) Valentin - Becoming a FreeBSD Developer (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/482/feedback/Valentin%20-%20Becoming%20a%20FreeBSD%20Developer.md) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***
We read FreeBSD’s third quarterly status report, OpenBSD on Sparc64, ZoL repo move to OpenZFS, GEOM NOP, keeping NetBSD up-to-date, and more. Headlines FreeBSD third quarterly status report for 2019 (https://www.freebsd.org/news/status/report-2019-07-2019-09.html) This quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work). Efficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner. Starting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October. Next quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January. OpenBSD on Sparc64 (https://eerielinux.wordpress.com/2019/10/10/openbsd-on-sparc64-6-0-to-6-5/) OpenBSD, huh? Yes, I usually write about FreeBSD and that’s in fact what I tried installing on the machine first. But I ran into problems with it very early on (never even reached single user mode) and put it aside for later. Since I powered up the SunFire again last month, I needed an OS now and chose OpenBSD for the simple reason that I have it available. First I wanted to call this article simply “OpenBSD on SPARC” – but that would have been misleading since OpenBSD used to support 32-bit SPARC processors, too. The platform was just put to rest after the 5.9 release. Version 6.0 was the last release of OpenBSD that came on CD-ROM. When I bought it, I thought that I’d never use the SPARC CD. But here was the chance! While it is an obsolete release, it comes with the cryptographic signatures to verify the next release. So the plan is to start at 6.0 as I can trust the original CDs and then update to the latest release. This will also be an opportunity to recap on some of the things that changed over the various versions. News Roundup ZoL repo move to OpenZFS (https://zfsonlinux.topicbox.com/groups/zfs-discuss/T13eedc32607dab41/zol-repo-move-to-openzfs) Because it will contain the ZFS source code for both Linux and FreeBSD, we will rename the "ZFSonLinux" code repository to "OpenZFS". Specifically, the repo at http://github.com/ZFSonLinux/zfs will be moved to the OpenZFS organization, at http://github.com/OpenZFS/zfs. The next major release of ZFS for Linux and FreeBSD will be "OpenZFS 2.0", and is expected to ship in 2020. Mcclure111 Sun Thread (https://twitter.com/mcclure111/status/1196557401710837762) A long time ago— like 15 years ago— I worked at Sun Microsystems. The company was nearly dead at the time (it died a couple years later) because they didn't make anything that anyone wanted to buy anymore. So they had a lot of strange ideas about how they'd make their comeback. GEOM NOP (https://oshogbo.vexillium.org/blog/71/) Sometimes while testing file systems or applications you want to simulate some errors on the disk level. The first time I heard about this need was from Baptiste Daroussin during his presentation at AsiaBSDCon 2016. He mentioned how they had built a test lab with it. The same need was recently discussed during the PGCon 2019, to test a PostgreSQL instance. If you are FreeBSD user, I have great news for you: there is a GEOM provider which allows you to simulate a failing device. GNOP allows us to configure transparent providers from existing ones. The first interesting option of it is that we can slice the device into smaller pieces, thanks to the ‘offset option’ and ‘stripsesize’. This allows us to observe how the data on the disk is changing. Let’s assume that we want to observe the changes in the GPT table when the GPT flags are added or removed (for example the bootme flags which are described here). We can use dd every time and analyze it using absolute values from the disks. Keeping NetBSD up-to-date with pkg_comp 2.0 (https://jmmv.dev/2017/02/pkg_comp-2.0-tutorial-netbsd.html) This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD. Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure. This tutorial is specifically targeted at NetBSD but should work on other platforms with some small changes. Expect, at the very least, a macOS-specific tutorial as soon as I create a pkg_comp standalone installer for that platform. Beastie Bits DragonFly - Radeon Improvements (http://lists.dragonflybsd.org/pipermail/commits/2019-November/720070.html) NomadBSD review (https://www.youtube.com/watch?v=7DglP7SbnlA&feature=share) Spongebob OpenBSD Security Comic (https://files.yukiisbo.red/openbsd_claim.png) Forth : The Early Years (https://colorforth.github.io/HOPL.html) LCM+L PDP-7 booting and running UNIX Version 0 (https://www.youtube.com/watch?v=pvaPaWyiuLA) Feedback/Questions Chris - Ctrl-T (http://dpaste.com/284E5BV) Improved Ctrl+t that shows kernel backtrace (https://asciinema.org/a/xfSpvPT61Cnd9iRgbfIjT6kYj) Brian - Migrating NexentaStore to FreeBSD/FreeNAS (http://dpaste.com/05GDK8H#wrap) Avery - How to get involved (http://dpaste.com/26KW801#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.
FreeBSD Q4 2018 status report, the GhostBSD alternative, the coolest 90s laptop, OpenSSH 8.0 with quantum computing resistant keys exchange, project trident: 18.12-U8 is here, and more.
FreeBSD Q4 2018 status report, the GhostBSD alternative, the coolest 90s laptop, OpenSSH 8.0 with quantum computing resistant keys exchange, project trident: 18.12-U8 is here, and more.
FreeBSD Q4 2018 status report, the GhostBSD alternative, the coolest 90s laptop, OpenSSH 8.0 with quantum computing resistant keys exchange, project trident: 18.12-U8 is here, and more. ##Headlines ###AsiaBSDcon 2019 recap Both Allan and I attended AsiaBSDcon 2019 in Tokyo in mid march. After a couple of days of Tokyo sightseeing and tasting the local food, the conference started with tutorials. Benedict gave his tutorial about “BSD-based Systems Monitoring with Icinga2 and OpenSSH”, while Allan ran the FreeBSD developer summit. On the next day, Benedict attended the tutorial “writing (network) tests for FreeBSD” held by Kristof Provost. I learned a lot about Kyua, where tests live and how they are executed. I took some notes, which will likely become an article or chapter in the developers handbook about writing tests. On the third day, Hiroki Sato officially opened the paper session and then people went into individual talks. Benedict attended Adventure in DRMland - Or how to write a FreeBSD ARM64 DRM driver by Emmanuel Vadot powerpc64 architecture support in FreeBSD ports by Piotr Kubaj Managing System Images with ZFS by Allan Jude FreeBSD - Improving block I/O compatibility in bhyve by Sergiu Weisz Security Fantasies and Realities for the BSDs by George V. Neville-Neil ZRouter: Remote update of firmware by Hiroki Mori Improving security of the FreeBSD boot process by Marcin Wojtas Allan attended Adventures in DRMland by Emmanuel Vadot Intel HAXM by Kamil Rytarowski BSD Solutions in Australian NGOs Container Migration on FreeBSD by Yuhei Takagawa Security Fantasies and Realities for the BSDs by George Neville-Neil ZRouter: Remote update of firmware by Hiroki Mori Improving security of the FreeBSD boot process by Marcin Wojtas When not in talks, time was spent in the hallway track and conversations would often continue over dinner. Stay tuned for announcements about where AsiaBSDcon 2020 will be, as the Tokyo Olympics will likely force some changes for next year. Overall, it was nice to see people at the conference again, listen to talks, and enjoy the hospitality of Japan. ###FreeBSD Quarterly Status Report - Fourth Quarter 2018 Since we are still on this island among many in this vast ocean of the Internet, we write this message in a bottle to inform you of the work we have finished and what lies ahead of us. These deeds that we have wrought with our minds and hands, they are for all to partake of - in the hopes that anyone of their free will, will join us in making improvements. In todays message the following by no means complete or ordered set of improvements and additions will be covered: i386 PAE Pagetables for up to 24GB memory support, Continuous Integration efforts, driver updates to ENA and graphics, ARM enhancements such as RochChip, Marvell 8K, and Broadcom support as well as more DTS files, more Capsicum possibilities, as well as pfsync improvements, and many more things that you can read about for yourselves. Additionally, we bring news from some islands further down stream, namely the nosh project, HardenedBSD, ClonOS, and the Polish BSD User-Group. We would, selfishly, encourage those of you who give us the good word to please send in your submissions sooner than just before the deadline, and also encourage anyone willing to share the good word to please read the section on which submissions we’re also interested in having. ###GhostBSD: A Solid Linux-Like Open Source Alternative The subject of this week’s Linux Picks and Pans is a representative of a less well-known computing platform that coexists with Linux as an open source operating system. If you thought that the Linux kernel was the only open source engine for a free OS, think again. BSD (Berkeley Software Distribution) shares many of the same features that make Linux OSes viable alternatives to proprietary computing platforms. GhostBSD is a user-friendly Linux-like desktop operating system based on TrueOS. TrueOS is, in turn, based on FreeBSD’s development branch. TrueOS’ goal is to combine the stability and security of FreeBSD with a preinstalled GNOME, MATE, Xfce, LXDE or Openbox graphical user interface. I stumbled on TrueOS while checking out new desktop environments and features in recent new releases of a few obscure Linux distros. Along the way, I discovered that today’s BSD computing family is not the closed source Unix platform the “BSD” name might suggest. In last week’s Redcore Linux review, I mentioned that the Lumina desktop environment was under development for an upcoming Redcore Linux release. Lumina is being developed primarily for BSD OSes. That led me to circle back to a review I wrote two years ago on Lumina being developed for Linux. GhostBSD is a pleasant discovery. It has nothing to do with being spooky, either. That goes for both the distro and the open source computing family it exposes. Keep reading to find out what piqued my excitement about Linux-like GhostBSD. ##News Roundup SPARCbook 3000ST - The coolest 90s laptop A few weeks back I managed to pick up an incredibly rare laptop in immaculate condition for $50 on Kijiji: a Tadpole Technologies SPARCbook 3000ST from 1997 (it also came with two other working Pentium laptops from the 1990s). Sun computers were an expensive desire for many computer geeks in the 1990s, and running UNIX on a SPARC-based laptop was, well, just as cool as it gets. SPARC was an open hardware platform that anyone could make, and Tadpole licensed the Solaris UNIX operating system from Sun for their SPARCbooks. Tadpole essentially made high-end UNIX/VAX workstations on costly, unusual platforms (PowerPC, DEC Alpha, SPARC) but only their SPARCbooks were popular in the high-end UNIX market of the 1990s. ###OpenSSH 8.0 Releasing With Quantum Computing Resistant Keys OpenSSH 7.9 came out with a host of bug fixes last year with few new features, as is to be expected in minor releases. However, recently, Damien Miller has announced that OpenSSH 8.0 is nearly ready to be released. Currently, it’s undergoing testing to ensure compatibility across supported systems. https://twitter.com/damienmiller/status/1111416334737244160 Better Security Copying filenames with scp will be more secure in OpenSSH 8.0 due to the fact that copying filenames from a remote to local directory will prompt scp to check if the files sent from the server match your request. Otherwise, an attack server would theoretically be able to intercept the request by serving malicious files in place of the ones originally requested. Knowing this, you’re probably better off never using scp anyway. OpenSSH advises against it: “The scp protocol is outdated, inflexible and not readily fixed. We recommend the use of more modern protocols like sftp and rsync for file transfer instead.” Interesting new features ssh(1): When prompting whether to record a new host key, accept the key fingerprint as a synonym for “yes”. This allows the user to paste a fingerprint obtained out of band at the prompt and have the client do the comparison for you. ###Project Trident : 18.12-U8 Available Thank you all for your patience! Project Trident has finally finished some significant infrastructure updates over the last 2 weeks, and we are pleased to announce that package update 8 for 18.12-RELEASE is now available. To switch to the new update, you will need to open the “Configuration” tab in the update manager and switch to the new “Trident-release” package repository. You can also perform this transition via the command line by running: sudo sysup --change-train Trident-release ##Beastie Bits BSD Router Project - Release 1.92 EuroBSDcon - New Proposals Funny UNIX shirt (René Magritte art parody) 51NB’s Thinkpad X210 DragonFly: No more gcc50 “FreeBSD Mastery: Jails” ebook escaping! FreeBSD talk at the Augsburger Linux Info Days (german) ##Feedback/Questions DJ - FuguIta Feedback Mike - Another Good Show Alex - GhostBSD and wifi Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv Your browser does not support the HTML5 video tag.
Project Trident 18.12 released, Spotifyd on NetBSD, OPNsense 18.7.10 is available, Ultra EPYC AMD Powered Sun Ultra 24 Workstation, OpenRsync, LLD porting to NetBSD, and more. ##Headlines ###AsiaBSDCon 2019 Call for Papers You have until Jan 30th to submit Full paper requirement is relaxed a bit this year (this year ONLY!) due to the short submission window. You don’t need all 10-12 pages, but it is still preferred. Send a message to secretary@asiabsdcon.org with your proposal. Could be either for a talk or a tutorial. Two days of tutorials/devsummit and two days of conference during Sakura season in Tokyo, Japan The conference is also looking for sponsors If accepted, flight and hotel is paid for by the conference ###Project Trident 18.12 Released Twitter account if you want to keep up on project news Screenshots Project Trident Community Telegram Channel DistroWatch Page LinuxActionNews Review RoboNuggie’s in depth review ###Building Spotifyd on NetBSD These are the steps I went through to build and run Spotifyd (this commit at the time of writing) on NetBSD AMD64. It’s a Spotify Connect client so it means I still need to control Spotify from another device (typically my phone), but the audio is played through my desktop… which is where my speakers and headphones are plugged in - it means I don’t have to unplug stuff and re-plug into my phone, work laptop, etc. This is 100% a “good enough for now solution” for me; I have had a quick play with the Go based microcontroller from spotcontrol and that allows a completely NetBSD only experience (although it is just an example application so doesn’t provide many features - great as a basis to build on though). ##News Roundup ###OPNsense 18.7.10 released 2019 means 19.1 is almost here. In the meantime accept this small incremental update with goodies such as Suricata 4.1, custom passwords for P12 certificate export as well as fresh fixes in the FreeBSD base. A lot of cleanups went into this update to make sure there will be a smooth transition to 19.1-RC for you early birds. We expect RC1 in 1-2 weeks and the final 19.1 on January 29. ###Introducing the Ultra EPYC AMD Powered Sun Ultra 24 Workstation A few weeks ago, I got an itch to build a workstation with AMD EPYC. There are a few constraints. First, I needed a higher-clock part. Second, I knew the whole build would be focused more on being an ultra high-end workstation rather than simply utilizing gaming components. With that, I decided it was time to hit on a bit of nostalgia for our readers. Mainly, I wanted to do an homage to Sun Microsystems. Sun made the server gear that the industry ran on for years, and as a fun fact, if you go behind the 1 Hacker Way sign at Facebook’s campus, they left the Sun Microsystems logo. Seeing that made me wonder if we could do an ultimate AMD EPYC build in a Sun Microsystems workstation. ###OpenRsync This is a clean-room implementation of rsync with a BSD (ISC) license. It is designed to be compatible with a modern rsync (3.1.3 is used for testing). It currently compiles and runs only on OpenBSD. This project is still very new and very fast-moving. It’s not ready for wide-spread testing. Or even narrow-spread beyond getting all of the bits to work. It’s not ready for strong attention. Or really any attention but by careful programming. Many have asked about portability. We’re just not there yet, folks. But don’t worry, the system is easily portable. The hard part for porters is matching OpenBSD’s pledge and unveil. ###The first report on LLD porting LLD is the link editor (linker) component of Clang toolchain. Its main advantage over GNU ld is much lower memory footprint, and linking speed. It is of specific interest to me since currently 8 GiB of memory are insufficient to link LLVM statically (which is the upstream default). The first goal of LLD porting is to ensure that LLD can produce working NetBSD executables, and be used to build LLVM itself. Then, it is desirable to look into trying to build additional NetBSD components, and eventually into replacing /usr/bin/ld entirely with lld. In this report, I would like to shortly summarize the issues I have found so far trying to use LLD on NetBSD. ###Ring in the new It’s the second week of 2019 already, which means I’m curious what Nate is going to do with his series This week in usability … reset the numbering from week 1? That series is a great read, to keep up with all the little things that change in KDE source each week — aside from the release notes. For the big ticket items of KDE on FreeBSD, you should read this blog instead. In ports this week (mostly KDE, some unrelated): KDE Plasma has been updated to the latest release, 5.14.5. KDE Applications 18.12.1 were released today, so we’re right on top of them. Marble was fixed for FreeBSD-running-on-Power9. Musescore caught up on 18 months of releases. Phonon updated to 4.10.1, along with its backends. And in development, Qt WebEngine 5.12 has been prepared in the incongruously-named plasma-5.13 branch in Area51; that does contain all the latest bits described above, as well. ##Beastie Bits NomadBSD 1.2-RC1 Released ZFS - The First Enterprise Blockchain Powersaving with DragonFly laptop NetBSD reaches 100% reproducable builds Potential Bhyve Web Interface? LibGDX proof of concept on OpenBSD - Video LiteCLI is a user-friendly CommandLine client for SQLite database In honor of Donald Knuth’s 81 birthday Stanford uploaded 111 lectures on Youtube Portland BSD Pizza Night - 2018-01-31 19:00 - Sweet Heart Pizza Stockholm BSD February meetup Polish BSD User Group: Jan 25 18:15 - 21:00 AsiaBSDcon 2019 CfP ##Feedback/Questions Greg - VLANs and jails Tara - ZFS on removable disks Casey - Interview with Kirk McKusick Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
OpenBSD on Microsoft Surface Go, FreeBSD Foundation August Update, What’s taking so long with Project Trident, pkgsrc config file versioning, and MacOS remnants in ZFS code. ##Headlines OpenBSD on the Microsoft Surface Go For some reason I like small laptops and the constraints they place on me (as long as they’re still usable). I used a Dell Mini 9 for a long time back in the netbook days and was recently using an 11" MacBook Air as my primary development machine for many years. Recently Microsoft announced a smaller, cheaper version of its Surface tablets called Surface Go which piqued my interest. Hardware The Surface Go is available in two hardware configurations: one with 4Gb of RAM and a 64Gb eMMC, and another with 8Gb of RAM with a 128Gb NVMe SSD. (I went with the latter.) Both ship with an Intel Pentium Gold 4415Y processor which is not very fast, but it’s certainly usable. The tablet measures 9.65" across, 6.9" tall, and 0.3" thick. Its 10" diagonal 3:2 touchscreen is covered with Gorilla Glass and has a resolution of 1800x1200. The bezel is quite large, especially for such a small screen, but it makes sense on a device that is meant to be held, to avoid accidental screen touches. The keyboard and touchpad are located on a separate, removable slab called the Surface Go Signature Type Cover which is sold separately. I opted for the “cobalt blue” cover which has a soft, cloth-like alcantara material. The cover attaches magnetically along the bottom edge of the device and presents USB-attached keyboard and touchpad devices. When the cover is folded up against the screen, it sends an ACPI sleep signal and is held to the screen magnetically. During normal use, the cover can be positioned flat on a surface or slightly raised up about 3/4" near the screen for better ergonomics. When using the device as a tablet, the cover can be rotated behind the screen which causes it to automatically stop sending keyboard and touchpad events until it is rotated back around. The keyboard has a decent amount of key travel and a good layout, with Home/End/Page Up/Page Down being accessible via Fn+Left/Right/Up/Down but also dedicated Home/End/Page Up/Page Down keys on the F9-F12 keys which I find quite useful since the keyboard layout is somewhat small. By default, the F1-F12 keys do not send F1-F12 key codes and Fn must be used, either held down temporarily or Fn pressed by itself to enable Fn-lock which annoyingly keeps the bright Fn LED illuminated. The keys are backlit with three levels of adjustment, handled by the keyboard itself with the F7 key. The touchpad on the Type Cover is a Windows Precision Touchpad connected via USB HID. It has a decent click feel but when the cover is angled up instead of flat on a surface, it sounds a bit hollow and cheap. Surface Go Pen The touchscreen is powered by an Elantech chip connected via HID-over-i2c, which also supports pen input. A Surface Pen digitizer is available separately from Microsoft and comes in the same colors as the Type Covers. The pen works without any pairing necessary, though the top button on it works over Bluetooth so it requires pairing to use. Either way, the pen requires an AAAA battery inside it to operate. The Surface Pen can attach magnetically to the left side of the screen when not in use. A kickstand can swing out behind the display to use the tablet in a laptop form factor, which can adjust to any angle up to about 170 degrees. The kickstand stays firmly in place wherever it is positioned, which also means it requires a bit of force to pull it out when initially placing the Surface Go on a desk. Along the top of the display are a power button and physical volume rocker buttons. Along the right side are the 3.5mm headphone jack, USB-C port, power port, and microSD card slot located behind the kickstand. Charging can be done via USB-C or the dedicated charge port, which accommodates a magnetically-attached, thin barrel similar to Apple’s first generation MagSafe adapter. The charging cable has a white LED that glows when connected, which is kind of annoying since it’s near the mid-line of the screen rather than down by the keyboard. Unlike Apple’s MagSafe, the indicator light does not indicate whether the battery is charged or not. The barrel charger plug can be placed up or down, but in either direction I find it puts an awkward strain on the power cable coming out of it due to the vertical position of the port. Wireless connectivity is provided by a Qualcomm Atheros QCA6174 802.11ac chip which also provides Bluetooth connectivity. Most of the sensors on the device such as the gyroscope and ambient light sensor are connected behind an Intel Sensor Hub PCI device, which provides some power savings as the host CPU doesn’t have to poll the sensors all the time. Firmware The Surface Go’s BIOS/firmware menu can be entered by holding down the Volume Up button, then pressing and releasing the Power button, and releasing Volume Up when the menu appears. Secure Boot as well as various hardware components can be disabled in this menu. Boot order can also be adjusted. A temporary boot menu can be brought up the same way but using Volume Down instead. ###FreeBSD Foundation Update, August 2018 MESSAGE FROM THE EXECUTIVE DIRECTOR Dear FreeBSD Community Member, It’s been a busy summer for the Foundation. From traveling around the globe spreading the word about FreeBSD to bringing on new team members to improve the Project’s Continuous Integration work, we’re very excited about what we’ve accomplished. Take a minute to check out the latest updates within our Foundation sponsored projects; read more about our advocacy efforts in Bangladesh and community building in Cambridge; don’t miss upcoming Travel Grant deadlines, and new Developer Summits; and be sure to find out how your support will ensure our progress continues into 2019. We can’t do this without you! Happy reading!! Deb August 2018 Development Projects Update Fundraising Update: Supporting the Project August 2018 Release Engineering Update BSDCam 2018 Recap October 2018 FreeBSD Developer Summit Call for Participation SANOG32 and COSCUP 2018 Recap MeetBSD 2018 Travel Grant Application Deadline: September 7 ##News Roundup Project Trident: What’s taking so long? What is taking so long? The short answer is that it’s complicated. Project Trident is quite literally a test of the new TrueOS build system. As expected, there have been quite a few bugs, undocumented features, and other optional bits that we discovered we needed that were not initially present. All of these things have to be addressed and retested in a constant back and forth process. While Ken and JT are both experienced developers, neither has done this kind of release engineering before. JT has done some release engineering back in his Linux days, but the TrueOS and FreeBSD build system is very different. Both Ken and JT are learning a completely new way of building a FreeBSD/TrueOS distribution. Please keep in mind that no one has used this new TrueOS build system before, so Ken and JT want to not only provide a good Trident release, but also provide a model or template for other potential TrueOS distributions too! Where are we now? Through perseverance, trial and error, and a lot of head-scratching we have reached the point of having successful builds. It took a while to get there, but now we are simply working out a few bugs with the new installer that Ken wrote as well as finding and fixing all the new Xorg configuration options which recently landed in FreeBSD. We also found that a number of services have been removed or replaced between TrueOS 18.03 and 18.06 so we are needing to adjust what we consider the “base” services for the desktop. All of these issues are being resolved and we are continually rebuilding and pulling in new patches from TrueOS as soon as they are committed. In the meantime we have made an early BETA release of Trident available to the users in our Telegram Channel for those who want to help out in testing these early versions. Do you foresee any other delays? At the moment we are doing many iterations of testing and tweaking the install ISO and package configurations in order to ensure that all the critical functionality works out-of-box (networking, sound, video, basic apps, etc). While we do not foresee any other major delays, sometimes things happen that our outside of our control. For an example, one of the recent delays that hit recently was completely unexpected: we had a hard drive failure on our build server. Up until recently, The aptly named “Poseidon” build server was running a Micron m500dc drive, but that drive is now constantly reporting errors. Despite ordering a replacement Western Digital Blue SSD several weeks ago, we just received it this past week. The drive is now installed with the builder back to full functionality, but we did lose many precious days with the delay. The build server for Project Trident is very similar to the one that JT donated to the TrueOS project. JT had another DL580 G7, so he donated one to the Trident Project for their build server. Poseidon also has 256GB RAM (64 x 4GB sticks) which is a smidge higher than what the TrueOS builder has. Since we are talking about hardware, we probably should address another question we get often, “What Hardware are the devs testing on?” So let’s go ahead and answer that one now. Developer Hardware JT: His main test box is a custom-built Intel i7 7700K system running 32GB RAM, dual Intel Optane 900P drives, and an Nvidia 1070 GTX with four 4K Acer Monitors. He also uses a Lenovo x250 ThinkPad alongside a desk full of x230t and x220 ThinkPads. One of which he gave away at SouthEast LinuxFest this year, which you can read about here. However it’s not done there, being a complete hardware hoarder, JT also tests on several Intel NUCs and his second laptop a Fujitsu t904, not to mention a Plethora of HP DL580 servers, a DL980 server, and a stack of BL485c, BL460c, and BL490c Blades in his HP c7000 and c3000 Bladecenter chassis. (Maybe it’s time for an intervention for his hardware collecting habits) Ken: For a laptop, he primarily uses a 3rd generation X1 Carbon, but also has an old Eee PC T101MT Netbook (dual core 1GHz, 2GB of memory) which he uses for verifying how well Trident works on low-end hardware. As far as workstations go, his office computer is an Intel i7 with an NVIDIA Geforce GTX 960 running three 4K monitors and he has a couple other custom-built workstations (1 AMD, 1 Intel+NVIDIA) at his home. Generally he assembled random workstations based on hardware that was given to him or that he could acquire cheap. Tim: is using a third gen X1 Carbon and a custom built desktop with an Intel Core i5-4440 CPU, 16 GiB RAM, Nvidia GeForce GTX 750 Ti, and a RealTek 8168 / 8111 network card. Rod: Rod uses… No one knows what Rod uses, It’s kinda like how many licks does it take to get to the center of a Tootsie-Roll Tootsie-Pop… the world may just never know. ###NetBSD GSoC: pkgsrc config file versioning A series of reports from the course of the summer on this Google Summer of Code project The goal of the project is to integrate with a VCS (Version Control System) to make managing local changes to config files for packages easier GSoC 2018 Reports: Configuration files versioning in pkgsrc, Part 1 Packages may install code (both machine executable code and interpreted programs), documentation and manual pages, source headers, shared libraries and other resources such as graphic elements, sounds, fonts, document templates, translations and configuration files, or a combination of them. Configuration files are usually the means through which the behaviour of software without a user interface is specified. This covers parts of the operating systems, network daemons and programs in general that don’t come with an interactive graphical or textual interface as the principal mean for setting options. System wide configuration for operating system software tends to be kept under /etc, while configuration for software installed via pkgsrc ends up under LOCALBASE/etc (e.g., /usr/pkg/etc). Software packaged as part of pkgsrc provides example configuration files, if any, which usually get extracted to LOCALBASE/share/examples/PKGBASE/. Don’t worry: automatic merging is disabled by default, set $VCSAUTOMERGE to enable it. In order to avoid breakage, installed configuration is backed up first in the VCS, separating user-modified files from files that have been already automatically merged in the past, in order to allow the administrator to easily restore the last manually edited file in case of breakage. VCS functionality only applies to configuration files, not to rc.d scripts, and only if the environment variable $NOVCS is unset. The version control system to be used as a backend can be set through $VCS. It default to RCS, the Revision Control System, which works only locally and doesn’t support atomic transactions. Other backends such as CVS are supported and more will come; these, being used at the explicit request of the administrator, need to be already installed and placed in a directory part of $PATH. GSoC 2018 Reports: Configuration files versioning in pkgsrc, part 2: remote repositories (git and CVS) pkgsrc is now able to deploy configuration from packages being installed from a remote, site-specific vcs repository. User modified files are always tracked even if automerge functionality is not enabled, and a new tool, pkgconftrack(1), exists to manually store user changes made outside of package upgrade time. Version Control software is executed as the same user running pkgadd or make install, unless the user is “root”. In this case, a separate, unprivileged user, pkgvcsconf, gets created with its own home directory and a working login shell (but no password). The home directory is not strictly necessary, it exists to facilitate migrations betweens repositories and vcs changes; it also serves to store keys used to access remote repositories. Using git instead of rcs is simply done by setting VCS=git in pkginstall.conf GSoC 2018 Reports: Configuration files versioning in pkgsrc, part 3: remote repositories (SVN and Mercurial) GSoC 2018 Reports: Configuration files versioning in pkgsrc, part 4: configuration deployment, pkgtools and future improvements Support for configuration tracking is in scripts, pkginstall scripts, that get built into binary packages and are run by pkgadd upon installation. The idea behind the proposal suggested that users of the new feature should be able to store revisions of their installed configuration files, and of package-provided default, both in local or remote repositories. With this capability in place, it doesn’t take much to make the scripts “pull” configuration from a VCS repository at installation time. That’s what setting VCSCONFPULL=yes in pkginstall.conf after having enabled VCSTRACKCONF does: You are free to use official, third party prebuilt packages that have no customization in them, enable these options, and point pkgsrc to a private conf repository. If it contains custom configuration for the software you are installing, an attempt will be made to use it and install it on your system. If it fails, pkginstall will fall back to using the defaults that come inside the package. RC scripts are always deployed from the binary package, if existing and PKGRCDSCRIPTS=yes in pkginstall.conf or the environment. This will be part of packages, not a separate solution like configuration management tools. It doesn’t support running scripts on the target system to customize the installation, it doesn’t come with its domain-specific language, it won’t run as a daemon or require remote logins to work. It’s quite limited in scope, but you can define a ROLE for your system in pkginstall.conf or in the environment, and pkgsrc will look for configuration you or your organization crafted for such a role (e.g., public, standalone webserver vs reverse proxy or node in a database cluster) ###A little bit of the one-time MacOS version still lingers in ZFS Once upon a time, Apple came very close to releasing ZFS as part of MacOS. Apple did this work in its own copy of the ZFS source base (as far as I know), but the people in Sun knew about it and it turns out that even today there is one little lingering sign of this hoped-for and perhaps prepared-for ZFS port in the ZFS source code. Well, sort of, because it’s not quite in code. Lurking in the function that reads ZFS directories to turn (ZFS) directory entries into the filesystem independent format that the kernel wants is the following comment: objnum = ZFSDIRENTOBJ(zap.zafirstinteger); / MacOS X can extract the object type here such as: * uint8t type = ZFSDIRENTTYPE(zap.zafirstinteger); */ Specifically, this is in zfsreaddir in zfsvnops.c . ZFS maintains file type information in directories. This information can’t be used on Solaris (and thus Illumos), where the overall kernel doesn’t have this in its filesystem independent directory entry format, but it could have been on MacOS (‘Darwin’), because MacOS is among the Unixes that support d_type. The comment itself dates all the way back to this 2007 commit, which includes the change ‘reserve bits in directory entry for file type’, which created the whole setup for this. I don’t know if this file type support was added specifically to help out Apple’s MacOS X port of ZFS, but it’s certainly possible, and in 2007 it seems likely that this port was at least on the minds of ZFS developers. It’s interesting but understandable that FreeBSD didn’t seem to have influenced them in the same way, at least as far as comments in the source code go; this file type support is equally useful for FreeBSD, and the FreeBSD ZFS port dates to 2007 too (per this announcement). Regardless of the exact reason that ZFS picked up maintaining file type information in directory entries, it’s quite useful for people on both FreeBSD and Linux that it does so. File type information is useful for any number of things and ZFS filesystems can (and do) provide this information on those Unixes, which helps make ZFS feel like a truly first class filesystem, one that supports all of the expected general system features. ##Beastie Bits Mac-like FreeBSD Laptop Syncthing on FreeBSD New ZFS Boot Environments Tool My system’s time was so wrong, that even ntpd didn’t work OpenSSH 7.8/7.8p1 (2018-08-24) EuroBSD (Sept 20-23rd) registration Early Bird Period is coming to an end MeetBSD (Oct 18-20th) is coming up fast, hurry up and register! AsiaBSDcon 2019 Dates ##Feedback/Questions Will - Kudos and a Question Peter - Fanless Computers Ron - ZFS disk clone or replace or something Bostjan - ZFS Record Size Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
How Intel docs were misinterpreted by almost any OS, a look at the mininet SDN emulator, do’s and don’ts for FreeBSD, OpenBSD community going gold, ed mastery is a must read, and the distributed object store minio on FreeBSD. Headlines Intel documentation flaw sees instruction misimplemented in almost every OS A statement in the System Programming Guide of the Intel 64 and IA-32 Architectures Software Developer's Manual (SDM) was mishandled in the development of some or all operating-system kernels, resulting in unexpected behavior for #DB exceptions that are deferred by MOV SS or POP SS, as demonstrated by (for example) privilege escalation in Windows, macOS, some Xen configurations, or FreeBSD, or a Linux kernel crash. OS kernels may not expect this order of events and may therefore experience unexpected behavior when it occurs. + A detailed white paper describes this behavior here + FreeBSD Commit Thank you to the MSRC Incident Response Team, and in particular Greg Lenti and Nate Warfield, for coordinating the response to this issue across multiple vendors. Thanks to Computer Recycling at The Working Center of Kitchener for making hardware available to allow us to test the patch on additional CPU families. + FreeBSD Security Advisory + DragonFlyBSD Post + NetBSD does not support debug register and so is not affected. + OpenBSD also appears to not be affected, “We are not aware of further vendor information regarding this vulnerability.” + IllumOS Not Impacted Guest Post – A Look at SDN Emulator Mininet A guest post on the FreeBSD Foundation’s blog by developer Ayaka Koshibe At this year’s AsiaBSDCon, I presented a talk about a SDN network emulator called Mininet, and my ongoing work to make it more portable. That presentation was focused on the OpenBSD version of the port, and I breezed past the detail that I also had a version or Mininet working on FreeBSD. Because I was given the opportunity, I’d like to share a bit about the FreeBSD version of Mininet. It will not only be about what Mininet is and why it might be interesting, but also a recounting of my experience as a user making a first-time attempt at porting an application to FreeBSD. Mininet started off as a tool used by academic researchers to emulate OpenFlow networks when they didn’t have convenient access to actual networks. Because of its history, Mininet became associated strongly with networks that use OpenFlow for their control channels. But, it has also become fairly popular among developers working in, and among several universities for research and teaching about, SDN (Software Defined Networking) I began using Mininet as an intern at my university’s network research lab. I was using FreeBSD by that time, and wasn’t too happy to learn that Mininet wouldn’t work on anything but Linux. I gradually got tired of having to run a Linux VM just to use Mininet, and one day it clicked in my mind that I can actually try porting it to FreeBSD. Mininet creates a topology using the resource virtualization features that Linux has. Specifically, nodes are bash processes running in network namespaces, and the nodes are interconnected using veth virtual Ethernet links. Switches and controllers are just nodes whose shells have run the right commands to configure a software switch or start a controller application. Mininet can therefore be viewed as a series of Python libraries that run the system commands necessary to create network namespaces and veth interfaces, assemble a specified topology, and coordinate how user commands aimed at nodes (since they are just shells) are run. Coming back to the port, I chose to use vnet jails to replace the network namespaces, and epair(4) links to replace the veth links. For the SDN functionality, I needed at least one switch and controller that can be run on FreeBSD. I chose OpenvSwitch(OVS) for the switch, since it was available in ports and is well-known by the SDN world, and Ryu for the controller since it’s being actively developed and used and supports more recent versions of OpenFlow. I have discussed the possibility of upstreaming my work. Although they were excited about it, I was asked about a script for creating VMs with Mininet preinstalled, and continuous integration support for my fork of the repository. I started taking a look at the release scripts for creating a VM, and after seeing that it would be much easier to use the scripts if I can get Mininet and Ryu added to the ports tree, I also tried a hand at submitting some ports. For CI support, Mininet uses Travis, which unfortunately doesn’t support FreeBSD. For this, I plan to look at a minimalistic CI tool called contbuild, which looks simple enough to get running and is written portably. This is very much a work-in-progress, and one going at a glacial pace. Even though the company that I work for does use Mininet, but doesn’t use FreeBSD, so this is something that I’ve been working on in my free time. Earlier on, it was the learning curve that made progress slow. When I started, I hadn’t done anything more than run FreeBSD on a laptop, and uneventfully build a few applications from the ports tree. Right off the bat, using vnet jails meant learning how to build and run a custom kernel. This was the easy part, as the handbook was clear about how to do this. When I moved from using FreeBSD 10.3 to 11, I found that I can panic my machine by quickly creating and destroying OVS switches and jails. I submitted a bug report, but decided to go one step further and actually try to debug the panic for myself. With the help of a few people well-versed in systems programming and the developer’s handbook, I was able to come up with a fix, and get it accepted. This pretty much brings my porting experiment to the present day, where I’m slowly working out the pieces that I mentioned earlier. In the beginning, I thought that this Mininet port would be a weekend project where I come out knowing thing or two about using vnet jails and with one less VM to run. Instead, it became a crash course in building and debugging kernels and submitting bug reports, patches, and ports. It’d like to mention that I wouldn’t have gotten far at all if it weren’t for the helpful folks, the documentation, and how debuggable FreeBSD is. I enjoy good challenges and learning experiences, and this has definitely been both. Thank you to Ayaka for working to port Mininet to the BSDs, and for sharing her experiences with us. If you want to see the OpenBSD version of the talk, the video from AsiaBSDCon is here, and it will be presented again at BSDCan. **iXsystems** [iXsystems LFNW Recap](https://www.ixsystems.com/blog/lfnw-2018-recap/) 10 Beginner Do's and Don't for FreeBSD 1) Don't mix ports and binary packages 2) Don't edit 'default' files 3) Don't mess with /etc/crontab 4) Don't mess with /etc/passwd and /etc/groups either! 5) Reconsider the removal of any options from your customized kernel configuration 6) Don't change the root shell to something else 7) Don't use the root user all the time 8) /var/backups is a thing 9) Check system integrity using /etc/mtree 10) What works for me doesn't have to work for you! News Roundup OpenBSD Community Goes Gold for 2018! Ken Westerback (krw@ when wearing his developer hat) writes: ``` Monthly paypal donations from the OpenBSD community have made the community the OpenBSD Foundation's first Gold level contributor for 2018! 2018 is the third consecutive year that the community has reached Gold status or better. These monthly paypal commitments by the community are our most reliable source of funds and thus the most useful for financial planning purposes. We are extremely thankful for the continuing support and hope the community matches their 2017 achievement of Platinum status. Or even their 2016 achievement of Iridium status. Sign up now for a monthly donation! Note that Bitcoin contributions have been re-enabled now that our Bitcoin intermediary has re-certified our Canadian paperwork. https://www.openbsdfoundation.org/donations.html ``` ed(1) mastery is a must read for real unix people In some circles on the Internet, your choice of text editor is a serious matter. We've all seen the threads on mailing lits, USENET news groups and web forums about the relative merits of Emacs vs vi, including endless iterations of flame wars, and sometimes even involving lesser known or non-portable editing environments. And then of course, from the Linux newbies we have seen an endless stream of tweeted graphical 'memes' about the editor vim (aka 'vi Improved') versus the various apparently friendlier-to-some options such as GNU nano. Apparently even the 'improved' version of the classical and ubiquitous vi(1) editor is a challenge even to exit for a significant subset of the younger generation. Yes, your choice of text editor or editing environment is a serious matter. Mainly because text processing is so fundamental to our interactions with computers. But for those of us who keep our systems on a real Unix (such as OpenBSD or FreeBSD), there is no real contest. The OpenBSD base system contains several text editors including vi(1) and the almost-emacs mg(1), but ed(1) remains the standard editor. Now Michael Lucas has written a book to guide the as yet uninitiated to the fundamentals of the original Unix text editor. It is worth keeping in mind that much of Unix and its original standard text editor written back when the standard output and default user interface was more likely than not a printing terminal. To some of us, reading and following the narrative of Ed Mastery is a trip down memory lane. To others, following along the text will illustrate the horror of the world of pre-graphic computer interfaces. For others again, the fact that ed(1) doesn't use your terminal settings much at all offers hope of fixing things when something or somebody screwed up your system so you don't have a working terminal for that visual editor. DigitalOcean Digital Ocean Promo Link for BSD Now Listeners Distributed Object Storage with Minio on FreeBSD Free and open source distributed object storage server compatible with Amazon S3 v2/v4 API. Offers data protection against hardware failures using erasure code and bitrot detection. Supports highly available distributed setup. Provides confidentiality, integrity and authenticity assurances for encrypted data with negligible performance overhead. Both server side and client side encryption are supported. Below is the image of example Minio setup. Architecture Diagram The Minio identifies itself as the ZFS of Cloud Object Storage. This guide will show You how to setup highly available distributed Minio storage on the FreeBSD operating system with ZFS as backend for Minio data. For convenience we will use FreeBSD Jails operating system level virtualization. Setup The setup will assume that You have 3 datacenters and assumption that you have two datacenters in whose the most of the data must reside and that the third datacenter is used as a ‘quorum/witness’ role. Distributed Minio supports up to 16 nodes/drives total, so we may juggle with that number to balance data between desired datacenters. As we have 16 drives to allocate resources on 3 sites we will use 7 + 7 + 2 approach here. The datacenters where most of the data must reside have 7/16 ratio while the ‘quorum/witness’ datacenter have only 2/16 ratio. Thanks to built in Minio redundancy we may loose (turn off for example) any one of those machines and our object storage will still be available and ready to use for any purpose. Jails First we will create 3 jails for our proof of concept Minio setup, storage1 will have the ‘quorum/witness’ role while storage2 and storage3 will have the ‘data’ role. To distinguish commands I type on the host system and storageX Jail I use two different prompts, this way it should be obvious what command to execute and where. WeI know the FreeNAS people have been working on integrating this Best practises for pledge(2) security Let's set the record straight for securing kcgi CGI and FastCGI applications with pledge(2). This is focussed on secure OpenBSD deployments. Theory Internally, kcgi makes considerable use of available security tools. But it's also designed to be invoked in a secure environment. We'll start with pledge(2), which has been around on OpenBSD since version 5.9. If you're reading this tutorial, you're probably on OpenBSD, and you probably have knowledge of pledge(2). How to begin? Read kcgi(3). It includes canonical information on which pledge(2) promises you'll need for each function in the library. This is just a tutorial—the manpage is canonical and overrides what you may read here. Next, assess the promises that your application needs. From kcgi(3), it's easy to see which promises we'll need to start. You'll need to augment this list with whichever tools you're also using. The general push is to start with the broadest set of required promises, then restrict as quickly as possible. Sometimes this can be done in a single pledge(2), but other times it takes a few. Beastie Bits April's London *BSD meetup - notes May’s London *BSD Meetup: May 22nd Call for Papers for EuroBSDcon 2018 FreeBSD Journal March/April Desktop/Laptop issue LWN followup on the PostgreSQL fsync() issue The Association for Computing Machinery recognizes Steve Bourne for outstanding contributions Feedback/Questions Ray - Speaking at Conferences Casey - Questions Jeremy - zfs in the enterprise HAST + ZFS Lars - Civil Infrastructure Platform use of *BSD Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
OpenBSD firewalling Windows 10, NetBSD’s return to ptrace, TCP Alternative Backoff, the BSD Poetic license, and AsiaBSDcon 2018 videos available. RSS Feeds: MP3 Feed | iTunes Feed | HD Vid Feed | HD Torrent Feed Become a supporter on Patreon: - Show Notes: - Headlines Preventing Windows 10 and untrusted software from having full access to the internet using OpenBSD Whilst setting up one of my development laptops to port some software to Windows I noticed Windows 10 doing crazy things like installing or updating apps and games by default after initial setup. The one I noticed in particular was Candy Crush Soda Saga which for those who don't know of it is some cheesy little puzzle game originally for consumer devices. I honestly did not want software like this near to a development machine. It has also been reported that Windows 10 now also updates core system software without notifying the user. Surely this destroys any vaguely deterministic behaviour, in my opinion making Windows 10 by default almost useless for development testbeds. Deciding instead to start from scratch but this time to set the inbuilt Windows Firewall to be very restrictive and only allow a few select programs to communicate. In this case all I really needed to be online was Firefox, Subversion and Putty. To my amusement (and astonishment) I found out that the Windows firewall could be modified to give access very easily by programs during installation (usually because this task needs to be done with admin privileges). It also seems that Windows store Apps can change the windows firewall settings at any point. One way to get around this issue could be to install a 3rd party firewall that most software will not have knowledge about and thus not attempt to break through. However the only decent firewall I have used was Sygate Pro which unfortunately is no longer supported by recent operating systems. The last supported versions was 2003, XP and 2000. In short, I avoid 3rd party firewalls. Instead I decided to trap Windows 10 (and all of it's rogue updaters) behind a virtual machine running OpenBSD. This effectively provided me with a full blown firewall appliance. From here I could then allow specific software I trusted through the firewall (via a proxy) in a safe, controlled and deterministic manner. For other interested developers (and security conscious users) and for my own reference, I have listed the steps taken here: 1) First and foremost disable the Windows DHCP service - this is so no IP can be obtained on any interface. This effectively stops any communication with any network on the host system. This can be done by running services.msc with admin privileges and stopping and disabling the service called DHCP Client. 2) Install or enable your favorite virtualization software - I have tested this with both VirtualBox and Hyper-V. Note that on non-server versions of Windows, in order to get Hyper-V working, your processor also needs to support SLAT which is daft so to avoid faffing about, I recommend using VirtualBox to get round this seemingly arbitrary restriction. 3) Install OpenBSD on the VM - Note, if you decide to use Hyper-V, its hardware support isn't 100% perfect to run OpenBSD and you will need to disable a couple of things in the kernel. At the initial boot prompt, run the following commands. config -e -o /bsd /bsd disable acpi disable mpbios 4) Add a host only virtual adapter to the VM - This is the one which we are going to connect through the VM with. Look at the IP that VirtualBox assigns this in network manager on the host machine. Mine was [b]192.168.56.1[/b]. Set up the adapter in the OpenBSD VM to have a static address on the same subnet. For example [b]192.168.56.2[/b]. If you are using Hyper-V and OpenBSD, make sure you add a "Legacy Interface" because no guest additions are available. Then set up a virtual switch which is host only. 5) Add a bridged adapter to the VM - then assign it to whichever interface you wanted to connect to the external network with. Note that if using Wireless, set the bridged adapters MAC address to the same as your physical device or the access point will reject it. This is not needed (or possible) on Hyper-V because the actual device is "shared" rather than bridged so the same MAC address is used. Again, if you use Hyper-V, then add another virtual switch and attach it to your chosen external interface. VMs in Hyper-V "share" an adapter within a virtual switch and there is the option to also disable the hosts ability to use this interface at the same time which is fine for an additional level of security if those pesky rogue apps and updaters can also enable / disable DHCP service one day which wouldn't be too surprising. 6) Connect to your network in the host OS - In case of Wireless, select the correct network from the list and type in a password if needed. Windows will probably say "no internet available", it also does not assign an IP address which is fine. 7) Install the Squid proxy package on the OpenBSD guest and enable the daemon ``` pkg_add squid echo 'squid_flags=""' >> /etc/rc.conf.local /etc/rc.d/squid start ``` We will use this service for a limited selection of "safe and trusted" programs to connect to the outside world from within the Windows 10 host. You can also use putty on the host to connect to the VM via SSH and create a SOCKS proxy which software like Firefox can also use to connect externally. 8) Configure the software you want to be able to access the external network with Firefox - go to the connection settings and specify the VMs IP address for the proxy. Subversion - modify the %HOME%AppDataRoamingSubversionservers file and change the HTTP proxy field to the VMs IP. This is important to communicate with GitHub via https:// (Yes, GitHub also supports Subversion). For svn:// addresses you can use Putty to port forward. Chromium/Chrome - unfortunately uses the global Windows proxy settings which defeats much of the purpose of this exercise if we were going to allow all of Windows access to the internet via the proxy. It would become mayhem again. However we can still use Putty to create a SOCKS proxy and then launch the browser with the following flags: --proxy-server="socks5://:" --host-resolver-rules="MAP * 0.0.0.0 , EXCLUDE " 9) Congratulations, you are now done - Admittedly this process can be a bit fiddly to set up but it completely prevents Windows 10 from making a complete mess. This solution is probably also useful for those who like privacy or don't like the idea of their software "phoning home". Hope you find this useful and if you have any issues, please feel free to leave questions in the comments. LLDB restoration and return to ptrace(2) I've managed to unbreak the LLDB debugger as much as possible with the current kernel and hit problems with ptrace(2) that are causing issues with further work on proper NetBSD support. Meanwhile, I've upstreamed all the planned NetBSD patches to sanitizers and helped other BSDs to gain better or initial support. LLDB Since the last time I worked on LLDB, we have introduced many changes to the kernel interfaces (most notably related to signals) that apparently fixed some bugs in Go and introduced regressions in ptrace(2). Part of the regressions were noted by the existing ATF tests. However, the breakage was only marked as a new problem to resolve. For completeness, the ptrace(2) code was also cleaned up by Christos Zoulas, and we fixed some bugs with compat32. I've fixed a crash in *NetBSD::Factory::Launch(), triggered on startup of the lldb-server application. Here is the commit message: ``` We cannot call process_up->SetState() inside the NativeProcessNetBSD::Factory::Launch function because it triggers a NULL pointer deference. The generic code for launching a process in: GDBRemoteCommunicationServerLLGS::LaunchProcess sets the mdebuggedprocessup pointer after a successful call to mprocessfactory.Launch(). If we attempt to call processup->SetState() inside a platform specific Launch function we end up dereferencing a NULL pointer in NativeProcessProtocol::GetCurrentThreadID(). Use the proper call processup->SetState(,false) that sets notifydelegates to false. ``` Sanitizers I suspended development of new features in sanitizers last month, but I was still in the process of upstreaming of local patches. This process was time-consuming as it required rebasing patches, adding dedicated tests, and addressing all other requests and comments from the upstream developers. I'm not counting hot fixes, as some changes were triggering build or test issues on !NetBSD hosts. Thankfully all these issues were addressed quickly. The final result is a reduction of local delta size of almost 1MB to less than 100KB (1205 lines of diff). The remaining patches are rescheduled for later, mostly because they depend on extra work with cross-OS tests and prior integration of sanitizers with the basesystem distribution. I didn't want to put extra work here in the current state of affairs and, I've registered as a mentor for Google Summer of Code for the NetBSD Foundation and prepared Software Quality improvement tasks in order to outsource part of the labour. Userland changes I've also improved documentation for some of the features of NetBSD, described in man-pages. These pieces of information were sometimes wrong or incomplete, and this makes covering the NetBSD system with features such as sanitizers harder as there is a mismatch between the actual code and the documented code. Some pieces of software also require better namespacing support, these days mostly for the POSIX standard. I've fixed few low-hanging fruits there and requested pullups to NetBSD-8(BETA). I thank the developers for improving the landed code in order to ship the best solutions for users. BSD collaboration in LLVM A One-man-show in human activity is usually less fun and productive than collaboration in a team. This is also true in software development. Last month I was helping as a reviewer to port LLVM features to FreeBSD and when possible to OpenBSD. This included MSan/FreeBSD, libFuzzer/FreeBSD, XRay/FreeBSD and UBSan/OpenBSD. I've landed most of the submitted and reviewed code to the mainstream LLVM tree. Part of the code also verified the correctness of NetBSD routes in the existing porting efforts and showed new options for improvement. This is the reason why I've landed preliminary XRay/NetBSD code and added missing NetBSD bits to ToolChain::getOSLibName(). The latter produced setup issues with the prebuilt LLVM toolchain, as the directory name with compiler-rt goodies were located in a path like ./lib/clang/7.0.0/lib/netbsd8.99.12 with a varying OS version. This could stop working after upgrades, so I've simplified it to "netbsd", similar to FreeBSD and Solaris. Prebuilt toolchain for testers I've prepared a build of Clang/LLVM with LLDB and compiler-rt features prebuilt on NetBSD/amd64 v. 8.99.12: llvm-clang-compilerrt-lldb-7.0.0beta_2018-02-28.tar.bz2 Plan for the next milestone With the approaching NetBSD 8.0 release I plan to finish backporting a few changes there from HEAD: Remove one unused feature from ptrace(2), PTSETSIGMASK & PTGETSIGMASK. I've originally introduced these operations with criu/rr-like software in mind, but they are misusing or even abusing ptrace(2) and are not regular process debuggers. I plan to remove this operation from HEAD and backport this to NetBSD-8(BETA), before the release, so no compat will be required for this call. Future ports of criu/rr should involve dedicated kernel support for such requirements. Finish the backport of UCMACHINE_FP() to NetBSD-8. This will allow use of the same code in sanitizers in HEAD and NetBSD-8.0. By popular demand, improve the regnsub(3) and regasub(3) API, adding support for more or less substitutions than 10. Once done, I will return to ptrace(2) debugging and corrections. DigitalOcean Working with the NetBSD kernel Overview When working on complex systems, such as OS kernels, your attention span and cognitive energy are too valuable to be wasted on inefficiencies pertaining to ancillary tasks. After experimenting with different environmental setups for kernel debugging, some of which were awkward and distracting from my main objectives, I have arrived to my current workflow, which is described here. This approach is mainly oriented towards security research and the study of kernel internals. Before delving into the details, this is the general outline of my environment: My host system runs Linux. My target system is a QEMU guest. I’m tracing and debugging on my host system by attaching GDB (with NetBSD x86-64 ABI support) to QEMU’s built-in GDB server. I work with NetBSD-current. All sources are built on my host system with the cross-compilation toolchain produced by build.sh. I use NFS to share the source tree and the build artifacts between the target and the host. I find IDEs awkward, so for codebase navigation I mainly rely on vim, tmux and ctags. For non-intrusive instrumentation, such as figuring out control flow, I’m using dtrace. Preparing the host system QEMU GDB NFS Exports Building NetBSD-current A word of warning Now is a great time to familiarize yourself with the build.sh tool and its options. Be especially carefull with the following options: -r Remove contents of TOOLDIR and DESTDIR before building. -u Set MKUPDATE=yes; do not run "make clean" first. Without this, everything is rebuilt, including the tools. Chance are, you do not want to use these options once you’ve successfully built the cross-compilation toolchain and your entire userland, because building those takes time and there aren’t many good reasons to recompile them from scratch. Here’s what to expect: On my desktop, running a quad-core Intel i5-3470 at 3.20GHz with 24GB of RAM and underlying directory structure residing on a SSD drive, the entire process took about 55 minutes. I was running make with -j12, so the machine was quite busy. On an old Dell D630 laptop, running Intel Core 2 Duo T7500 at 2.20GHz with 4GB of RAM and a slow hard drive (5400RPM), the process took approximatelly 2.5 hours. I was running make with -j4. Based on the temperature alerts and CPU clock throttling messages, it was quite a struggle. Acquiring the sources Compiling the sources Preparing the guest system Provisioning your guest Pkgin and NFS shares Tailoring the kernel for debugging Installing the new kernel Configuring DTrace Debugging the guest’s kernel News Roundup Add support for the experimental Internet-Draft "TCP Alternative Backoff” ``` Add support for the experimental Internet-Draft "TCP Alternative Backoff with ECN (ABE)" proposal to the New Reno congestion control algorithm module. ABE reduces the amount of congestion window reduction in response to ECN-signalled congestion relative to the loss-inferred congestion response. More details about ABE can be found in the Internet-Draft: https://tools.ietf.org/html/draft-ietf-tcpm-alternativebackoff-ecn The implementation introduces four new sysctls: net.inet.tcp.cc.abe defaults to 0 (disabled) and can be set to non-zero to enable ABE for ECN-enabled TCP connections. net.inet.tcp.cc.newreno.beta and net.inet.tcp.cc.newreno.betaecn set the multiplicative window decrease factor, specified as a percentage, applied to the congestion window in response to a loss-based or ECN-based congestion signal respectively. They default to the values specified in the draft i.e. beta=50 and betaecn=80. net.inet.tcp.cc.abe_frlossreduce defaults to 0 (disabled) and can be set to non-zero to enable the use of standard beta (50% by default) when repairing loss during an ECN-signalled congestion recovery episode. It enables a more conservative congestion response and is provided for the purposes of experimentation as a result of some discussion at IETF 100 in Singapore. The values of beta and betaecn can also be set per-connection by way of the TCPCCALGOOPT TCP-level socket option and the new CCNEWRENOBETA or CCNEWRENOBETA_ECN CC algo sub-options. Submitted by: Tom Jones tj@enoti.me Tested by: Tom Jones tj@enoti.me, Grenville Armitage garmitage@swin.edu.au Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D11616 ``` Meltdown-mitigation syspatch/errata now available The recent changes in -current mitigating the Meltdown vulnerability have been backported to the 6.1 and 6.2 (amd64) releases, and the syspatch update (for 6.2) is now available. 6.1 ``` Changes by: bluhm@cvs.openbsd.org 2018/02/26 05:36:18 Log message: Implement a workaround against the Meltdown flaw in Intel CPUs. The following changes have been backported from OpenBSD -current. Changes by: guenther@cvs.openbsd.org 2018/01/06 15:03:13 Log message: Handle %gs like %[def]s and reset set it in cpu_switchto() instead of on every return to userspace. Changes by: mlarkin@cvs.openbsd.org 2018/01/06 18:08:20 Log message: Add identcpu.c and specialreg.h definitions for the new Intel/AMD MSRs that should help mitigate spectre. This is just the detection piece, these features are not yet used. Part of a larger ongoing effort to mitigate meltdown/spectre. i386 will come later; it needs some machdep.c cleanup first. Changes by: mlarkin@cvs.openbsd.org 2018/01/07 12:56:19 Log message: remove all PG_G global page mappings from the kernel when running on Intel CPUs. Part of an ongoing set of commits to mitigate the Intel "meltdown" CVE. This diff does not confer any immunity to that vulnerability - subsequent commits are still needed and are being worked on presently. ok guenther, deraadt Changes by: mlarkin@cvs.openbsd.org 2018/01/12 01:21:30 Log message: IBRS -> IBRS,IBPB in identifycpu lines Changes by: guenther@cvs.openbsd.org 2018/02/21 12:24:15 Log message: Meltdown: implement user/kernel page table separation. On Intel CPUs which speculate past user/supervisor page permission checks, use a separate page table for userspace with only the minimum of kernel code and data required for the transitions to/from the kernel (still marked as supervisor-only, of course): - the IDT (RO) - three pages of kernel text in the .kutext section for interrupt, trap, and syscall trampoline code (RX) - one page of kernel data in the .kudata section for TLB flush IPIs (RW) - the lapic page (RW, uncachable) - per CPU: one page for the TSS+GDT (RO) and one page for trampoline stacks (RW) When a syscall, trap, or interrupt takes a CPU from userspace to kernel the trampoline code switches page tables, switches stacks to the thread's real kernel stack, then copies over the necessary bits from the trampoline stack. On return to userspace the opposite occurs: recreate the iretq frame on the trampoline stack, switch stack, switch page tables, and return to userspace. mlarkin@ implemented the pmap bits and did 90% of the debugging, diagnosing issues on MP in particular, and drove the final push to completion. Many rounds of testing by naddy@, sthen@, and others Thanks to Alex Wilson from Joyent for early discussions about trampolines and their data requirements. Per-CPU page layout mostly inspired by DragonFlyBSD. ok mlarkin@ deraadt@ Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:18:59 Log message: The GNU assembler does not understand 1ULL, so replace the constant with 1. Then it compiles with gcc, sign and size do not matter here. Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:27:14 Log message: The compile time assertion for cpu info did not work with gcc. Rephrase the condition in a way that both gcc and clang accept it. Changes by: guenther@cvs.openbsd.org 2018/02/22 13:36:40 Log message: Set the PG_G (global) bit on the special page table entries that are shared between the u-k and u+k tables, because they're actually in all tables. OpenBSD 6.1 errata 037 ``` 6.2 ``` Changes by: bluhm@cvs.openbsd.org 2018/02/26 05:29:48 Log message: Implement a workaround against the Meltdown flaw in Intel CPUs. The following changes have been backported from OpenBSD -current. Changes by: guenther@cvs.openbsd.org 2018/01/06 15:03:13 Log message: Handle %gs like %[def]s and reset set it in cpu_switchto() instead of on every return to userspace. Changes by: mlarkin@cvs.openbsd.org 2018/01/06 18:08:20 Log message: Add identcpu.c and specialreg.h definitions for the new Intel/AMD MSRs that should help mitigate spectre. This is just the detection piece, these features are not yet used. Part of a larger ongoing effort to mitigate meltdown/spectre. i386 will come later; it needs some machdep.c cleanup first. Changes by: mlarkin@cvs.openbsd.org 2018/01/07 12:56:19 Log message: remove all PG_G global page mappings from the kernel when running on Intel CPUs. Part of an ongoing set of commits to mitigate the Intel "meltdown" CVE. This diff does not confer any immunity to that vulnerability - subsequent commits are still needed and are being worked on presently. Changes by: mlarkin@cvs.openbsd.org 2018/01/12 01:21:30 Log message: IBRS -> IBRS,IBPB in identifycpu lines Changes by: guenther@cvs.openbsd.org 2018/02/21 12:24:15 Log message: Meltdown: implement user/kernel page table separation. On Intel CPUs which speculate past user/supervisor page permission checks, use a separate page table for userspace with only the minimum of kernel code and data required for the transitions to/from the kernel (still marked as supervisor-only, of course): - the IDT (RO) - three pages of kernel text in the .kutext section for interrupt, trap, and syscall trampoline code (RX) - one page of kernel data in the .kudata section for TLB flush IPIs (RW) - the lapic page (RW, uncachable) - per CPU: one page for the TSS+GDT (RO) and one page for trampoline stacks (RW) When a syscall, trap, or interrupt takes a CPU from userspace to kernel the trampoline code switches page tables, switches stacks to the thread's real kernel stack, then copies over the necessary bits from the trampoline stack. On return to userspace the opposite occurs: recreate the iretq frame on the trampoline stack, switch stack, switch page tables, and return to userspace. mlarkin@ implemented the pmap bits and did 90% of the debugging, diagnosing issues on MP in particular, and drove the final push to completion. Many rounds of testing by naddy@, sthen@, and others Thanks to Alex Wilson from Joyent for early discussions about trampolines and their data requirements. Per-CPU page layout mostly inspired by DragonFlyBSD. Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:18:59 Log message: The GNU assembler does not understand 1ULL, so replace the constant with 1. Then it compiles with gcc, sign and size do not matter here. Changes by: bluhm@cvs.openbsd.org 2018/02/22 13:27:14 Log message: The compile time assertion for cpu info did not work with gcc. Rephrase the condition in a way that both gcc and clang accept it. Changes by: guenther@cvs.openbsd.org 2018/02/22 13:36:40 Log message: Set the PG_G (global) bit on the special page table entries that are shared between the u-k and u+k tables, because they're actually in all tables. OpenBSD 6.2 errata 009 ``` syspatch iXsystems a2k18 Hackathon Report: Ken Westerback on dhclient and more Ken Westerback (krw@) has sent in the first report from the (recently concluded) a2k18 hackathon: YYZ -> YVR -> MEL -> ZQN -> CHC -> DUD -> WLG -> AKL -> SYD -> BNE -> YVR -> YYZ For those of you who don’t speak Airport code: Toronto -> Vancouver -> Melbourne -> Queenstown -> Christchurch -> Dunedin Then: Dunedin -> Wellington -> Auckland -> Sydney -> Brisbane -> Vancouver -> Toronto ``` Whew. Once in Dunedin the hacking commenced. The background was a regular tick of new meltdown diffs to test in addition to whatever work one was actually engaged in. I was lucky (?) in that none of the problems with the various versions cropped up on my laptop. ``` ``` I worked with rpe@ and tb@ to make the install script create the 'correct' FQDN when dhclient was involved. I worked with tb@ on some code cleanup in various bits of the base. dhclient(8) got some nice cleanup, further pruning/improving log messages in particular. In addition the oddball -q option was flipped into the more normal -v. I.e. be quiet by default and verbose on request. More substantially the use of recorded leases was made less intrusive by avoiding continual reconfiguration of the interface with the same information. The 'request', 'require' and 'ignore' dhclient.conf(5) statement were changed so they are cumulative, making it easier to build longer lists of affected options. I tweaked softraid(4) to remove a handrolled version of duid_format(). I sprinkled a couple of M_WAITOK into amd64 and i386 mpbios to document that there is really no need to check for NULL being returned from some malloc() calls. I continued to help test the new filesystem quiescing logic that deraadt@ committed during the hackathon. I only locked myself out of my room once! Fueled by the excellent coffee from local institutions The Good Earth Cafe and The Good Oil Cafe, and the excellent hacking facilities and accommodations at the University of Otago it was another enjoyable and productive hackathon south of the equator. And I even saw penguins. Thanks to Jim Cheetham and the support from the project and the OpenBSD Foundation that made it all possible ``` Poetic License I found this when going through old documents. It looks like I wrote it and never posted it. Perhaps I didn’t consider it finished at the time. But looking at it now, I think it’s good enough to share. It’s a redrafting of the BSD licence, in poetic form. Maybe I had plans to do other licences one day; I can’t remember. I’ve interleaved it with the original license text so you can see how true, or otherwise, I’ve been to it. Enjoy :-) ``` Copyright (c) , All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: ``` You may redistribute and use – as source or binary, as you choose, and with some changes or without – this software; let there be no doubt. But you must meet conditions three, if in compliance you wish to be. 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. The first is obvious, of course – To keep this text within the source. The second is for binaries Place in the docs a copy, please. A moral lesson from this ode – Don’t strip the copyright on code. The third applies when you promote: You must not take, from us who wrote, our names and make it seem as true we like or love your version too. (Unless, of course, you contact us And get our written assensus.) THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. One final point to be laid out (You must forgive my need to shout): THERE IS NO WARRANTY FOR THIS WHATEVER THING MAY GO AMISS. EXPRESS, IMPLIED, IT’S ALL THE SAME – RESPONSIBILITY DISCLAIMED. WE ARE NOT LIABLE FOR LOSS NO MATTER HOW INCURRED THE COST THE TYPE OR STYLE OF DAMAGE DONE WHATE’ER THE LEGAL THEORY SPUN. THIS STILL REMAINS AS TRUE IF YOU INFORM US WHAT YOU PLAN TO DO. When all is told, we sum up thus – Do what you like, just don’t sue us. Beastie Bits AsiaBSDCon 2018 Videos The January/February 2018 FreeBSD Journal is Here Announcing the pkgsrc-2017Q4 release (2018-01-04) BSD Hamburg Event ZFS User conference Unreal Engine 4 Being Brought Natively To FreeBSD By Independent Developer Tarsnap ad Feedback/Questions Philippe - I heart FreeBSD and other questions Cyrus - BSD Now is excellent Architect - Combined Feedback Dale - ZFS on Linux moving to ZFS on FreeBSD Tommi - New BUG in Finland Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv
OpenBSD firewalling Windows 10, NetBSD’s return to ptrace, TCP Alternative Backoff, the BSD Poetic license, and AsiaBSDcon 2018 videos available.
OpenBSD firewalling Windows 10, NetBSD’s return to ptrace, TCP Alternative Backoff, the BSD Poetic license, and AsiaBSDcon 2018 videos available.
AsiaBSDcon review, Meltdown and Spectre Patches in FreeBSD stable, Interview with MidnightBSD founder, 8 months with TrueOS, mysteries of GNU and BSD split This episode was brought to you by Headlines AsiaBSDCon 2018 has concluded (https://2018.asiabsdcon.org/) We have just returned from AsiaBSDCon in Tokyo, Japan last weekend Please excuse our jetlag The conference consisted two days of meeting followed by 2 days of paper presentations We arrived a few days early to see some sights and take a few extra delicious meals in Tokyo The first day of meetings was a FreeBSD developer summit (while Benedict was teaching his two tutorials) where we discussed the FreeBSD release cycle and our thoughts on improving it, the new Casper capsicum helper service, and developments in SDIO which will eventually enable WiFi and SD card readers on more embedded devices The second day of meetings consisted of bhyvecon, a miniconf that covered development in all hypervisors on all BSDs. It also included presentations on the porting of bhyve to IllumOS. Then the conference started There were a number of great presentations, plus an amazing hallway track as usual It was great to see many old friends and to spend time discussing the latest happenings in BSD. A couple of people came by and asked to take a picture with us and we were happy to do that. *** FreeBSD releases Spectre and Meltdown mitigations for 11.1 (https://www.freebsd.org/security/advisories/FreeBSD-SA-18:03.speculative_execution.asc) Speculative execution vulnerability mitigation is a work in progress. This advisory addresses the most significant issues for FreeBSD 11.1 on amd64 CPUs. We expect to update this advisory to include 10.x for amd64 CPUs. Future FreeBSD releases will address this issue on i386 and other CPUs. freebsd-update will include changes on i386 as part of this update due to common code changes shared between amd64 and i386, however it contains no functional changes for i386 (in particular, it does not mitigate the issue on i386). Many modern processors have implementation issues that allow unprivileged attackers to bypass user-kernel or inter-process memory access restrictions by exploiting speculative execution and shared resources (for example, caches). An attacker may be able to read secret data from the kernel or from a process when executing untrusted code (for example, in a web browser). + Meltdown: The mitigation is known as Page Table Isolation (PTI). PTI largely separates kernel and user mode page tables, so that even during speculative execution most of the kernel's data is unmapped and not accessible. A demonstration of the Meltdown vulnerability is available at https://github.com/dag-erling/meltdown. A positive result is definitive (that is, the vulnerability exists with certainty). A negative result indicates either that the CPU is not affected, or that the test is not capable of demonstrating the issue on the CPU (and may need to be modified). A patched kernel will automatically enable PTI on Intel CPUs. The status can be checked via the vm.pmap.pti sysctl PTI introduces a performance regression. The observed performance loss is significant in microbenchmarks of system call overhead, but is much smaller for many real workloads. + Spectre V2: There are two common mitigations for Spectre V2. This patch includes a mitigation using Indirect Branch Restricted Speculation, a feature available via a microcode update from processor manufacturers. The alternate mitigation, Retpoline, is a feature available in newer compilers. The feasibility of applying Retpoline to stable branches and/or releases is under investigation. The patch includes the IBRS mitigation for Spectre V2. To use the mitigation the system must have an updated microcode; with older microcode a patched kernel will function without the mitigation. IBRS can be disabled via the hw.ibrsdisable sysctl (and tunable), and the status can be checked via the hw.ibrsactive sysctl. IBRS may be enabled or disabled at runtime. Additional detail on microcode updates will follow. + Wiki tracking the vulnerabilities and mitigations on different platforms (https://wiki.freebsd.org/SpeculativeExecutionVulnerabilities) Interview with MidnightBSD Founder and Lead Dev Lucas Holt (https://itsfoss.com/midnightbsd-founder-lucas-holt/) Recently, I have taken a little dip into the world of BSD. As part of my attempt to understand the BSD world a little better, I connected with Lucas Holt (MidnightBSD founder and lead developer) to ask him a few questions about his project. Here are his answers. It's FOSS: Please explain MidnightBSD in a nutshell. How is it different than other BSDs? Lucas Holt: MidnightBSD is a desktop focused operating system. When it's considered stable, it will provide a full desktop experience. This differs from other efforts such as TrueOS or GhostBSD in that it's not a distro of FreeBSD, but rather a fork. MidnightBSD has its own package manager, mport as well as unique package cluster software and several features built into user land such as mDNSresponder, libdispatch, and customizations throughout the system. It's FOSS: Who is MidnightBSD aimed at? Lucas Holt: The goal with MidnightBSD has always been to provide a desktop OS that's usable for everyday tasks and that even somewhat non technical people can use. Early versions of Mac OS X were certainly an inspiration. In practice, we're rather far from that goal at this point, but it's been an excellent learning opportunity. It's FOSS: What is your background in computers? Lucas Holt: I started in technical support at a small ISP and moved into web design and system administration. While there, I learned BSDi, Solaris and Linux. I also started tinkering with programming web apps in ASP and a little perl CGI. I then did a mix of programming and system administration jobs through college and graduated with a bachelors in C.S. from Eastern Michigan University. During that time, I learned NetBSD and FreeBSD. I started working on several projects such as porting Apple's HFS+ code to FreeBSD 6 and working on getting the nforce2 chipset SATA controller working with FreeBSD 6, with the latter getting committed. I got a real taste for BSD and after seeing the lack of interest in the community for desktop BSDs, I started MidnightBSD. I began work on it in late 2005. Currently, I'm a Senior Software Engineer focusing on backend rest services by day and a part-time graduate student at the University of Michigan Flint. It's FOSS: I recently installed TrueOS. I was disappointed that a couple of the programs I wanted were not available. The FreeBSD port system looked mildly complicated for beginners. I'm used to using pacman to get the job done quickly. How does MidnightBSD deal with ports? Lucas Holt: MidnightBSD has it's own port system, mports, which shared similarities with FreeBSD ports as well as some ideas from OpenBSD. We decided early on that decent package management was essential for regular users. Power users will still use ports for certain software, but it's just so time consuming to build everything. We started work on our own package manager, mport. Every package is a tar lzma archive with a sqlite3 manifest file as well as a sqlite 3 index that's downloaded from our server. This allows users to query and customize the package system with standard SQL queries. We're also building more user friendly graphical tools. Package availability is another issue that most BSDs have. Software tends to be written for one or two operating systems and many projects are reluctant to support other systems, particularly smaller projects like MidnightBSD. There are certainly gaps. All of the BSD projects need more volunteers to help with porting software and keeping it up to date. It's FOSS: During your June 2015 interview on BSDNow, you mentioned that even though you support both i386 and amd64, that you recommend people choose amd64. Do you have any plans to drop i386 support in the future, like many have done? Lucas Holt: Yes, we do plan to drop i386 support, mostly because of the extra work needed to build and maintain packages. I've held off on this so far because I had a lot of feedback from users in South America that they still needed it. For now, the plan is to keep i386 support through 1.0 release. That's probably a year or two out. It's FOSS: What desktop environments does MidnightBSD support? Lucas Holt: The original plan was to use Etoile as a desktop environment, but that project changed focus. We currently support Xfce, Gnome 3, WindowMaker + GNUstep + Gworkspace as primary choices. We also have several other window managers and desktop environments available such as Enlightenment, rat poison, afterstep, etc. Early versions offered KDE 3.x but we had some issues with KDE 4. We may revisit that with newer versions. It's FOSS: What is MidnightBSD's default filesystem? Do you support DragonflyBSD's HAMMER filesystem? What other filesystems? Lucas Holt: Boot volumes are UFS2. We also support ZFS for additional storage. We have read support for ExFat, NTFS, ext2, CD9660. NFS v3 and v4 are also supported for network file systems. We do not support HAMMER, although it was considered. I would love to see HAMMER2 get added to MidnightBSD eventually. It's FOSS: Is MidnightBSD affected by the recent Spectre and Meltdown issues? Lucas Holt: Yes. Most operating systems were affected by these issues. We were not informed of the issue until the general public became aware. Work is ongoing to come up with appropriate mitigations. Unfortunately, we do not have a patch yet. It's FOSS: The Raspberry Pi and its many clones have made the ARM platform very popular. Are there any plans to make MidnightBSD available on that platform? Lucas Holt: No immediate plans. ARM is an interesting architecture, but by the very nature of SoC designs, takes a lot of work to support a broad number of devices. It might be possible when we stop supporting i386 or if someone volunteers to work on the ARM port. Eventually, I think most hobby systems will need to run ARM chips. Intel's planning on locking down hardware with UEFI 3 and this may make it difficult to run on commodity hardware in the future not only for MidnightBSD but other systems as well. At one point, MidinightBSD ran on sparc64. When workstations were killed off, we dropped support. A desktop OS on a server platform makes little sense. It's FOSS: Does MidnightBSD offer support for Linux applications? Lucas Holt: Yes, we offer Linux emulation. It's emulating a 2.6.16 kernel currently and that needs to be updated so support newer apps. It's possible to run semi-recent versions of Firefox, Thunderbird, Java, and OpenOffice on it though. I've also used it to host game servers in the past and play older games such as Quake 3, enemy territory, etc. It's FOSS: Could you comment on the recent dust-up between the Pale Moon browser developers and the team behind the OpenBSD ports system? [Author's Note: For those who haven't heard about this, let me summarize. Last month, someone from the OpenBSD team added the Pale Moon browser to their ports collection. A Pale Moon developer demanded that they include Pale Moon's libraries instead of using system libraries. As the conversation continued, it got more hostile, especially on the Pale Moon side. The net result is that Pale Moon will not be available on OpenBSD, MidnightBSD, or FreeBSD.] Lucas Holt: I found this discussion frustrating. Many of the BSD projects hear a lot of complaints about browser availability and compatibility. With Firefox moving to Rust, it makes it even more difficult. Then you get into branding issues. Like Firefox, the Pale Moon developers have decided to protect their brand at the cost of users. Unlike the Firefox devs, they've made even stranger requirements for branding. It is not possible to use a system library version of anything with Pale Moon and keep their branding requirements. As such, we cannot offer Pale Moon in MidnightBSD. The reason this is an issue for an open source project is that many third party libraries are used in something as complex as a web browser. For instance, Gecko-based browsers use several multimedia libraries, sqlite3 (for bookmarks), audio and video codecs, etc. Trying to maintain upstream patches for each of these items is difficult. That's why the BSDs have ports collections to begin with. It allows us to track and manage custom patches to make all these libraries work. We go through a lot of effort in keeping these up to date. Sometimes upstream patches don't get included. That means our versions are the only working copies. With pale moon's policy, we'd need to submit separate patches to their customized versions of all these libraries too and any new release of the browser would not be available as changes occur. It might not even be possible to compile pale moon without a patch locally. With regard to Rust, it requires porting the language, as well as an appropriate version of LLVM before you can even start on the browser. It's FOSS: If someone wanted to contribute to your project, both financial and technical, how can they do that? Lucas Holt: Financial assistance for the project can be submitted online. We have a page outlining how to make donations with Patreon, Paypal or via bitcoin. Donations are not tax deductible. You can learn more at http://www.midnightbsd.org/donate/ We also need assistance with translations, porting applications, and working on the actual OS. Interested parties can contact us on the mailing list or through IRC on freenode #midnightbsd We also could use assistance with mirroring ISOs and packages. I would like to thank Lucas for taking the time to reply to my many questions. For more information about MidnightBSD or to download it, please visit their website. The most recent version of MidnightBSD is 0.8.6. News Roundup 8 months with TrueOS (https://inflo.ws/blog/post/2018-03-03-trueos-8th-month-review/) Purpose of this review - what it is and what it is not. I vowed to write down what I felt about TrueOS if I ever got to the six month mark of usage. This is just that. This is neither a tutorial, nor a piece of evangelism dedicated towards it. This is also not a review of specific parts of TrueOS such as Lumina or AppCafe, since I don't use them at all. In the spirit of presenting a screen shot, here is my i3wm displaying 4 windows in one screen - a configuration that I never use. https://inflo.ws/blog/images/trues-screenshot.png The primary tasks I get done with my computer. I need a tiling wm with multi-desktop capability. As regards what I do with a computer, it is fairly straightforward to describe if I just list down my most frequently used applications. xterm (CLI) Emacs (General editing and org mode) Intellij IDEA (Java, Kotlin, SQL) Firefox (Main web browser, with Multi-Account Containers) Thunderbird (Work e-mail) Notmuchmail (Personal e-mail) Chromium/Iridium (Dumb web browser) Telegram Desktop weechat (with wee-slack) cmus (Music player) mpv (Video player) mps-youtube (Youtube client) transmission-gtk Postgresql10 (daemon) Rabbitmq (daemon) Seafile (file sync) Shotwell (manage pictures) GIMP (Edit pictures) Calibre (Manage e-books) VirtualBox All of these are available as binary packages from the repository. Since I use Intellij Ultimate edition, I decided to download the no-jdk linux version from the website rather than install it. This would make sure that it gets updated regularly. Why did I pick TrueOS ? I ran various Linux distributions from 2001 all the way till 2009, till I discovered Arch, and continued with it till 2017. I tried out Void for two months before I switched to TrueOS. Over the last few years, I started feeling like no matter which Linux distribution I touched, they all just stopped making a lot of sense. Generally in the way things were organised, and particularly in terms of software like systemd, which just got pushed down my throat. I couldn't wrap my head around half the things going on in my computer. Mostly I found that Linux distributions stopped becoming a collection of applications that got developed together to something more coupled by software mechanisms like systemd - and that process was more and more opaque. I don't want to talk about the merits and de-merits of systemd, lets just say that I found it of no use and an unnecessary hassle. In February, I found myself in charge of the entire technology stack of a company, and I was free to make choices. A friend who was a long time FreeBSD user convinced me to try it on the servers. My requirement then was to run Postgres, Rabbitmq, Nginx and a couple of JVM processes. The setup was zero hassle and it hasn't changed much in a year. About three months of running FreeBSD-11.x on servers was enough for me to consider it for my laptop. I was very apprehensive of hardware support, but luckily my computer is a Thinkpad, and Thinkpads sort of work out of the box with various BSDs. My general requirements were: Must run Intellij IDEA. Must have proper graphics and sound driver support. Must be able to run VirtualBox. I had to pick from FreeBSD, NetBSD and OpenBSD, since these were the major BSDs that I was familiar with. One of my requirements was that I needed to be able to run VMs just in case I needed to test something on Windows/Linux. This ruled out OpenBSD. Then I was left with NetBSD and FreeBSD. NetBSD's driver support for newer Intel chip-sets were questionable, and FreeBSD was the only choice then. When I was digging through FreeBSD forums, I found out that running the 11.x RELEASE on my laptop was out of the question since it didn't have proper drivers for my chip-set either. A few more hours of digging led me to GhostBSD and TrueOS. I picked TrueOS straightaway because - well because TrueOS came from the old PC-BSD and it was built off FreeBSD-12-CURRENT with the latest drivers integrated. I downloaded the UNSTABLE version available in June 2017, backed up ALL my data and home directory, and then installed it. There were no glitches during installation - I simply followed the installation as described in the handbook and everything was fine. My entire switch from Arch/Void to TrueOS took about an hour, discounting the time it took to backup my data to an external hard disk. It was that easy. Everything I wanted to work just worked, everything was available in the repo. Tweaks from cooltrainer.org : I discovered this excellent tutorial that describes setting up a FreeBSD 11 desktop. It documents several useful tweaks, some of which I applied. A few examples - Fonts, VirtualBox, Firewall, UTF-8 sections. TrueOS (and FreeBSD) specific things I liked Open-rc The open-rc init system is familiar and is well documented. TrueOS specific parts are described here. When I installed postgresql10-server, there was no open-rc script for it, but I could cobble one together in two hours with zero prior experience writing init scripts. Later on I figured out that the init script for postgresql9 would work for 10 as well, and used that. Boot Environments This was an alien concept to me, but the first time I did an update without waiting for a CDN sync to finish, my computer booted into the shell and remained there. The friendly people at TrueOS discourse asked me to roll back to an older BE and wait for sync to finish. I dug through the forums and found "ZFS / Snapshots basics & How-To's for those new to TrueOS". This describes ZFS and BEs, and is well worth reading. ZFS My experience with boot environments was enough to convince me about the utility of ZFS. I am still reading about it and trying things out, and whatever I read just convinces me more about why it is good. File-system layout Coming from the Linux world, how the FreeBSD file-system is laid out seemed odd at first. Then I realised that it was the Linux distros that were doing the odd thing. e.g : The whole OS is split into base system and applications. All the non base system configurations and apps go into /usr/local. That made a lot of sense. The entire OS is developed along with its applications as a single coherent entity, and that shows. Documentation The handbooks for both TrueOS and FreeBSD are really really good. For e.g, I kept some files in an LUKS encrypted drive (when I used Arch Linux). To find an equivalent, all I had to do was read the handbook and look at the GELI section. It is actually nice being able to go to a source like Handbook and things from there just work. Arch Linux and Gentoo has excellent documentation as well, if anyone is wondering about Linux distros. Community The TrueOS community on both Telegram as well as on Discourse are very friendly and patient. They help out a lot and do not get upset when I pose really stupid questions. TrueOS core developers hangout in the Telegram chat-room too, and it is nice being able to talk to them directly about things. What did not work in TrueOS ? The following things that worked during my Linux tenure doesn't work in TrueOS. Netflix Google Hangouts Electron based applications (Slack, Skype) These are not major concerns for the kind of work I do, so it doesn't bother me much. I run a WinXP VM to play some old games, and a Bunsenlabs installation for Linux things like Hangouts/Netflix. I don't have a video calling system setup in TrueOS because I use my phone for both voice and video calls exclusively. Why am I staying on TrueOS ? Great community - whether on Discourse or on the telegram channel, the people make you feel welcome. If things go unanswered, someone will promise to work on it/file a bug/suggest work-arounds. Switching to TrueOS was philosophical as well - I thought a lot more about licenses, and I have arrived at the conclusion that I like BSD more than GPL. I believe it is a more practical license. I believe TrueOS is improving continuously, and is a great desktop UNIX if you put some time into it. AsiaBSDCon 2016 videos now available (https://www.youtube.com/playlist?list=PLnTFqpZk5ebD-FfVScL-x6ZnZSecMA1jI) The videos from AsiaBSDCon 2016 have been posted to youtube, 30 videos in all We'll cover the videos from 2017 next week The videos from 2018 should be posted in 4-6 weeks I are working on a new version of https://papers.freebsd.org/ that will make it easier to find the papers, slides, and videos of all talks related to FreeBSD *** syspatches will be provided for both supported releases (https://undeadly.org/cgi?action=article;sid=20180307234243) Good news for people doing upgrades only once per year: syspatches will be provided for both supported releases. The commit from T.J. Townsend (tj@) speaks for itself: ``` Subject: CVS: cvs.openbsd.org: www From: T.J. Townsend Date: 2018-03-06 22:09:12 CVSROOT: /cvs Module name: www Changes by: tj@cvs.openbsd.org 2018/03/06 15:09:12 Modified files: . : errata61.html stable.html faq : faq10.html Log message: syspatches will now be provided for both supported releases. ``` Thanks to all the developers involved in providing these! Update: An official announcement has been released: ``` I'm happy to announce that we are now able to provide two releases worth of syspatches on the amd64 and i386 platforms. The binary patches for 6.1 will hit the mirrors shortly, so you will be able to catch up with the errata on https://www.openbsd.org/errata61.html using the syspatch utility. People running amd64 will thus get the meltdown workaround. This means in particular that 6.2 will remain supported by syspatch when 6.3 comes out. Thanks to robert and ajacoutot for their amazing work on syspatch and for all their help. Thanks also to tj and the volunteers from #openbsd for their timely tests and of course to Theo for overseeing it all. ``` Exploring permutations and a mystery with BSD and GNU split filenames (https://www.lorainekv.com/permutations_split_and_gsplit/) Recently, I was playing around with the split command-line tool on Mac OS X, and I decided to chop a 4000-line file into 4000 separate single-line files. However, when I attempted to run split -l1, I ran into a funny error: split: too many files Curious to see if any splitting had occurred, I ran ls and sure enough, a huge list of filenames appeared, such as: xaa xab ... xzy xzz Now I could see why you'd run out of unique filenames - there are only 26 letters in the alphabet and these filenames were only three letters long. Also, they all seemed to begin with the letter "x". BSD split's filename defaults I checked the manual for split's defaults and confirmed what I was seeing: each file into which the file is split is named by the prefix followed by a lexically ordered suffix using suffix_length characters in the range 'a-z'. If -a is not specified, two letters are used as the suffix....with the prefix 'x' and with suffixes as above. Got it, so running split with the defaults for prefix name and suffix length will give me filenames that always start with the letter "x" followed by two-letter alphabetical permutations composed of a-z letters, with repeats allowed. I say "repeats allowed" because I noticed filenames such as xaa and xbb in the output. Side node: The reason why I say "permutations" rather than "combinations" is because letter order matters. For example, xab and xba are two distinct and legitimate filenames. Here's a nice explanation about the difference between permutations and combinations. Some permutation math So how many filenames can you get from the BSD split tool using the defaults? There are permutation formulas out there for repeating values and non-repeating values. Based on split's behavior, I wanted to use the repeating values formula: n^r where n equals the number of possible values (26 for a-z) and r equals the number of values (2, since there are only 2 letters after "x" in the filename). 26^2 = 676 So the total number of filename permutations allowed with BSD split's defaults should be 676. To double check, I ran ls | wc -l to get the total number of files in my split_test directory. The output was 677. If you subtract my original input file, input.txt, then you have 676, or the number of permutations split would allow before running out of filenames! Neat. But I still wanted my 4000 files. Moar permutations pls While 26^2 permutations doesn't support 4000 different filenames, I wondered if I could increase r to 3. Then, I'd have 17,576 different filename permutations to play with - more than enough. Earlier, I remembered the manual mentioning suffix length: -a suffixlength Use suffixlength letters to form the suffix of the file name. So I passed 3 in with the -a flag and guess what? I got my 4000 files! split -l1 -a3 input.txt ls | wc -l 4001 But that was a lot of work. It would be great if split would just handle these permutations and suffix lengths by default! In fact, I vaguely remember splitting large files into smaller ones with numerical filenames, which I prefer. I also remember not having to worry about suffixes in the past. But numerical filenames didn't seem to be an option with split installed on Mac OS X - there was no mention of it in the manual. Turns out that I was remembering GNU split from using the Debian OS two years ago, a different flavor of the split tool with different defaults and behaviors. Beastie Bits Michael Lucas is speaking at mug.org 10 April 2018 (https://blather.michaelwlucas.com/archives/3121) PkgsrcCon 2018 July 7+8 Berlin (http://pkgsrc.org/pkgsrcCon/2018/) Tint2 rocks (http://www.vincentdelft.be/post/post_20180310) Open Source Summit Europe 2018 Call for Proposals (https://www.freebsdfoundation.org/news-and-events/call-for-papers/open-source-summit-europe-2018-call-for-proposals/) Travel Grants for BSDCan 2018 (https://www.freebsdfoundation.org/blog/bsdcan-2018-travel-grant-application-now-open/) BSDCan 2018 FreeBSD Developers Summit Call for Proposals (https://www.freebsdfoundation.org/news-and-events/call-for-papers/bsdcan-2018-freebsd-developers-summit-call-for-proposals/) OpenBSD vmm(4) update, by Mike Larkin (https://www.openbsd.org/papers/asiabsdcon2018-vmm-slides.pdf) Feedback/Questions Morgan ZFS Install Question (http://dpaste.com/3NZN49P#wrap) Andre - Splitting ZFS Array, or not (http://dpaste.com/3V09BZ5#wrap) Jake - Python Projects (http://dpaste.com/2CY5MRE#wrap) Dave - Screen Sharing & Video Conference (http://dpaste.com/257WGCB#wrap) James - ZFS disk id switching (http://dpaste.com/3HAPZ90#wrap)
We explain the physics behind ZFS, DTrace switching to the GPL, Emacs debugging, syncookies coming to PF & FreeBSD's history on EC2. This episode was brought to you by Headlines 128 bit storage: Are you high? (https://blogs.oracle.com/bonwick/128-bit-storage:-are-you-high) For people who have heard about ZFS boiling oceans and wonder where that is coming from, we dug out this old piece from 2004 on the blog of ZFS co-creator Jeff Bonwick, originally from the Sun website. 64 bits would have been plenty ... but then you can't talk out of your ass about boiling oceans then, can you? Well, it's a fair question. Why did we make ZFS a 128-bit storage system? What on earth made us think it's necessary? And how do we know it's sufficient? Let's start with the easy one: how do we know it's necessary? Some customers already have datasets on the order of a petabyte, or 2^50 bytes. Thus the 64-bit capacity limit of 2^64 bytes is only 14 doublings away. Moore's Law for storage predicts that capacity will continue to double every 9-12 months, which means we'll start to hit the 64-bit limit in about a decade. Storage systems tend to live for several decades, so it would be foolish to create a new one without anticipating the needs that will surely arise within its projected lifetime. If 64 bits isn't enough, the next logical step is 128 bits. That's enough to survive Moore's Law until I'm dead, and after that, it's not my problem. But it does raise the question: what are the theoretical limits to storage capacity? Although we'd all like Moore's Law to continue forever, quantum mechanics imposes some fundamental limits on the computation rate and information capacity of any physical device. In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 10^51 operations per second on at most 10^31 bits of information [see Seth Lloyd, "Ultimate physical limits to computation." Nature 406, 1047-1054 (2000)]. A fully-populated 128-bit storage pool would contain 2^128 blocks = 2^137 bytes = 2^140 bits; therefore the minimum mass required to hold the bits would be (2^140 bits) / (10^31 bits/kg) = 136 billion kg. That's a lot of gear. To operate at the 1031 bits/kg limit, however, the entire mass of the computer must be in the form of pure energy. By E=mc^2, the rest energy of 136 billion kg is 1.2x1028 J. The mass of the oceans is about 1.4x1021 kg. It takes about 4,000 J to raise the temperature of 1 kg of water by 1 degree Celcius, and thus about 400,000 J to heat 1 kg of water from freezing to boiling. The latent heat of vaporization adds another 2 million J/kg. Thus the energy required to boil the oceans is about 2.4x106 J/kg * 1.4x1021 kg = 3.4x1027 J. Thus, fully populating a 128-bit storage pool would, literally, require more energy than boiling the oceans. Best part of all: you don't have to understand any of this to use ZFS. Rest assured that you won't hit any limits with that filesystem for a long time. You still have to buy bigger disks over time, though... *** dtrace for Linux, Oracle relicenses dtrace (https://gnu.wildebeest.org/blog/mjw/2018/02/14/dtrace-for-linux-oracle-does-the-right-thing/) At Fosdem we had a talk on dtrace for linux in the Debugging Tools devroom. Not explicitly mentioned in that talk, but certainly the most exciting thing, is that Oracle is doing a proper linux kernel port: ``` commit e1744f50ee9bc1978d41db7cc93bcf30687853e6 Author: Tomas Jedlicka tomas.jedlicka@oracle.com Date: Tue Aug 1 09:15:44 2017 -0400 dtrace: Integrate DTrace Modules into kernel proper This changeset integrates DTrace module sources into the main kernel source tree under the GPLv2 license. Sources have been moved to appropriate locations in the kernel tree. ``` That is right, dtrace dropped the CDDL and switched to the GPL! The user space code dtrace-utils and libdtrace-ctf (a combination of GPLv2 and UPL) can be found on the DTrace Project Source Control page. The NEWS file mentions the license switch (and that it is build upon elfutils, which I personally was pleased to find out). The kernel sources (GPLv2+ for the core kernel and UPL for the uapi) are slightly harder to find because they are inside the uek kernel source tree, but following the above commit you can easily get at the whole linux kernel dtrace directory. The UPL is the Universal Permissive License, which according to the FSF is a lax, non-copyleft license that is compatible with the GNU GPL. Thank you Oracle for making everyone's life easier by waving your magic relicensing wand! Now there is lots of hard work to do to actually properly integrate this. And I am sure there are a lot of technical hurdles when trying to get this upstreamed into the mainline kernel. But that is just hard work. Which we can now start collaborating on in earnest. Like systemtap and the Dynamic Probes (dprobes) before it, dtrace is a whole system observability tool combining tracing, profiling and probing/debugging techniques. Something the upstream linux kernel hackers don't always appreciate when presented as one large system. They prefer having separate small tweaks for tracing, profiling and probing which are mostly separate from each other. It took years for the various hooks, kprobes, uprobes, markers, etc. from systemtap (and other systems) to get upstream. But these days they are. And there is now even a byte code interpreter (eBPF) in the mainline kernel as originally envisioned by dprobes, which systemtap can now target through stapbpf. So with all those techniques now available in the linux kernel it will be exciting to see if dtrace for linux can unite them all. Debugging Emacs or: How I Learned to Stop Worrying and Love DTrace (http://nullprogram.com/blog/2018/01/17/) For some time Elfeed was experiencing a strange, spurious failure. Every so often users were seeing an error (spoiler warning) when updating feeds: “error in process sentinel: Search failed.” If you use Elfeed, you might have even seen this yourself. From the surface it appeared that curl, tasked with the responsibility for downloading feed data, was producing incomplete output despite reporting a successful run. Since the run was successful, Elfeed assumed certain data was in curl's output buffer, but, since it wasn't, it failed hard. Unfortunately this issue was not reproducible. Manually running curl outside of Emacs never revealed any issues. Asking Elfeed to retry fetching the feeds would work fine. The issue would only randomly rear its head when Elfeed was fetching many feeds in parallel, under stress. By the time the error was discovered, the curl process had exited and vital debugging information was lost. Considering that this was likely to be a bug in Emacs itself, there really wasn't a reliable way to capture the necessary debugging information from within Emacs Lisp. And, indeed, this later proved to be the case. A quick-and-dirty work around is to use condition-case to catch and swallow the error. When the bizarre issue shows up, rather than fail badly in front of the user, Elfeed could attempt to swallow the error — assuming it can be reliably detected — and treat the fetch as simply a failure. That didn't sit comfortably with me. Elfeed had done its due diligence checking for errors already. Someone was lying to Elfeed, and I intended to catch them with their pants on fire. Someday. I'd just need to witness the bug on one of my own machines. Elfeed is part of my daily routine, so surely I'd have to experience this issue myself someday. My plan was, should that day come, to run a modified Elfeed, instrumented to capture extra data. I would have also routinely run Emacs under GDB so that I could inspect the failure more deeply. For now I just had to wait to hunt that zebra. Bryan Cantrill, DTrace, and FreeBSD Over the holidays I re-discovered Bryan Cantrill, a systems software engineer who worked for Sun between 1996 and 2010, and is most well known for DTrace. My first exposure to him was in a BSD Now interview in 2015. I had re-watched that interview and decided there was a lot more I had to learn from him. He's become a personal hero to me. So I scoured the internet for more of his writing and talks. Some interesting operating system technology came out of Sun during its final 15 or so years — most notably DTrace and ZFS — and Bryan speaks about it passionately. Almost as a matter of luck, most of it survived the Oracle acquisition thanks to Sun releasing it as open source in just the nick of time. Otherwise it would have been lost forever. The scattered ex-Sun employees, still passionate about their prior work at Sun, along with some of their old customers have since picked up the pieces and kept going as a community under the name illumos. It's like an open source flotilla. Naturally I wanted to get my hands on this stuff to try it out for myself. Is it really as good as they say? Normally I stick to Linux, but it (generally) doesn't have these Sun technologies available. The main reason is license incompatibility. Sun released its code under the CDDL, which is incompatible with the GPL. Ubuntu does infamously include ZFS, but other distributions are unwilling to take that risk. Porting DTrace is a serious undertaking since it's got its fingers throughout the kernel, which also makes the licensing issues even more complicated. Linux has a reputation for Not Invented Here (NIH) syndrome, and these licensing issues certainly contribute to that. Rather than adopt ZFS and DTrace, they've been reinvented from scratch: btrfs instead of ZFS, and a slew of partial options instead of DTrace. Normally I'm most interested in system call tracing, and my go to is strace, though it certainly has its limitations — including this situation of debugging curl under Emacs. Another famous example of NIH is Linux's epoll(2), which is a broken version of BSD kqueue(2). So, if I want to try these for myself, I'll need to install a different operating system. I've dabbled with OmniOS, an OS built on illumos, in virtual machines, using it as an alien environment to test some of my software (e.g. enchive). OmniOS has a philosophy called Keep Your Software To Yourself (KYSTY), which is really just code for “we don't do packaging.” Honestly, you can't blame them since they're a tiny community. The best solution to this is probably pkgsrc, which is essentially a universal packaging system. Otherwise you're on your own. There's also openindiana, which is a more friendly desktop-oriented illumos distribution. Still, the short of it is that you're very much on your own when things don't work. The situation is like running Linux a couple decades ago, when it was still difficult to do. If you're interested in trying DTrace, the easiest option these days is probably FreeBSD. It's got a big, active community, thorough documentation, and a huge selection of packages. Its license (the BSD license, duh) is compatible with the CDDL, so both ZFS and DTrace have been ported to FreeBSD. What is DTrace? I've done all this talking but haven't yet described what DTrace really is. I won't pretend to write my own tutorial, but I'll provide enough information to follow along. DTrace is a tracing framework for debugging production systems in real time, both for the kernel and for applications. The “production systems” part means it's stable and safe — using DTrace won't put your system at risk of crashing or damaging data. The “real time” part means it has little impact on performance. You can use DTrace on live, active systems with little impact. Both of these core design principles are vital for troubleshooting those really tricky bugs that only show up in production. There are DTrace probes scattered all throughout the system: on system calls, scheduler events, networking events, process events, signals, virtual memory events, etc. Using a specialized language called D (unrelated to the general purpose programming language D), you can dynamically add behavior at these instrumentation points. Generally the behavior is to capture information, but it can also manipulate the event being traced. Each probe is fully identified by a 4-tuple delimited by colons: provider, module, function, and probe name. An empty element denotes a sort of wildcard. For example, syscall::open:entry is a probe at the beginning (i.e. “entry”) of open(2). syscall:::entry matches all system call entry probes. Unlike strace on Linux which monitors a specific process, DTrace applies to the entire system when active. To run curl under strace from Emacs, I'd have to modify Emacs' behavior to do so. With DTrace I can instrument every curl process without making a single change to Emacs, and with negligible impact to Emacs. That's a big deal. So, when it comes to this Elfeed issue, FreeBSD is much better poised for debugging the problem. All I have to do is catch it in the act. However, it's been months since that bug report and I'm not really making this connection yet. I'm just hoping I eventually find an interesting problem where I can apply DTrace. Bryan Cantrill: Talks I have given (http://dtrace.org/blogs/bmc/2018/02/03/talks/) *** News Roundup a2k18 Hackathon preview: Syncookies coming to PF (https://undeadly.org/cgi?action=article;sid=20180207090000) As you may have heard, the a2k18 hackathon is in progress. As can be seen from the commit messages, several items of goodness are being worked on. One eagerly anticipated item is the arrival of TCP syncookies (read: another important tool in your anti-DDoS toolset) in PF. Henning Brauer (henning@) added the code in a series of commits on February 6th, 2018, with this one containing the explanation: ``` syncookies for pf. when syncookies are on, pf will blindly answer each and every SYN with a syncookie-SYNACK. Upon reception of the ACK completing the 3WHS, pf will reconstruct the original SYN, shove it through pf_test, where state will be created if the ruleset permits it. Then massage the freshly created state (we won't see the SYNACK), set up the sequence number modulator, and call into the existing synproxy code to start the 3WHS with the backend host. Add an - somewhat basic for now - adaptive mode where syncookies get enabled if a certain percentage of the state table is filled up with half-open tcp connections. This makes pf firewalls resilient against large synflood attacks. syncookies are off by default until we gained more experience, considered experimental for now. see http://bulabula.org/papers/2017/bsdcan/ for more details. joint work with sashan@, widely discussed and with lots of input by many ``` The first release to have this feature available will probably be the upcoming OpenBSD 6.3 if a sufficient number of people test this in their setups (hint, hint). More info is likely to emerge soon in post-hackathon writeups, so watch this space! [Pale Moon] A Perfect example of how not to approach OS developers/packagers Removed from OpenBSD Ports due to Licensing Issues (https://github.com/jasperla/openbsd-wip/issues/86) FreeBSD Palemoon branding violation (https://lists.freebsd.org/pipermail/freebsd-ports/2018-February/112455.html) Mightnight BSD's response (https://twitter.com/midnightbsd/status/961232422091280386) *** FreeBSD EC2 History (http://www.daemonology.net/blog/2018-02-12-FreeBSD-EC2-history.html) A couple years ago Jeff Barr published a blog post with a timeline of EC2 instances. I thought at the time that I should write up a timeline of the FreeBSD/EC2 platform, but I didn't get around to it; but last week, as I prepared to ask for sponsorship for my work I decided that it was time to sit down and collect together the long history of how the platform has evolved and improved over the years. Normally I don't edit blog posts after publishing them (with the exception of occasional typographical corrections), but I do plan on keeping this post up to date with future developments. August 25, 2006: Amazon EC2 launches. It supports a single version of Ubuntu Linux; FreeBSD is not available. December 13, 2010: I manage to get FreeBSD running on EC2 t1.micro instances. March 22, 2011: I manage to get FreeBSD running on EC2 "cluster compute" instances. July 8, 2011: I get FreeBSD 8.2 running on all 64-bit EC2 instance types, by marking it as "Windows" in order to get access to Xen/HVM virtualization. (Unfortunately this meant that users had to pay the higher "Windows" hourly pricing.) January 16, 2012: I get FreeBSD 9.0 running on 32-bit EC2 instances via the same "defenestration" trick. (Again, paying the "Windows" prices.) August 16, 2012: I move the FreeBSD rc.d scripts which handle "EC2" functionality (e.g., logging SSH host keys to the console) into the FreeBSD ports tree. October 7, 2012: I rework the build process for FreeBSD 9.1-RC1 and later to use "world" bits extracted from the release ISOs; only the kernel is custom-built. Also, the default SSH user changes from "root" to "ec2-user". October 31, 2012: Amazon launches the "M3" family of instances, which support Xen/HVM without FreeBSD needing to pay the "Windows" tax. November 21, 2012: I get FreeBSD added to the AWS Marketplace. October 2, 2013: I finish merging kernel patches into the FreeBSD base system, and rework the AMI build (again) so that FreeBSD 10.0-ALPHA4 and later use bits extracted from the release ISOs for the entire system (world + kernel). FreeBSD Update can now be used for updating everything (because now FreeBSD/EC2 uses a GENERIC kernel). October 27, 2013: I add code to EC2 images so that FreeBSD 10.0-BETA2 and later AMIs will run FreeBSD Update when they first boot in order to download and install any critical updates. December 1, 2013: I add code to EC2 images so that FreeBSD 10.0-BETA4 and later AMIs bootstrap the pkg tool and install packages at boot time (by default, the "awscli" package). December 9, 2013: I add configinit to FreeBSD 10.0-RC1 and later to allow systems to be easily configured via EC2 user-data. July 1, 2014: Amazon launches the "T2" family of instances; now the most modern family for every type of EC2 instance (regular, high-memory, high-CPU, high-I/O, burstable) supports HVM and there should no longer be any need for FreeBSD users to pay the "Windows tax". November 24, 2014: I add code to FreeBSD 10.2 and later to automatically resize their root filesystems when they first boot; this means that a larger root disk can be specified at instance launch time and everything will work as expected. April 1, 2015: I integrate the FreeBSD/EC2 build process into the FreeBSD release building process; FreeBSD 10.2-BETA1 and later AMIs are built by the FreeBSD release engineering team. January 12, 2016: I enable Intel 82599-based "first generation EC2 Enhanced Networking" in FreeBSD 11.0 and later. June 9, 2016: I enable the new EC2 VGA console functionality in FreeBSD 11.0 and later. (The old serial console also continues to work.) June 24, 2016: Intel 82599-based Enhanced Networking works reliably in FreeBSD 11.0 and later thanks to discovering and working around a Xen bug. June 29, 2016: I improve throughput on Xen blkfront devices (/dev/xbd*) by enabling indirect segment I/Os in FreeBSD 10.4 and later. (I wrote this functionality in July 2015, but left it disabled by default a first because a bug in EC2 caused it to hurt performance on some instances.) July 7, 2016: I fix a bug in FreeBSD's virtual memory initialization in order to allow it to support boot with 128 CPUs; aka. FreeBSD 11.0 and later support the EC2 x1.32xlarge instance type. January 26, 2017: I change the default configuration in FreeBSD 11.1 and later to support EC2's IPv6 networking setup out of the box (once you flip all of the necessary switches to enable IPv6 in EC2 itself). May 20, 2017: In collaboration with Rick Macklem, I make FreeBSD 11.1 and later compatible with the Amazon "Elastic File System" (aka. NFSv4-as-a-service) via the newly added "oneopenown" mount option (and lots of bug fixes). May 25, 2017: I enable support for the Amazon "Elastic Network Adapter" in FreeBSD 11.1 and later. (The vast majority of the work — porting the driver code — was done by Semihalf with sponsorship from Amazon.) December 5, 2017: I change the default configuration in FreeBSD 11.2 and later to make use of the Amazon Time Sync Service (aka. NTP-as-a-service). The current status The upcoming FreeBSD release (11.2) supports: IPv6, Enhanced Networking (both generations), Amazon Elastic File System, Amazon Time Sync Service, both consoles (Serial VGA), and every EC2 instance type (although I'm not sure if FreeBSD has drivers to make use of the FPGA or GPU hardware on those instances). Colin's Patreon' page if you'd like to support him (https://www.patreon.com/cperciva) X network transparency X's network transparency has wound up mostly being a failure (https://utcc.utoronto.ca/~cks/space/blog/unix/XNetworkTransparencyFailure) I was recently reading Mark Dominus's entry about some X keyboard problems, in which he said in passing (quoting himself): I have been wondering for years if X's vaunted network transparency was as big a failure as it seemed: an interesting idea, worth trying out, but one that eventually turned out to be more trouble than it was worth. [...] My first reaction was to bristle, because I use X's network transparency all of the time at work. I have several programs to make it work very smoothly, and some core portions of my environment would be basically impossible without it. But there's a big qualification on my use of X's network transparency, namely that it's essentially all for text. When I occasionally go outside of this all-text environment of xterms and emacs and so on, it doesn't go as well. X's network transparency was not designed as 'it will run xterm well'; originally it was to be something that should let you run almost everything remotely, providing a full environment. Even apart from the practical issues covered in Daniel Stone's slide presentation, it's clear that it's been years since X could deliver a real first class environment over the network. You cannot operate with X over the network in the same way that you do locally. Trying to do so is painful and involves many things that either don't work at all or perform so badly that you don't want to use them. In my view, there are two things that did in general X network transparency. The first is that networks turned out to not be fast enough even for ordinary things that people wanted to do, at least not the way that X used them. The obvious case is web browsers; once the web moved to lots of images and worse, video, that was pretty much it, especially with 24-bit colour. (It's obviously not impossible to deliver video across the network with good performance, since YouTube and everyone else does it. But their video is highly encoded in specialized formats, not handled by any sort of general 'send successive images to the display' system.) The second is that the communication facilities that X provided were too narrow and limited. This forced people to go outside of them in order to do all sorts of things, starting with audio and moving on to things like DBus and other ways of coordinating environments, handling sophisticated configuration systems, modern fonts, and so on. When people designed these additional communication protocols, the result generally wasn't something that could be used over the network (especially not without a bunch of setup work that you had to do in addition to remote X). Basic X clients that use X properties for everything may be genuinely network transparent, but there are very few of those left these days. (Not even xterm is any more, at least if you use XFT fonts. XFT fonts are rendered in the client, and so different hosts may have different renderings of the same thing, cf.) < What remains of X's network transparency is still useful to some of us, but it's only a shadow of what the original design aimed for. I don't think it was a mistake for X to specifically design it in (to the extent that they did, which is less than you might think), and it did help X out pragmatically in the days of X terminals, but that's mostly it. (I continue to think that remote display protocols are useful in general, but I'm in an usual situation. Most people only ever interact with remote machines with either text mode SSH or a browser talking to a web server on the remote machine.) PS: The X protocol issues with synchronous requests that Daniel Stone talks about don't help the situation, but I think that even with those edges sanded off X's network transparency wouldn't be a success. Arguably X's protocol model committed a lesser version of part of the NeWS mistake. X's network transparency was basically free at the time (https://utcc.utoronto.ca/~cks/space/blog/unix/XFreeNetworkTransparency) I recently wrote an entry about how X's network transparency has wound up mostly being a failure for various reasons. However, there is an important flipside to the story of X's network transparency, and that is that X's network transparency was almost free at the time and in the context it was created. Unlike the situation today, in the beginning X did not have to give up lots of performance or other things in order to get network transparency. X originated in the mid 1980s and it was explicitly created to be portable across various Unixes, especially BSD-derived ones (because those were what universities were mostly using at that time). In the mid to late 1980s, Unix had very few IPC methods, especially portable ones. In particular, BSD systems did not have shared memory (it was called 'System V IPC' for the obvious reasons). BSD had TCP and Unix sockets, some System V machines had TCP (and you could likely assume that more would get it), and in general your safest bet was to assume some sort of abstract stream protocol and then allow for switchable concrete backends. Unsurprisingly, this is exactly what X did; the core protocol is defined as a bidirectional stream of bytes over an abstracted channel. (And the concrete implementation of $DISPLAY has always let you specify the transport mechanism, as well as allowing your local system to pick the best mechanism it has.) Once you've decided that your protocol has to run over abstracted streams, it's not that much more work to make it network transparent (TCP provides streams, after all). X could have refused to make the byte order of the stream clear or required the server and the client to have access to some shared files (eg for fonts), but I don't think either would have been a particularly big win. I'm sure that it took some extra effort and care to make X work across TCP from a different machine, but I don't think it took very much. (At the same time, my explanation here is probably a bit ahistorical. X's initial development seems relatively strongly tied to sometimes having clients on different machines than the display, which is not unreasonable for the era. But it doesn't hurt to get a feature that you want anyway for a low cost.) I believe it's important here that X was intended to be portable across different Unixes. If you don't care about portability and can get changes made to your Unix, you can do better (for example, you can add some sort of shared memory or process to process virtual memory transfer). I'm not sure how the 1980s versions of SunView worked, but I believe they were very SunOS dependent. Wikipedia says SunView was partly implemented in the kernel, which is certainly one way to both share memory and speed things up. PS: Sharing memory through mmap() and friends was years in the future at this point and required significant changes when it arrived. Beastie Bits Grace Hopper Celebration 2018 Call for Participation (https://www.freebsdfoundation.org/news-and-events/call-for-papers/grace-hopper-celebration-2018-call-for-participation/) Google Summer of Code: Call for Project Ideas (https://www.freebsdfoundation.org/blog/google-summer-of-code-call-for-project-ideas/) The OpenBSD Foundation 2018 Fundraising Campaign (https://undeadly.org/cgi?action=article;sid=20180129190641) SSH Mastery 2/e out (https://blather.michaelwlucas.com/archives/3115) AsiaBSDcon 2018 Registration is open (https://2018.asiabsdcon.org/) Tarsnap support for Bitcoin ending April 1st; and a Chrome bug (http://mail.tarsnap.com/tarsnap-announce/msg00042.html) Feedback/Questions Todd - Couple Questions (http://dpaste.com/195HGHY#wrap) Seth - Tar Snap (http://dpaste.com/1N7NQVQ#wrap) Alex - sudo question (http://dpaste.com/3D9P1DW#wrap) Thomas - FreeBSD on ARM? (http://dpaste.com/24NMG47#wrap) Albert - Austria BSD User Group (http://dpaste.com/373CRX7#wrap)
We provide you with updates to Spectre and Meltdown from various BSD projects, a review of TrueOS from Linux, how to set up FreeBSD on ThinkPad x240, and a whole bunch of beastie bits. This episode was brought to you by Headlines KPTI patch lands in FreeBSD -current (https://svnweb.freebsd.org/base?view=revision&revision=328083) After a heroic effort by Konstantin Belousov kib@FreeBSD.org, the first meltdown patch has landed in FreeBSD This creates separate page tables for the Kernel and userland, and switches between them when executions enters the kernel, and when it returns to userland It is currently off by default, but you are encouraged to test it, so it can be merged back to the release branches. Set vm.pmap.pti=1 in /boot/loader.conf The existing implementation of PCID (process-context identifiers), is not compatible with the new PTI code, and is disabled when PTI is enabled, decreasing performance. A future patch will use PCID in a way that is compatible with PTI. PCID allows the OS to annotate memory mappings to specific processes, so that they can be flushed selectively, and so that they are only used when in the context of that application. Once the developers are relatively confident in the correctness of the code that has landed in -current, it will be ported back to FreeBSD 10 and 11, and released as a security advisory. Apparently porting back to FreeBSD 11 only has some relatively simple merge conflicts, but 10 will be more work. Former FreeBSD Security Officer Dag-Erling Smørgrav has created a meltdown testing and PoC tool (https://github.com/dag-erling/meltdown) that you can use to check your system. It is not finished yet, and doesn't seem to work with newer processors (haswell and newer). The first partial mitigation for Spectre variant 2 (https://svnweb.freebsd.org/changeset/base/328011) for bhyve on AMD64 has also been committed The latest information is always available on the FreeBSD Wiki (https://wiki.freebsd.org/action/edit/SpeculativeExecutionVulnerabilities) *** Some thoughts on Spectre and Meltdown (http://www.daemonology.net/blog/2018-01-17-some-thoughts-on-spectre-and-meltdown.html) Colin Percival breaks down how these vulnerabilities work, with same nice analogies What is a side channel: I want to know when my girlfriend's passport expires, but she won't show me her passport (she complains that it has a horrible photo) and refuses to tell me the expiry date. I tell her that I'm going to take her to Europe on vacation in August and watch what happens: If she runs out to renew her passport, I know that it will expire before August; while if she doesn't get her passport renewed, I know that it will remain valid beyond that date. Her desire to ensure that her passport would be valid inadvertently revealed to me some information: Whether its expiry date was before or after August. Spectre Variant 1: I tell my girlfriend that I'm going to take her on vacation in June, but I don't tell her where yet; however, she knows that it will either be somewhere within Canada (for which she doesn't need a passport, since we live in Vancouver) or somewhere in Europe. She knows that it takes time to get a passport renewed, so she checks her passport and (if it was about to expire) gets it renewed just in case I later reveal that I'm going to take her to Europe. If I tell her later that I'm only taking her to Ottawa — well, she didn't need to renew her passport after all, but in the meantime her behaviour has already revealed to me whether her passport was about to expire. This is what Google refers to "variant 1" of the Spectre vulnerability: Even though she didn't need her passport, she made sure it was still valid just in case she was going to need it. Spectre Variant 2: I spend a week talking about how Oxford is a wonderful place to visit and I really enjoyed the years I spent there, and then I tell her that I want to take her on vacation. She very reasonably assumes that — since I've been talking about Oxford so much — I must be planning on taking her to England, and runs off to check her passport and potentially renew it... but in fact I tricked her and I'm only planning on taking her to Ottawa. Meltdown: I tell my girlfriend that I want to take her to the Korean peninsula. She knows that her passport is valid for long enough; but she immediately runs off to check that her North Korean visa hasn't expired. Why does she have a North Korean visa, you ask? Good question. She doesn't — but she runs off to check its expiry date anyway! Because she doesn't have a North Korean visa, she (somehow) checks the expiry date on someone else's North Korean visa, and then (if it is about to expire) runs out to renew it — and so by telling her that I want to take her to Korea for a vacation I find out something she couldn't have told me even if she wanted to. Final thoughts on vulnerability disclosure The way these issues were handled was a mess; frankly, I expected better of Google, I expected better of Intel, and I expected better of the Linux community. When I found that Hyper-Threading was easily exploitable, I spent five months notifying the security community and preparing everyone for my announcement of the vulnerability; but when the embargo ended at midnight UTC and FreeBSD published its advisory a few minutes later, the broader world was taken entirely by surprise. Nobody knew what was coming aside from the people who needed to know; and the people who needed to know had months of warning. Contrast that with what happened this time around. Google discovered a problem and reported it to Intel, AMD, and ARM on June 1st. Did they then go around contacting all of the operating systems which would need to work on fixes for this? Not even close. FreeBSD was notified the week before Christmas, over six months after the vulnerabilities were discovered. Now, FreeBSD can occasionally respond very quickly to security vulnerabilities, even when they arise at inconvenient times — on November 30th 2009 a vulnerability was reported at 22:12 UTC, and on December 1st I provided a patch at 01:20 UTC, barely over 3 hours later — but that was an extremely simple bug which needed only a few lines of code to fix; the Spectre and Meltdown issues are orders of magnitude more complex. To make things worse, the Linux community was notified and couldn't keep their mouths shut. Standard practice for multi-vendor advisories like this is that an embargo date is set, and nobody does anything publicly prior to that date. People don't publish advisories; they don't commit patches into their public source code repositories; and they definitely don't engage in arguments on public mailing lists about whether the patches are needed for different CPUs. As a result, despite an embargo date being set for January 9th, by January 4th anyone who cared knew about the issues and there was code being passed around on Twitter for exploiting them. This is not the first time I've seen people get sloppy with embargoes recently, but it's by far the worst case. As an industry we pride ourselves on the concept of responsible disclosure — ensuring that people are notified in time to prepare fixes before an issue is disclosed publicly — but in this case there was far too much disclosure and nowhere near enough responsibility. We can do better, and I sincerely hope that next time we do. CPU microcode update code for amd64 (https://undeadly.org/cgi?action=article;sid=20180115073406) (https://marc.info/?l=openbsd-tech&m=151588857304763&w=2) Patrick Wildt (patrick@) recently committed some code that will update the Intel microcode on many Intel CPUs, a diff initially written by Stefan Fritsch (sf@). The microcode of your CPU is basically the firmware that runs on your (Intel) processor, defining its instruction set in terms of so called "microinstructions". The new code depends, of course, on the corresponding firmware package, ported by Patrick which can be installed using a very recent fw_update(1). Of course, this all plays into the recently revealed problems in Intel (and other) CPUs, Meltdown and Spectre. Now Theo has explained the workings of the code on openbsd-tech, detailing some of the challenges in updating microcode on CPUs where your OS is already starting to run. Theo hints at future updates to the intel-firmware package in his mail: (https://marc.info/?l=openbsd-tech&m=151588857304763&w=2) Patrick and others committed amd64 Intel cpu microcode update code over the last few days. The approach isn't perfect, but it is good enough for a start. I want to explain the situation. When you fw_update, you'll get the firmware files. Upon a reboot, it will attempt to update the microcode on your cpus. Maybe there isn't a new microcode. Maybe your BIOS has a copy of the microcode and installs it before booting OpenBSD. This firmware installation is done a little late. Doing it better will require some work in the bootblocks to find the firmware files, but time is a bit short to do that right now. The branch-target-cache flushing features added in new microcode are not being used yet. There is more code which has to be written, but again other work is happening first. Also, Intel is saying their new microcodes sucks and people should wait a little. "Hi, my name is Intel and I'm an cheating speculator". Several developers are working on mitigations for these issues, attacking the problem from several angles. Expect to see more updates to a CVS tree near you soon. Intel: as a *BSD user, I am fucking pissed! (https://malcont.net/2018/01/dont-like-meltdown-spectre-releated-bugs-handled/) I wasn't going to write anything on the recently found x64 architecture – related bugs. I'm not a kernel developer nor even a programmer and I can't say that I have a solid understanding of what Meltdown and Spectre attacks are. Also there already is a ton of articles and posts written by people who have no grasp of the subject. I'm however a malcontent and I find this a good way to express my feelings: Intel: as a *BSD user, I am fucking pissed! Meltdown, Spectre and BSD – the “pissed” part Part of my work is UNIX-like systems administration – including BSDs and Linuces. As much as I am happy with Linux changes already made, I am beyond pissed about how the BSDs were handled by Intel – because they were not. FreeBSD Security Team received some heads-up just before Xmas, while OpenBSD, NetBSD and DragonflyBSD teams received no prior warnings. Meltdown and Spectre attacks are hard to perform. It is a hard work to mitigate them in the software, as the bugs lay in the CPUs and are not fixable by microcode updates. Developers are trying to mitigate these bugs in a way that will deliver smallest performance losses. A lot of time consuming work is needed to fix CPU vendors' mistakes. Linux developers had this time. BSD developers did not. BSD user base too small? BSD user base is small in comparison to Linux. Seems that it's too small for Intel. PlayStation4 consoles are FreeBSD-based (and use AMD CPUs) but I think it's safe to say that gaming devices are not the most important systems to be fixed. Netflix serves their content off FreeBSD but the bugs are not remotely exploitable (possibly not including JavaScript, but it's running someone's code locally) so there's probably not much harm to be done here either. However gamers and Netflix aren't the only ones who use *BSD systems. I'd say that there is more than a few FreeBSD, NetBSD, OpenBSD and DragonFlyBSD servers on the internet. In March 2017, Intel promised “more timely support to FreeBSD”. They knew about flaws in their CPUs in June and decided that a timely manner is the end of December – short before the embargo was to be lifted. Intel and Google (probably Intel more): it was your job to pick the correct people to whom the bugs can be disclosed. In my humble opinion you chose poorly by disclosing these issues with ONLY Apple, Microsoft, and the Linux Foundation, of OS vendors. You did much harm to the BSD community. Intel: It's your bugs. And you offered “more support” to the FreeBSD Foundation less than 3 months prior to being informed (my guess is that you knew much earlier) on the flaws in YOUR products. I don't want to write more here as the wording would be too strong. Interview - Viewer Questions These days, do you consider yourself more of an programmer or a sysadmin? Which one do you enjoy more? Does FreeBSD/BSD enable your business or would another OS suit your needs just as well? You've hinted that you use FreeBSD as part of your business. Can you elaborate on that and give some technical detail on how it's used in that environment? If you were allowed three wishes for anything at all to be implemented or changed in ZFS, what would they be, and why? Per Dataset throughput and IOPS limiting Per-File Cloning and/or zfsmv (move a file from one dataset to another, without copying) Cluster support Allan, you have previously mentioned that you have worked on FreeBSD on MIPS, what made you choose the Onion Omega over something like the Raspberry Pi? What is BSD Now's association with Jupiter broadcasting, and how did the relationship come to be? Jupiter seems to be associated with several Linux-themed podcasts, and I'm wondering how and why BSD Now joined Jupiter. The two communities (the Linuxes and BSDs) don't always seem to mix freely -- or do they? What kind of keyboard is that? Have you ever tried an ErgoDox? The ErgoDox EZ is made by a Canadian. You mentioned when doing one of your talks on UCL for FreeBSD that you had only recently learned C. I am also aware of your history also on contributing to the FreeBSD handbook and to documentation in general. Given you started with C relatively recently, what made you want to learn it, how quickly did you pick it up, and is it your favourite language? It is most inspiring to me, as you are clearly so talented, and of all the languages I have learned (including C++), I still prefer C in my heart of hearts. I'd be really interested to hear your answer, many thanks. *** News Roundup LinuxAndUbuntu Review Of TrueOS A Unix Based OS (http://www.linuxandubuntu.com/home/linuxandubuntu-review-of-trueos-a-unix-based-os) Trust me, the name TrueOS takes me back to 1990s when Tru64 UNIX operating system made its presence. TrueOS is PC-BSD's new unified brand built upon FreeBSD-CURRENT code base. Note that TrueOS is not a Linux distro but is BSD Unix. FreeBSD is known for its cutting-edge features, security, scalability, and ability to work both as a server and desktop operating system. TrueOS aims at having user-friendliness with the power of FreeBSD OS. Let us start with going into details of different aspects of the TrueOS. TrueOS History ? TrueOS was founded by Kris Moore in 2005 with name PC-BSD. Initial version focused to make FreeBSD easy to use starting with providing GUI based installer (to relatively complicated FreeBSD installer). In the year 2006, PC-BSD was acquired by iXsystems. Before rebranding as TrueOS in Sept 2016, PC-BSD reached a stage starting considering better than vanilla FreeBSD. Older PC-BSD version used to support both x86 and x86-64 architecture. Kris Moore, the developer founder, says about rebranding: “We've already been using TrueOS for the server side of PC-BSD, and it made sense to unify the names. PC-BSD doesn't reflect server or embedded well. TrueOS Desktop/Server/Embedded can be real products, avoids some of the alphabet soup, and gives us a more catchy name.” TrueOS First Impression ? The startup is little longer; may be due to starting up of many services. The heavy KDE well suited to PC-BSD. The C++/Qt5 based Lumina desktop environment is light and fast. The Lumina offers an easy way to configure menu and panels. I did not face any problems for continuous use of two weeks on a virtual machine having the minimal configuration: 1 GB RAM, 20 GB hard disk and Intel 3.06 GHz i3 processor. The Lumina desktop is light and fast. The developers of Lumina know what they are doing and have a good idea of what makes a good IDE. As it happens with any new desktop environment, it needs some time to settle. Let us hope that they keep to the path they are on with it. Conclusion ? The TrueOS is impressive when consider it as relatively young. It is a daring step that TrueOS developers took FreeBSD Current rather than FreeBSD Stable code base. Overall it has created its own place from the legacy shadow of PC-BSD. Starting with easy installation TrueOS is a good combination of software and utilities that make the system ready to use. Go and get a TrueOS ISO to unleash the “bleeding edge” tag of FreeBSD Thinkpad x240 - FreeBSD Setup (http://stygix.org/nix/x240-freebsd.php) What follows is a record of how I set up FreeBSD to be my daily driver OS on the Lenovo Thinkpad X240. Everything seems to work great. Although, the touchpad needs some tweaking. I've tried several configurations, even recompiling Xorg with EVDEV support and all that, to no avail. Eventually I will figure it out. Do not sleep the laptop from the command line. Do it from within Xorg, or it will not wake up. I don't know why. You can do it from a terminal within Xorg, just not from the naked command line without Xorg started. It also will not sleep by closing the lid. I included a sudo config that allows you to run /usr/sbin/zzz without a password, so what I do is I have a key combo assigned within i3wm to run "sudo /usr/sbin/zzz". It works fine this way. I go into detail when it comes to setting up Xorg with i3wm. You can skip this if you want, but if you've never used a tiling window manager, it will handle screen real estate very efficiently on a laptop with a 12.5-inch screen and a touchpad. First, download the amd64 image for 11.1-RELEASE and flash it to a USB pen drive. For the Unices, use this: # dd if=FreeBSD-11.1-RELEASE-amd64-memstick.img of=/dev/da0 bs=1M conv=sync Obviously, you'll change /dev/da0 to whatever the USB pen drive is assigned. Plug it in, check dmesg. Leave it plugged in, restart the laptop. When prompted, tap Enter to halt the boot process, then F12 to select a bootable device. Choose the USB drive. I won't go through the actual install process, but it is pretty damn easy so just look at a guide or two and you'll be fine. If you can install Debian, you can install FreeBSD. I will, however, recommend ZFS if you have over 4GB of RAM (my particular variant of the X240 has 8GB of RAM, so yours should have at least 4GB), along with an encrypted disk, and an encrypted SWAP partition. When prompted to add an additional user, and you get to the question where it asks for additional groups, please make sure you add the user to "wheel". The rest should be self-explanatory during the install. Now for the good shit. You just booted into a fresh FreeBSD install. Now what? Well, time to fire up vi and open some config files... CNN Article about CDROM.com and FreeBSD, from 1999 (https://www.cnn.com/TECH/computing/9904/08/cdrom.idg/index.html) Walnut Creek CDROM sells a lot of CD-ROMs, but it gives away even more data. Specifically, anyone who has Internet access is free to log into wcarchive (ftp.cdrom.com) and start downloading bits. Even with a good Internet connection, however, you should expect to be at it for a while. At the present time, wcarchive resides on half a terabyte (500 GB) of RAID 5-disk storage. Even if your 56-Kbps modem can deliver seven kilobytes per second, downloading the complete archive would take you 70 million seconds. Even then, some of the files would be more than two years out of date, so a bit of "back and fill" would be needed. Of course, nobody uses wcarchive that way. Instead, they just drop in when they need the odd file or two. The FTP server is very accommodating; 3,600 simultaneous download sessions is the current limit and an upgrade to 10,000 sessions is in the works. This translates to about 800 GB per day of downloads. Bob Bruce (Walnut Creek's founder) says he's thinking about issuing a press release when they reach a terabyte a day. But 800 GB isn't all that shabby.... The hardware Because FTP archives don't do a lot of thinking, wcarchive doesn't need a massive cluster of CPUs. In fact, it gets by with a single 200-MHz P6 Pentium Pro and a measly(!) 1 GB of RAM. The I/O support, however, is fairly impressive. A six-channel Mylex RAID controller (DAC960SXI; Ultra-Wide SCSI-SCSI) is the centerpiece of the I/O subsystem. Two channels link it to the PC ("Personal Computer"!?!), via a dual-channel Adaptec card (AHA-3940AUW; PCI to Ultra-Wide SCSI). An 256-MB internal cache helps it to eliminate recurring disk accesses. Four nine-drive disk arrays provide the actual storage. The two larger arrays use 18-GB IBM drives; the two smaller arrays use 9-GB Micropolis and Quantum drives. A separate 4-GB Quantum drive is used as the "system disk." The output side is handled by a single Intel 100Base-T controller (Pro/100B PCI), which feeds into the Internet through a number of shared DS3 (45 Mbps) and OC3 (155 Mbps) circuits. A detailed description of the system is available as ftp.cdrom.com/archive-info/configuration; The software The system software is rather prosaic: a copy of FreeBSD, supplemented by home-grown FTP mirroring and server code. Because of the massive hardware support, the software "only" needs to keep the I/O going in an efficient and reliable manner. FreeBSD, the "prosaic" operating system mentioned above, merits a bit more discussion. Like Linux, FreeBSD is open source. Anyone can examine, modify, and/or redistribute the source code. And, like Linux, an active user community helps the authors to find bugs, improve documentation, and generally support the OS. Unlike Linux, FreeBSD is derived from the Berkeley Unix code that forms the foundation for most commercial Unix variants. When you use the "fast file system" (cylinder groups, long file names, symbolic links, etc.), TCP/IP networking, termcap, or even vi, you are using Berkeley Unix additions. The version of BSD underlying FreeBSD, however, is "pure" BSD; don't look for the System V modifications you see in Solaris. Instead, think of it as SunOS, brought up to date with Kerberos, modern sendmail, an updated filesystem, and more. Solid, fast, and free! One of FreeBSD's finest innovations, the Ports Collection, makes FreeBSD a delight for open source application users. The Ports Collection automates the downloading, building, and installation (including de-installation) of 2,300+ open source packages. The company Walnut Creek CDROM has been around for several years now, so you are likely to be familiar with its offerings. You may not realize, however, that it provides the major financial support for FreeBSD. The FreeBSD support has two purposes. First, it provides the company with a solid base to run wcarchive and other massive projects. Second, it ties in with the company's mission of making software (and data) economically accessible. Bob Bruce, the firm's founder, is an interesting guy: laid back and somewhat conservative in manner, but productive and innovative in practice. Here is a possibly illustrative story. When Bob started selling CD-ROMs, disc caddies were selling for $15 each. Bob thought that was rather high, so he started investigating the marketplace. A long-distance call to Japan got him Sony's fax number; a series of faxes got him in touch with the salespeople. It turned out that caddies were available, in bulk, for only a few dollars each. Bulk, in this case, meant pallet-loads of 10,000 caddies. In an act of great faith, Bob purchased a pallet of caddies, then proceeded to sell them for five dollars each. The results were everything he might have wished. Folks who bought his CD-ROMs added caddies to their orders; folks who bought piles of caddies added in a disc or two. Either way, Walnut Creek CDROM was making a name for itself. Many pallet-loads later, the company is still selling caddies, making and distributing CD-ROMs, and giving away bits. Walnut Creek CDROM is a real open-source success story; its breadth and depth of offerings is well worth a look. Beastie Bits OpenBSD adds kqueue event support to DRM, to detect device changes like HDMI cables being plugged in, and trigger randr events (https://github.com/openbsd/src/commit/b8584f4233dc11a328cd245a5843ec3d67462200) Thesis describing QUAD3, a unix-like, multi-tasking operating system for the 6502 processor (https://archive.org/details/AMultiTaskingOperatingSystemForMicrocomputers) Windows is getting chmod and chown... (https://blogs.msdn.microsoft.com/commandline/2018/01/12/chmod-chown-wsl-improvements/) Timeline: How they kept Meltdown and Spectre secret for so long (https://www.theverge.com/platform/amp/2018/1/11/16878670/meltdown-spectre-disclosure-embargo-google-microsoft-linux) bsd.network is a *BSD-themed Mastodon Instance (https://bsd.network/): Peter Hessler is administering a new Mastodon instance, running in an OpenBSD VM on top of an OpenBSD vmm hypervisor Computer-Aided Instruction on UNIX (https://virtuallyfun.com/wordpress/wp-content/uploads/2017/12/whfUb.pdf) AsiaBSDCon 2018 Travel Grant Application Now Open (https://www.freebsdfoundation.org/blog/asiabsdcon-2018-travel-grant-application-now-open/) AsiaBSDCon 2018 FreeBSD Developers Summit Call for Proposals (https://www.freebsdfoundation.org/news-and-events/call-for-papers/asiabsdcon-2018-freebsd-developers-summit-call-for-proposals/) LinuxFest Northwest 2018 Call for Proposals (https://www.freebsdfoundation.org/news-and-events/call-for-papers/linuxfest-northwest-2018-call-for-proposals/) Feedback/Questions Jason - Dont break my ports (http://dpaste.com/05PRNG2) Wilyarti - show content (http://dpaste.com/1BG8GZW) https://clinetworking.wordpress.com/2017/12/08/data-de-duplication-file-diff-ing-and-s3-style-object-storage-using-digital-ocean-spaces Scott - Your show is Perfect! (http://dpaste.com/0KER8YE#wrap) Ken - Community Culture (http://dpaste.com/0WT8285#wrap)
We review the information about Spectre & Meltdown thus far, we look at NetBSD memory sanitizer progress, Postgres on ZFS & show you a bit about NomadBSD. This episode was brought to you by Headlines Meltdown Spectre Official Site (https://meltdownattack.com/) Kernel-memory-leaking Intel processor design flaw forces Linux, Windows redesign (https://www.theregister.co.uk/2018/01/02/intel_cpu_design_flaw/) Intel's official response (https://newsroom.intel.com/news/intel-responds-to-security-research-findings/) The Register mocks intels response with pithy annotations (https://www.theregister.co.uk/2018/01/04/intel_meltdown_spectre_bugs_the_registers_annotations/) Intel's Analysis PDF (https://newsroom.intel.com/wp-content/uploads/sites/11/2018/01/Intel-Analysis-of-Speculative-Execution-Side-Channels.pdf) XKCD (https://xkcd.com/1938/) Response from FreeBSD (https://lists.freebsd.org/pipermail/freebsd-security/2018-January/009719.html) FreeBSD's patch WIP (https://reviews.freebsd.org/D13797) Why Raspberry Pi isn't vulnerable to Spectre or Meltdown (https://www.raspberrypi.org/blog/why-raspberry-pi-isnt-vulnerable-to-spectre-or-meltdown/) Xen mitigation patches (https://lists.xenproject.org/archives/html/xen-devel/2018-01/msg00110.html) Overview of affected FreeBSD Platforms/Architectures (https://wiki.freebsd.org/SpeculativeExecutionVulnerabilities) Groff's response (https://twitter.com/GroffTheBSDGoat/status/949372300368867328) ##### We'll cover OpenBSD, NetBSD, and DragonflyBSD's responses in next weeks episode. *** ###The LLVM Memory Sanitizer support work in progress (https://blog.netbsd.org/tnf/entry/the_llvm_memory_sanitizer_support) > In the past 31 days, I've managed to get the core functionality of MSan to work. This is an uninitialized memory usage detector. MSan is a special sanitizer because it requires knowledge of every entry to the basesystem library and every entry to the kernel through public interfaces. This is mandatory in order to mark memory regions as initialized. Most of the work has been done directly for MSan. However, part of the work helped generic features in compiler-rt. Sanitizers > Changes in the sanitizer are listed below in chronological order. Almost all of the changes mentioned here landed upstream. A few small patches were reverted due to breaking non-NetBSD hosts and are rescheduled for further investigation. I maintain these patches locally and have moved on for now to work on the remaining features. NetBSD syscall hooks > I wrote a large patch (815kb!) adding support for NetBSD syscall hooks for use with sanitizers. NetBSD ioctl(2) hooks > Similar to the syscall hooks, there is need to handle every ioctl(2) call. I've created the needed patch, this time shorter - for less than 300kb. New patches still pending for upstream review > There are two corrections that I've created, and they are still pending upstream for review: Add MSan interceptor for fstat(2)](https://reviews.llvm.org/D41637) Correct the setitimer interceptor on NetBSD)](https://reviews.llvm.org/D41502) > I've got a few more local patches that require cleanup before submitting to review. NetBSD basesystem corrections Sanitizers in Go The MSan state as of today Solaris support in sanitizers > I've helped the Solaris team add basic support for Sanitizers (ASan, UBsan). This does not help NetBSD directly, however indirectly it improves the overall support for non-Linux hosts and helps to catch more Linuxisms in the code. Plan for the next milestone > I plan to continue the work on MSan and correct sanitizing of the NetBSD basesystem utilities. This mandates me to iterate over the basesystem libraries implementing the missing interceptors and correcting the current support of the existing ones. My milestone is to build all src/bin programs against Memory Sanitizer and when possible execute them cleanly. This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: http://netbsd.org/donations/#how-to-donate (http://netbsd.org/donations/#how-to-donate) *** ##News Roundup ###MWL's 2017 Wrap-Up (https://blather.michaelwlucas.com/archives/3078) > The obvious place to start is my 2016 wrap-up post](https://blather.michaelwlucas.com/archives/2822), where I listed goals for 2017. As usual, these goals were wildly delusional. > The short answer is, my iron was back up to normal. My writing speed wasn't, though. I'd lost too much general health, and needed hard exercise to recover it. Yes, writing requires physical endurance. Maintaining that level of concentration for several hours a day demands a certain level of blood flow to the brain. I could have faked it in a day job, but when self-employed as an artist? Not so much. > Then there's travel. I did my usual BSDCan trip, plus two educational trips to Lincoln City, Oregon. The current political mayhem convinced me that if I wanted to hit EuroBSDCon any time in the next few years, I should do it in the very near future. So I went to Paris, where I promptly got pickpocketed. (Thankfully, they didn't get my passport.) I was actively writing the third edition of Absolute FreeBSD, so I visited BSDCam in Cambridge to get the latest information and a sense of where FreeBSD was going. I also did weekends at Kansas LinuxFest (because they asked and paid for my trip) and Penguicon. > (Because people will ask: why EuroBSDCon and not AsiaBSDCon? A six-hour transatlantic flight requires that I take a substantial dose of heavy-grade tranquilizers. I'm incapable of making intelligent decisions while on those drugs, or for several hours afterward. They don't last long enough for twelve-hour flight to Japan, so I need to be accompanied by someone qualified to tell me when I need to take the next dose partway through the flight. This isn't a predetermined time that I can set an alarm for; it depends on how the clonazepam affects me at those altitudes. A drug overdose while flying over the North Pole would be bad. When I can arrange that qualified companion, I'll make the trip.) > I need most of the preceding week to prepare for long trips. I need the following week to recover from time shifts and general exhaustion. Additionally, I have to hoard people juice for a few weeks beforehand so I can deal with folks during these expeditions. Travel disrupts my dojo time as well, which impacts my health. > Taken as a whole: I didn't get nearly as much done as I hoped. I wrote more stories, but Kris Rusch bludgeoned me into submitting them to trad markets. (The woman is a brute, I tell you. Cross her at your peril.) Among my 2017 titles, my fiction outsold the tech books. No, not Prohibition Orcs–all four of the people who buy those love them, but the sales tell me I've done something wrong with those tales. My cozy mystery git commit murder outsold Relayd and Httpd Mastery. But what outdid them both, as well as most of my older books? What title utterly dominated my sales for the last quarter of the year? It was of course, my open source software political satire disguised as porn Savaged by Systemd: an Erotic Unix Encounter. (https://www.michaelwarrenlucas.com/index.php/romance#sbs) > I can't believe I just wrote that paragraph. The good news is, once I recovered from EuroBSDCon, my writing got better. I finished Absolute FreeBSD, 3rd edition and submitted it to the publisher. I wrote the second edition of SSH Mastery (no link, because you can't order it yet.) I'm plowing through git sync murder, the sequel to git commit murder. I don't get to see the new Star Wars movie until I finish GSM, so hopefully that'll be this month. All in all, I wrote 480,200 words in 2017. Most of that was after September. It's annoyingly close to breaking half a million, but after 2016's scandalous 195,700, I'll take it. *** ###PG Phriday: Postgres on ZFS (https://blog.2ndquadrant.com/pg-phriday-postgres-zfs/) > ZFS is a filesystem originally created by Sun Microsystems, and has been available for BSD over a decade. While Postgres will run just fine on BSD, most Postgres installations are historically Linux-based systems. ZFS on Linux has had much more of a rocky road to integration due to perceived license incompatibilities. > As a consequence, administrators were reluctant or outright refused to run ZFS on their Linux clusters. It wasn't until OpenZFS was introduced in 2013 that this slowly began to change. These days, ZFS and Linux are starting to become more integrated, and Canonical of Ubuntu fame even announced direct support for ZFS in their 16.04 LTS release. > So how can a relatively obscure filesystem designed by a now-defunct hardware and software company help Postgres? Let's find out! Eddie waited til he finished high school > Old server hardware is dirt cheap these days, and make for a perfect lab for testing suspicious configurations. This is the server we'll be using for these tests for those following along at home, or want some point of reference: Dell R710 x2 Intel X5660 CPUs, for up to 24 threads 64GB RAM x4 1TB 7200RPM SATA HDDs H200 RAID card configured for Host Bus Adapter (HBA) mode 250GB Samsung 850 EVO SSD > The H200 is particularly important, as ZFS acts as its own RAID system. It also has its own checksumming and other algorithms that don't like RAID cards getting in the way. As such, we put the card itself in a mode that facilitates this use case. > Due to that, we lose out on any battery-backed write cache the RAID card might offer. To make up for it, it's fairly common to use an SSD or other persistent fast storage to act both as a write cache, and a read cache. This also transforms our HDDs into hybrid storage automatically, which is a huge performance boost on a budget. She had a guitar and she taught him some chords > First things first: we need a filesystem. This hardware has four 1TB HDDs, and a 250GB SSD. To keep this article from being too long, we've already placed GPT partition tables on all the HDDs, and split the SSD into 50GB for the OS, 32GB for the write cache, and 150GB for the read cache. A more robust setup would probably use separate SSDs or a mirrored pair for these, but labs are fair game. They moved into a place they both could afford > Let's start by getting a performance baseline for the hardware. We might expect peak performance at 12 or 24 threads because the server has 12 real CPUs and 24 threads, but query throughput actually topped out at concurrent 32 processes. We can scratch our heads over this later, for now, we can consider it the maximum capabilities of this hardware. Here's a small sample: ``` $> pgbench -S -j 32 -c 32 -M prepared -T 20 pgbench ... tps = 264661.135288 (including connections establishing) tps = 264849.345595 (excluding connections establishing) ``` So far, this is pretty standard behavior. 260k prepared queries per second is great read performance, but this is supposed to be a filesystem demonstration. Let's get ZFS involved. + The papers said Ed always played from the heart Let's repeat that same test with writes enabled. Once that happens, filesystem syncs, dirty pages, WAL overhead, and other things should drastically reduce overall throughput. That's an expected result, but how much are we looking at, here? ``` $> pgbench -j 32 -c 32 -M prepared -T 10 pgbench ... tps = 6153.877658 (including connections establishing) tps = 6162.392166 (excluding connections establishing) ``` SSD cache or not, storage overhead is a painful reality. Still, 6000 TPS with writes enabled is a great result for this hardware. Or is it? Can we actually do better? Consider the Postgres fullpagewrites parameter. Tomas Vondra has written about it in the past as a necessity to prevent WAL corruption due to partial writes. The WAL is both streaming replication and crash recovery, so its integrity is of utmost importance. As a result, this is one parameter almost everyone should leave alone. ZFS is Copy on Write (CoW). As a result, it's not possible to have a torn page because a page can't be partially written without reverting to the previous copy. This means we can actually turn off fullpagewrites in the Postgres config. The results are some fairly startling performance gains: $> pgbench -j 32 -c 32 -M prepared -T 10 pgbench tps = 10325.200812 (including connections establishing) tps = 10336.807218 (excluding connections establishing) That's nearly a 70% improvement. Due to write amplification caused by full page writes, Postgres produced 1.2GB of WAL files during a 1-minute pgbench test, but only 160MB with full page writes disabled. To be fair, a 32-thread pgbench write test is extremely abusive and certainly not a typical usage scenario. However, ZFS just ensured our storage a much lower write load by altering one single parameter. That means the capabilities of the hardware have also been extended to higher write workloads as IO bandwidth is not being consumed by WAL traffic. + They both met movie stars, partied and mingled Astute readers may have noticed we didn't change the default ZFS block size from 128k to align with the Postgres default of 8kb. As it turns out, the 128kb blocks allow ZFS to better combine some of those 8kb Postgres pages to save space. That will allow our measly 2TB to go a lot further than is otherwise possible. Please note that this is not de-duplication, but simple lz4 compression, which is nearly real-time in terms of CPU overhead. De-duplication on ZFS is currently an uncertain bizzaro universe populated with misshapen horrors crawling along a broken landscape. It's a world of extreme memory overhead for de-duplication tables, and potential lost data due to inherent conflicts with the CoW underpinnings. Please don't use it, let anyone else use it, or even think about using it, ever. + They made a record and it went in the chart We're still not done. One important aspect of ZFS as a CoW filesystem, is that it has integrated snapshots. Consider the scenario where a dev is connected to the wrong system and drops what they think is a table in a QA environment. It turns out they were in the wrong terminal and just erased a critical production table, and now everyone is frantic. + The future was wide open It's difficult to discount an immediately observable reduction in write overhead. Snapshots have a multitude of accepted and potential use cases, as well. In addition to online low-overhead compression, and the hybrid cache layer, ZFS boasts a plethora of features we didn't explore. Built-in checksums with integrated self-healing suggest it isn't entirely necessary to re-initialize an existing Postgres instance to enable checksums. The filesystem itself ensures checksums are validated and correct, especially if we have more than one drive resource in our pool. It even goes the extra mile and actively corrects inconsistencies when encountered. I immediately discounted ZFS back in 2012 because the company I worked for at the time was a pure Linux shop. ZFS was only available using the FUSE driver back then, meaning ZFS only worked through userspace with no real kernel integration. It was fun to tinker with, but nobody sane would use that on a production server of any description. Things have changed quite drastically since then. I've stopped waiting for btrfs to become viable, and ZFS has probably taken the throne away from XFS as my filesystem of choice. Future editions of the Postgres High Availability Cookbook will reflect this as well. Postgres MVCC and ZFS CoW seem made for each other. I'm curious to see what will transpire over the next few years now that ZFS has reached mainstream acceptance in at least one major Linux distribution. NomadBSD (https://github.com/mrclksr/NomadBSD) About NomadBSD is a live system for flash drives, based on FreeBSD. Screenshots http://freeshell.de/~mk/download/nomadbsd-ss1.png http://freeshell.de/~mk/download/nomadbsd-ss2.png Requirements for building the image A recent FreeBSD system Requirements for running NomadBSD A 4GB (or more) flash drive A System capable running FreeBSD 11.1 (amd64) Building the image ~~ csh # make image ~~ Writing the image to an USB memory stick ~~ csh # dd if=nomadbsd.img of=/dev/da0 bs=10240 conv=sync ~~ Resize filesystem to use the entire USB memory Boot NomadBSD into single user mode, and execute: ~~ # gpart delete -i 2 da0s1 # gpart resize -i 1 da0 # gpart commit da0s1 ~~ Determine the partition size in megabytes using fdisk da0 and calculate the remaining size of da0s1a: = - . ~~ # gpart resize -i 1 -s M da0s1 # gpart add -t freebsd-swap -i 2 da0s1 # glabel label NomadBSDsw da0s1b # service growfs onestart # reboot ~~ FreeBSD forum thread (https://forums.freebsd.org/threads/63888/) A short screen capture video of the NomadBSD system running in VirtualBox (https://freeshell.de/~mk/download/nomad_capture.mp4) *** ##Beastie Bits Coolpkg, a package manager inspired by Nix for OpenBSD (https://github.com/andrewchambers/coolpkg) zrepl - ZFS replication (https://zrepl.github.io/) OpenBSD hotplugd automount script (https://bijanebrahimi.github.io/blog/openbsd-hotplugd-scripting.html) Ancient troff sources vs. modern-day groff (https://virtuallyfun.com/2017/12/22/learn-ancient-troff-sources-vs-modern-day-groff/) Paypal donation balance and status.. thanks everyone! (http://lists.dragonflybsd.org/pipermail/users/2017-December/313752.html) Supervised FreeBSD rc.d script for a Go daemon (updated in last few days) (https://redbyte.eu/en/blog/supervised-freebsd-init-script-for-go-deamon/) A Brief History of sed (https://blog.sourcerer.io/a-brief-history-of-sed-6eaf00302ed) Flamegraph: Why does my AWS instance boot so slow? (http://www.daemonology.net/timestamping/tslog-c5.4xlarge.svg) *** ##Feedback/Questions Jeremy - Replacing Drive in a Zpool (http://dpaste.com/319593M#wrap) Dan's Blog (https://dan.langille.org/2017/08/16/swapping-5tb-in-3tb-out/) Tim - Keeping GELI key through reboot (http://dpaste.com/11QTA06) Brian - Mixing 2.5 and 3.5 drives (http://dpaste.com/2JQVD10#wrap) Troels - zfs swap on FreeBSD (http://dpaste.com/147WAFR#wrap) ***
We walk through dumping a PS4 kernel in only 6 days, tell you the news that NetBSD 7.1.1 has been released, details on how to run FreeBSD on a Thinkpad T470, and there's progress in OpenBSD's pledge. This episode was brought to you by Headlines NetBSD 7.1.1 released (http://www.netbsd.org/releases/formal-7/NetBSD-7.1.1.html) The NetBSD Project is pleased to announce NetBSD 7.1.1, the first security/critical update of the NetBSD 7.1 release branch. It represents a selected subset of fixes deemed important for security or stability reasons. Complete source and binaries for NetBSD 7.1.1 are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, and other services may be found at https://www.NetBSD.org/mirrors/. We encourage users who wish to install via ISO or USB disk images to download via BitTorrent by using the torrent files supplied in the images area. A list of hashes for the NetBSD 7.1.1 distribution has been signed with the well-connected PGP key for the NetBSD Security Officer: https://ftp.NetBSD.org/pub/NetBSD/security/hashes/NetBSD-7.1.1_hashes.asc NetBSD is free. All of the code is under non-restrictive licenses, and may be used without paying royalties to anyone. Free support services are available via our mailing lists and website. Commercial support is available from a variety of sources. More extensive information on NetBSD is available from our website: NetBSD website (www.NetBSD.org) +Changes Between 7.1 and 7.1.1 Below is an abbreviated list of changes in this release. The complete list can be found in the CHANGES-7.1.1 file in the top level directory of the NetBSD 7.1.1 release tree. Security Advisory Fixes The following security advisories were fixed: NetBSD-SA2017-004 buffer overflow via cmap for 4 graphics drivers. NetBSD-SA2017-005 x86: vulnerabilities in context handling. NetBSD-SA2017-006 Vnode reference leak in the openat system call. NetBSD-SA2018-001 Several vulnerabilities in context handling NetBSD-SA2018-002 Local DoS in virecover Note: Advisories prior to NetBSD-SA2017-004 do not affect NetBSD 7.1.1. Userland changes dhcrelay(8): Fix bug that prevented proper operation when run in the background. Heimdal: Update to 7.1. Fix CVE-2017-11103. mtree(8): Don't modify strings stored in hash, otherwise filling up of directory hierarchy stops if the same hash value occurs in directory and leaf. ping(8): Fix cksum calculation for clearing the cached route. resize_ffs(8): Fix numerous overflow errors which can lead to superblock corruption on large filesystems. rtadvd(8): Fix the default value of rltime. PR bin/51994. Update BIND to 9.10.5-P2. Update expat to 2.2.1. Update ntp to 4.2.8p10. Update root.cache to 2017102400. Update tzdata to 2017c. vi(1): Don't garble display when when resizing nvi in xterm. wpa_supplicant/hostapd: Update to 2.6. Apply fixes for CVEs 2017-13077 through 2017-13082 and CVEs 2017-13086 through 2017-13088. X: Apply fixes for CVEs 2017-12176 through 2017-12187, 2017-10971, 2017-10972, 2017-13722, 2017-13720, 2017-16611, and 2017-16612. *** ###Dumping a PS4 Kernel in "Only" 6 Days (https://fail0verflow.com/blog/2017/ps4-crashdump-dump/) > What if a secure device had an attacker-viewable crashdump format? What if that same device allowed putting arbitrary memory into the crashdump? Amazingly, the ps4 tempted fate by supporting both of these features! Let's see how that turned out… Crashdumps on PS4 The crash handling infrastructure of the ps4 kernel is interesting for 2 main reasons: It is ps4-specific code (likely to be buggy) If the crashdump can be decoded, we will gain very useful info for finding bugs and creating reliable exploits On a normal FreeBSD system, a kernel panic will create a dump by calling kernreboot with the RBDUMP flag. This then leads to doadump being called, which will dump a rather tiny amount of information about the kernel image itself to some storage device. On ps4, the replacement for doadump is mdbgrundump, which can be called from panic or directly from trapfatal. The amount of information stored into the dump is gigantic by comparison - kernel state for all process, thread, and vm objects are included, along with some metadata about loaded libraries. Other obvious changes from the vanilla FreeBSD method are that the mdbgrun_dump encodes data recorded into the dump on a field-by-field basis and additionally encrypts the resulting buffer before finally storing it to disk. Dumping Anything Let's zoom in to a special part of mdbgrundump - where it iterates over all process' threads and tries to dump some pthread state: dumpstate is a temporary buffer which will eventually make it into the crashdump. To summarize, sysdump_internalcallreaduser can be made to function as a read-anywhere oracle. This is because fsbase will point into our (owned) webkit process' usermode address space. Thus, even without changing the actual fsbase value, we may freely change the value of tcbthread, which is stored at fsbase + 0x10. Further, sysdump_internalcall_readuser will happily read from a kernel address and put the result into the dump. We can now put any kernel location into the dump, but we still need to decrypt and decode it… Aside from that, there's also the issue that we may only add 0x10 bytes per thread in this manner… Further reading: Crashdump Crypto Crashdump Decoding Crashdump Automation Triggering the Vulnerability The Fix (Kind of…) Fin Appendix Crashdump Decryptor NXDP Decoder *** ###BSDTW 2017 Conference Recap: Li-Wen Hsu (https://www.freebsdfoundation.org/blog/bsdtw-2017-conference-recap-li-wen-hsu/) BSDTW 2017 Conference Recap: Li-Wen Hsu 12/28/2017 > Last month, we held BSDTW 2017 on November 11-12th, 2017 in Taipei, Taiwan. It was the second largest BSD conference in Taiwan and the first one in this decade. In 2004, the first AsiaBSDCon was also held in Taipei. Then all of the following AsiaBSDCon conferences were held in Tokyo, Japan. (AsiaBSDCon 2018 will be in Tokyo again next year, please submit your talk proposal by December 31th 2017, and attend the conference on March 8th-11th) > We wanted to start small with the first BSDTW because we were not sure how much sponsorship or how many volunteers we might have. BSDTW 2017 was a single track, two-day conference with 11 selected 50 minute presentations and 1 WIP/lightning talk session consisting of 8 short talks. I do regret that we did not have any local presenters this year. It is also a similar problem at AsiaBSDCon. Unsurprisingly, as with AsiaBSDcon, the travel reimbursement took up a large part of the whole conference budget. We do have many good people that work in Asia, but we still need to encourage people to present their work more. > We had over 130 registered attendees, with 30% of them coming from outside of Taiwan. To our knowledge, in recent years, this is the only open source conference in Taiwan to be held entirely in English, and to have such a large portion of international attendees. This is also the first open source conference in Taiwan to focus entirely on operating systems. The attendees included students, professors, engineers or CTOs, and CEOs from technology companies. This is also the first time that GroffTheBSDGoat visited Taiwan! We were surprised that after the silence for so many years, there are still so many people that use and love BSD near us. We saw many old friends, who had “disappeared” for a long time, came back, and were glad to meet many new friends at the conference. I am really happy that this conference was able to bring together these people, from local and abroad. After attending BSD conferences around the world for many years, I feel that the friendship between BSD users is the most important thing in the BSD community, and one of the main reasons people stay. It has been my pleasure to bring this community back to my friends in my homeland. > After the two-day event, I truly understand that bootstrapping a new conference is a very hard job. One with many aspects that you don't even imagine until you're really in the process of planning an event. I now have an even greater respect for all of the conference organizers and realize that we need to have more people help them, to keep these conferences continue to get better and better. Plus, there will always be room for a new conference! > Thanks to the FreeBSD Foundation for being the biggest sponsor of BSDTW 2017 and always being the strongest backend of our community. We are excited about the many local companies and organizations that helped us whether with people, materials or financially. We even had 21 personal sponsors, more than two times the number of other big open source conferences in Taiwan. > As I said in the closing session, I'm not sure if there will be 2nd BSDTW next year. It still depends on the amount of sponsorship and number of volunteers. However, we will definitely hold more smaller meetups in the next year to keep building up the local BSD community. > Finally, in the beginning of this month, we had a “post-conference media workshop” for organizing the media files we collected in the BSDTW 2017. Here are the review article in Traditional Chinese and the photos: https://medium.com/@bsdtw/bsdtw-2017-總回顧-a402788daede (https://medium.com/@bsdtw/bsdtw-2017-總回顧-a402788daede) && https://www.flickr.com/photos/bsdtw/albums/72157689410035911 (https://www.flickr.com/photos/bsdtw/albums/72157689410035911) *** ##News Roundup ###Running FreeBSD on a Lenovo T470s (https://blog.grem.de/pages/t470s.html) Running FreeBSD on the Lenovo T470s ThinkPad > Installing FreeBSD on this machine was super easy. As I couldn't find a comprehensive/encouraging how-to about installing FreeBSD on a recent ThinkPad, I just wrote up the one below. It includes details about my personal setup, which are not required to run FreeBSD on this model, but which are more to my own taste. I still think this can be a quite useful inspiration for others who want to run their own customized configurations. Specs > The system I use has these specifications: Type: 20JS-001EGE CPU: Intel Core i7-6600U, 2x 2.60GHz RAM: 20GB DDR4 SSD: 512GB NVMe Graphics: Intel HD Graphics 520 (IGP), 1x HDMI 1.4 Display: 14", 1920x1080, non-glare, IPS Ports: 3x USB-A 3.0, 1x Thunderbolt 3, 1x Gb LAN Wireless: WLAN 802.11a/b/g/n/ac, Bluetooth 4.1, LTE (Micro-SIM) Cardreader: SD/SDHC/SDXC/MMC Webcam: 0.9 Megapixel Extras: MIL-STD-810G, Pointing Stick, Fingerprint-Reader, Docking port Things that work > Basically everything I care about: Accelerated video Keyboard Touchpad/ClickPad (like expected in a modern laptop) SSD WiFi Sound HDMI out Suspend to RAM Webcam Things that don't work Fingerprint reader Potentially anything I didn't test Battery life is okay, but could be better. Installation of the base system > I used a snapshot release of 12-CURRENT as the basis of my installation, particularly the one of 13th of December 2017. > I dd'ed it onto a memory stick and boot the laptop. I started a standard installation and created an encrypted ZFS pool on nvme0, using encryption, swap encryption and partition scheme "GPT (UEFI)". > After installation, it boots straight up. Ports tree used > All work is based on a head ports tree from about Dec 18, 22:15 CET, which should be more or less r456672. Preferred ClickPad configuration > As I'm not a fan of the the pointing stick, I disabled it in the bios. My final ClickPad configuration will be: Click to click (not tap), no middle button, right button in the lower right corner. As the old synaptics driver doesn't provide good thumb detection, libinput will be used. Check out the laptop list on the FreeBSD wiki for compatibility: (https://wiki.freebsd.org/Laptops/) *** ###FreeBSD desktop LiveCD creator (https://github.com/pkgdemon/comet) Introduction > The purpose of this tool is quickly generate bloat free images containing stock FreeBSD, and supported desktop environments. Features FreeBSD 11.1-RELEASE AMD64 Gnome & KDE desktop environments Hybrid DVD/USB image Screenshots [Gnome LiveCD])https://github.com/pkgdemon/comet/raw/master/screenshots/gnome-livecd.png?raw=true) KDE LiveCD (https://github.com/pkgdemon/comet/raw/master/screenshots/kde-livecd.png?raw=true) System Requirements FreeBSD 11.1, or higher for AMD64 20GB of free disk space 1GB of free memory UFS, or ZFS Initial Setup Install the required packages: pkg install git grub2-pcbsd grub2-efi xorriso Clone the repo: git clone https://www.github.com/pkgdemon/comet Enter the directory for running the LiveCD creator: cd comet/src Credentials for live media > User: liveuser > Password: freebsd *** ###iXsystems StorageCrypter Ransomware: Security Threat or Clickbait? (https://www.ixsystems.com/blog/storagecrypter/) ###pledge() work in progress (https://undeadly.org/cgi?action=article;sid=20171208082246) > I wanted to give an update that a two pledge-related changes are being worked on. The semantics and integration are complicated so it is taking some time. > One is execpromises. This will become the 2nd argument of pledge(). This allows one to set the pledge for the new image after pledge "exec"-allowed execve(). A warning though: utilizing this in software isn't as easy as you might think! The fork+exec + startup sequences needed to be studied quite carefully to ensure the newly-executed child doesn't ask for more than the parent's execpromises. In my experiments such a circumstance is exceedingly common, so the problem is eased by introducing a new pledge feature which allows pledge violations to return ENOSYS or such rather than killing the process. > This feature also needs to be used with great caution (especially in privileged programs) because programs which fail to observe errors may continue operating forward very incorrectly; you've lost the ability to catch it failing, and provide care by fixing the problem. > The other is pledgepaths. The semantics are still being tuned a bit. Before the first call to pledge() in a process, one can pledgepath() directories. Then later after pledge(), file access operations only work if the traversal of the path crosses one of those pre-declared directories (but better make sure you don't move a directory, because the kernel remembers and reasons about the vnode of the directory rather than the path). Something similar is being worked on for files, but we are still adjusting that, as well as a flag parameter for the pledgepath() call which may constrain the operations done on such files. > As such, pledgepath() will become a filesystem containment mechanism unlike chroot() because paths will still be based upon true /. > Patience. *** ###The anatomy of tee program on OpenBSD (http://nanxiao.me/en/the-anatomy-of-tee-program-on-openbsd/) > The tee command is used to read content from standard input and displays it not only in standard output but also saves to other files simultaneously. The source code of tee in OpenBSD is very simple, and I want to give it an analysis: > (1) tee leverages Singlely-linked List defined in sys/queue.h to manage outputted files (including standard output): struct list { SLIST_ENTRY(list) next; int fd; char *name; }; SLIST_HEAD(, list) head; ...... static void add(int fd, char *name) { struct list *p; ...... SLIST_INSERT_HEAD(&head, p, next); } int main(int argc, char *argv[]) { struct list *p; ...... SLIST_INIT(&head); ...... SLIST_FOREACH(p, &head, next) { ...... } } > To understand it easily, I extract the macros from sys/queue.h and created a file which utilizes the marcos: #define SLIST_HEAD(name, type) struct name { struct type *slh_first; /* first element */ } #define SLIST_ENTRY(type) struct { struct type *sle_next; /* next element */ } #define SLIST_FIRST(head) ((head)->slh_first) #define SLIST_END(head) NULL #define SLIST_EMPTY(head) (SLIST_FIRST(head) == SLIST_END(head)) #define SLIST_NEXT(elm, field) ((elm)->field.sle_next) #define SLIST_FOREACH(var, head, field) for((var) = SLIST_FIRST(head); (var) != SLIST_END(head); (var) = SLIST_NEXT(var, field)) #define SLIST_INIT(head) { SLIST_FIRST(head) = SLIST_END(head); } #define SLIST_INSERT_HEAD(head, elm, field) do { (elm)->field.sle_next = (head)->slh_first; (head)->slh_first = (elm); } while (0) struct list { SLIST_ENTRY(list) next; int fd; char *name; }; SLIST_HEAD(, list) head; int main(int argc, char *argv[]) { struct list *p; SLIST_INIT(&head); SLIST_INSERT_HEAD(&head, p, next); SLIST_FOREACH(p, &head, next) { } } > Then employed gcc‘s pre-processing function: # gcc -E slist.c # 1 "slist.c" # 1 "" # 1 "" # 1 "slist.c" # 30 "slist.c" struct list { struct { struct list *sle_next; } next; int fd; char *name; }; struct { struct list *slh_first; } head; int main(int argc, char *argv[]) { struct list *p; { ((&head)->slh_first) = NULL; }; do { (p)->next.sle_next = (&head)->slh_first; (&head)->slh_first = (p); } while (0); for((p) = ((&head)->slh_first); (p) != NULL; (p) = ((p)->next.sle_next)) { } } > It becomes clear now! The head node in list contains only 1 member: slhfirst, which points to the first valid node. For the elements in the list, it is embedded with next struct which uses slenext to refer to next buddy. > (2) By default, tee will overwrite the output files. If you want to append it, use -a option, and the code is as following: while (*argv) { if ((fd = open(*argv, O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC), DEFFILEMODE)) == -1) { ...... } ...... } > (3) The next part is the skeleton of saving content to files: while ((rval = read(STDIN_FILENO, buf, sizeof(buf))) > 0) { SLIST_FOREACH(p, &head, next) { n = rval; bp = buf; do { if ((wval = write(p->fd, bp, n)) == -1) { ...... } bp += wval; } while (n -= wval); } } > We need to iterates every opened file descriptor and write contents into it. > (4) Normally, theinterrupt signal will cause tee exit: # tee fdkfkdfjk fdkfkdfjk ^C # > To disable this feature, use -i option: # tee -i fdhfhd fdhfhd ^C^C > The corresponding code is like this: ...... case 'i': (void)signal(SIGINT, SIG_IGN); break; *** ##Beastie Bits What I learned from reading the OpenBSD's network stack source code (https://bijanebrahimi.github.io/blog/openbsds-network-stack-part-1.html) Broadcom BCM43224 and BCM43225 Wi-Fi cards now supported by bwn(4) (https://github.com/freebsd/freebsd/commit/888843e26a4e393f405c1c6cbdfc5b701670d363) Ingo details searching man pages (https://marc.info/?l=openbsd-misc&m=151320195122669&w=2) DTrace & ZFS Being Updated On NetBSD, Moving Away From Old OpenSolaris Code (https://www.phoronix.com/scan.php?page=news_item&px=NetBSD-ZFS-DTrace-Updating) Linux Professional Institute and BSD Certification Group Join Efforts (http://www.lpi.org/articles/linux-professional-institute-and-bsd-certification-group-join-efforts) The FreeBSD Foundation thanks Donors (https://www.freebsdfoundation.org/blog/thank-you-2/) ##Feedback/Questions Alex - My first freebsd bug (http://dpaste.com/3DSV7BC#wrap) John - Suggested Speakers (http://dpaste.com/2QFR4MT#wrap) Todd - Two questions (http://dpaste.com/2FQ450Q#wrap) Matthew - CentOS to FreeBSD (http://dpaste.com/3KA29E0#wrap) ***
TrueOS stable 17.12 is out, we have an OpenBSD workstation guide for you, learnings from the PDP-11, FreeBSD 2017 Releng recap and Duo SSH. This episode was brought to you by Headlines TrueOS stable release 17.12 (https://www.trueos.org/blog/trueos-17-12-release/) We are pleased to announce a new release of the 6-month STABLE version of TrueOS! This release cycle focused on lots of cleanup and stabilization of the distinguishing features of TrueOS: OpenRC, boot speed, removable-device management, SysAdm API integrations, Lumina improvements, and more. We have also been working quite a bit on the server offering of TrueOS, and are pleased to provide new text-based server images with support for Virtualization systems such as bhyve! This allows for simple server deployments which also take advantage of the TrueOS improvements to FreeBSD such as: Sane service management and status reporting with OpenRC Reliable, non-interactive system update mechanism with fail-safe boot environment support. Graphical management of remote TrueOS servers through SysAdm (also provides a reliable API for administrating systems remotely). LibreSSL for all base SSL support. Base system managed via packages (allows for additional fine-tuning). Base system is smaller due to the removal of the old GCC version in base. Any compiler and/or version may be installed and used via packages as desired. Support for newer graphics drivers and chipsets (graphics, networking, wifi, and more) TrueOS Version 17.12 (2017, December) is now available for download from the TrueOS website. Both the STABLE and UNSTABLE package repositories have also been updated in-sync with each other, so current users only need to follow the prompts about updating their system to run the new release. We are also pleased to announce the availability of TrueOS Sponsorships! If you would like to help contribute to the project financially we now have the ability to accept both one-time donations as well as recurring monthly donations which wil help us advocate for TrueOS around the world. Thank you all for using and supporting TrueOS! Notable Changes: Over 1100 OpenRC services have been created for 3rd-party packages. This should ensure the functionality of nearly all available 3rd-party packages that install/use their own services. The OpenRC services for FreeBSD itself have been overhauled, resulting in significantly shorter boot times. Separate install images for desktops and servers (server image uses a text/console installer) Bhyve support for TrueOS Server Install FreeBSD base is synced with 12.0-CURRENT as of December 4th, 2017 (Github commit: 209d01f) FreeBSD ports tree is synced as of November 30th (pre-FLAVOR changes) Lumina Desktop has been updated/developed from 1.3.0 to 1.4.1 PCDM now supports multiple simultaneous graphical sessions Removable devices are now managed through the “automounter” service. Devices are “announced” as available to the system via *.desktop shortcuts in /media. These shortcuts also contain a variety of optional “Actions” that may be performed on the device. Devices are only mounted while they are being used (such as when browsing via the command line or a file manager). Devices are automatically unmounted as soon as they stop being accessed. Integrated support for all major filesystems (UFS, EXT, FAT, NTFS, ExFAT, etc..) NOTE: The Lumina desktop is the only one which supports this functionality at the present time. The TrueOS update system has moved to an “active” update backend. This means that the user will need to actually start the update process by clicking the “Update Now” button in SysAdm, Lumina, or PCDM (as well as the command-line option). The staging of the update files is still performed automatically by default but this (and many other options) can be easily changed in the “Update Manager” settings as desired. Known Errata: [VirtualBox] Running FreeBSD within a VirtualBox VM is known to occasionally receive non-existent mouse clicks – particularly when using a scroll wheel or two-finger scroll. Quick Links: TrueOS Forums (https://discourse.trueos.org/) TrueOS Bugs (https://github.com/trueos/trueos-core/issues) TrueOS Handbook (https://www.trueos.org/handbook/trueos.html) TrueOS Community Chat on Telegram (https://t.me/TrueOSCommunity) *** OpenBSD Workstation Guide (https://begriffs.com/posts/2017-05-17-linux-workstation-guide.html) Design Goals User actions should complete instantaneously. While I understand if compiling code and rendering videos takes time, opening programs and moving windows should have no observable delay. The system should use minimalist tools. Corollary: cache data offline when possible. Everything from OpenStreetMaps to StackExchange can be stored locally. No reason to repeatedly hit the internet to query them. This also improves privacy because the initial download is indiscriminate and doesn't reveal personal queries or patterns of computer activity. No idling program should use a perceptible amount of CPU. Why does CalendarAgent on my Macbook sometimes use 150% CPU for fifteen minutes? Who knows. Why are background ChromeHelpers chugging along at upper-single-digit CPU? I didn't realize that holding a rendered DOM could be so challenging. Avoid interpreted languages, web-based desktop apps, and JavaScript garbage. There, I said it. Take your Electron apps with you to /dev/null! Stability. Old fashioned programs on a conservative OS on quality mainstream hardware. There are enough challenges to tackle without a bleeding edge system being one of them. Delegate to quality hardware components. Why use a janky ncurses software audio mixer when you can use…an actual audio mixer? Hardware privacy. No cameras or microphones that I can't physically disconnect. Also real hardware protection for cryptographic keys. Software privacy. Commercial software and operating systems have gotten so terrible about this. I even catch Mac command line tools trying to call Google Analytics. Sorry homebrew, your cute emojis don't make up for the surveillance. The Hardware Core To get the best hardware for the money I'm opting for a desktop computer. Haven't had one since the early 2000s and it feels anachronistic, but it will outperform a laptop of similar cost. After much searching, I found the HP Z240 Tower Workstation. It's no-nonsense and supports exactly the customizations I was looking for: No operating system pre-loaded (Cut out the “Windows tax”) Intel Xeon E3-1270 v6 processor (Supports ECC ram) 16 GB (2x8 GB) DDR4-2400 ECC Unbuffered memory (2400Mhz is the full memory clock speed supported by the Xeon) 256 GB HP Z Turbo Drive G2 PCIe SSD (Uses NVMe rather than SATA for faster throughput, supported by nvme(4)) No graphics card (We'll add our own) Intel® Ethernet I210-T1 PCIe (Supported by em(4)) A modest discrete video card will enable 2D Glamor acceleration on X11. The Radeon HD 6450 (sold separately) is fanless and listed as supported by radeon(4). Why build a solid computer and not protect it? Externally, the APC BR1300G UPS will protect the system from power surges and abrupt shutdowns. Peripherals The Matias Ergo Pro uses mechanical switches for that old fashioned clicky sound. It also includes dedicated buttons along the side for copying and pasting. Why is that cool? Well, it improves secondary selection, a technique that Sun computers used but time forgot. Since we're talking about a home office workstation, you may want a printer. The higher quality printers speak PostScript and PDF natively. Unix machines connect to them on TCP port 9100 and send PostScript commands directly. (You can print via telnet if you know the commands!) The Brother HL-L5100DN is a duplex LaserJet which allows that “raw” TCP printing. Audio/Video I know a lot of people enjoy surrounding themselves with a wall of monitors like they're in the heart of NASA Mission Control, but I find multi-monitor setups slightly disorienting. It introduces an extra bit of cognitive overhead to determine which monitor is for what exactly. That's why I'd go with a modest, crisp Dell UltraSharp 24" U2417H. It's 1080p and yeah there are 4k monitors nowadays, but text and icons are small enough as it is for me! If I ever considered a second monitor it would be e-ink for comfortably reading electronic copies of books or long articles. The price is currently too high to justify the purchase, but the most promising monitor seems to be the Dasung Paperlike. In the other direction, video input, it's more flexible to use a general-purpose HDMI capture box like the Rongyuxuan than settle on a particular webcam. This allows hooking up a real camera, or any other video device. Although the motherboard for this system has built-in audio, we should use a card with better OpenBSD support. The WBTUO PCIe card uses a C-Media CMI8768 chipset, handled by cmpci(4). The card provides S/PDIFF in and out ports if you ever want to use an external DAC or ADC. The way to connect it with other things is with a dedicated hardware mixer. The Behringer Xenyx 802 has all the connections needed, and the ability to route audio to and from the computer and a variety of devices at once. The mixer may seem an odd peripheral, but I want to mix the computer with an old fashioned CD player, ham radio gear, and amplifier so this unifies the audio setup. When doing remote pair programming or video team meetings it's nice to have a quality microphone. The best ones for this kind of work are directional, with a cardioid reception pattern. The MXL 770 condenser mic is perfect, and uses a powered XLR connection supplied by the mixer. Backups We're going dead simple and old-school, back to tapes. There are a set of tape standards called LTO-n. As n increases the tape capacity gets bigger, but the tape drive gets more expensive. In my opinion the best balance these days for the home user is LTO-3. You can usually find an HP Ultrium 960 LTO-3 on eBay for 150 dollars. The cartridges hold 800GB and are about 15 dollars apiece. Hard drives keep coming down in price, but these tapes are very cheap and simpler than keeping a bunch of disk drives. Also tape has proven longevity, and good recoverability. To use old fashioned tech like this you need a SCSI host bus adapter like the Adaptec 29320LPE, supported by ahd(4). Cryptography You don't want to generate and store secret keys on a general purpose network attached computer. The attack surface is a mile wide. Generating or manipulating “offline” secret keys needs to happen on a separate computer with no network access. Little boards like the Raspberry Pi would be good except they use ARM processors (incompatible with Tails OS) and have wifi. The JaguarBoard is a small x86 machine with no wireless capability. Just switch the keyboard and monitor over to this machine for your “cleanroom.” jaguar board: Generating keys requires entropy. The Linux kernel on Tails samples system properties to generate randomness, but why not help it out with a dedicated true random number generator (TRNG)? Bit Babbler supplies pure randomness at a high bitrate through USB. (OneRNG works better on the OpenBSD main system, via uonerng(4).) bit babbler: This little computer will save its results onto a OpenPGP Smartcard V2.1. This card provides write-only access to keys, and computes cryptographic primitives internally to sign and encrypt messages. To use it with a regular computer, hook up a Cherry ST2000 card reader. This reader has a PIN pad built in, so no keylogger on the main computer could even obtain your decryption PIN. The Software We take the beefed up hardware above and pair it with ninja-fast software written in C. Some text-based, others raw X11 graphical apps unencumbered by ties to any specific window manager. I'd advise OpenBSD for the underlying operating system, not a Linux. OpenBSD has greater internal consistency, their man pages are impeccable, and they make it a priority to prune old code to keep the system minimal. What Have We Learned from the PDP-11? (https://dave.cheney.net/2017/12/04/what-have-we-learned-from-the-pdp-11) The paper I have chosen tonight is a retrospective on a computer design. It is one of a series of papers by Gordon Bell, and various co-authors, spanning the design, growth, and eventual replacement of the companies iconic line of PDP-11 mini computers. This year represents the 60th anniversary of the founding of the company that produced the PDP-11. It is also 40 years since this paper was written, so I thought it would be entertaining to review Bell's retrospective through the lens of our own 20/20 hindsight. To set the scene for this paper, first we should talk a little about the company that produced the PDP-11, the Digital Equipment Corporation of Maynard, Massachusetts. Better known as DEC. It's also worth noting that the name PDP is an acronym for “Programmed Data Processor”, as at the time, computers had a reputation of being large, complicated, and expensive machines, and DEC's venture capitalists would not support them if they built a “computer” A computer is not solely determined by its architecture; it reflects the technological, economic, and human aspects of the environment in which it was designed and built. […] The finished computer is a product of the total design environment. “Right from the get go, Bell is letting us know that the success of any computer project is not abstractly building the best computer but building the right computer, and that takes context.” It is the nature of computer engineering to be goal-oriented, with pressure to produce deliverable products. It is therefore difficult to plan for an extensive lifetime. Because of the open nature of the PDP-11, anything which interpreted the instructions according to the processor specification, was a PDP-11, so there had been a rush within DEC, once it was clear that the PDP-11 market was heating up, to build implementations; you had different groups building fast, expensive ones and cost reduced slower ones The first weakness of minicomputers was their limited addressing capability. The biggest (and most common) mistake that can be made in a computer design is that of not providing enough address bits for memory addressing and management. A second weakness of minicomputers was their tendency not to have enough registers. This was corrected for the PDP-11 by providing eight 16-bit registers. Later, six 32-bit registers were added for floating-point arithmetic. […] More registers would increase the multiprogramming context switch time and confuse the user. “It's also interesting to note Bell's concern that additional registers would confuse the user. In the early 1970's the assumption that the machine would be programmed directly in assembly was still the prevailing mindset.” A third weakness of minicomputers was their lack of hardware stack capability. In the PDP-11, this was solved with the autoincrement/autodecrement addressing mechanism. This solution is unique to the PDP-11 and has proven to be exceptionally useful. (In fact, it has been copied by other designers.) “Nowadays it's hard to imagine hardware that doesn't have a notion of a stack, but consider that a stack isn't important if you don't need recursion.” “The design for the PDP-11 was laid down in 1969 and if we look at the programming languages of the time, FORTRAN and COBOL, neither supported recursive function calls. The function call sequence would often store the return address at a blank word at the start of the procedure making recursion impossible.” A fourth weakness, limited interrupt capability and slow context switching, was essentially solved with the device of UNIBUS interrupt vectors, which direct device interrupts. The basic mechanism is very fast, requiring only four memory cycles from the time an interrupt request is issued until the first instruction of the interrupt routine begins execution. A fifth weakness of prior minicomputers, inadequate character-handling capability, was met in the PDP-11 by providing direct byte addressing capability. “Strings and character handling were of increasing importance during the 1960's as scientific and business computing converged. The predominant character encodings at the time were 6 bit character sets which provided just enough space for upper case letters, the digits 0 to 9, space, and a few punctuation characters sufficient for printing financial reports.” “Because memory was so expensive, placing one 6 bit character into a 12 or 18 bit word was simply unacceptable so characters would be packed into words. This proved efficient for storage, but complex for operations like move, compare, and concatenate, which had to account for a character appearing in the top or bottom of the word, expending valuable words of program storage to cope.” “The problem was addressed in the PDP-11 by allowing the machine to operate on memory as both a 16-bit word, and the increasingly popular 8-bit byte. The expenditure of 2 additional bits per character was felt to be worth it for simpler string handling, and also eased the adoption of the increasingly popular 7-bit ASCII standard of which DEC were a proponent at the time. Bell concludes this point with the throw away line:” Although string instructions are not yet provided in the hardware, the common string operations (move, compare, concatenate) can be programmed with very short loops. A sixth weakness, the inability to use read-only memories, was avoided in the PDP-11. Most code written for the PDP-11 tends to be pure and reentrant without special effort by the programmer, allowing a read-only memory (ROM) to be used directly. A seventh weakness, one common to many minicomputers, was primitive I/O capabilities. A ninth weakness of minicomputers was the high cost of programming them. Many users program in assembly language, without the comfortable environment of editors, file systems, and debuggers available on bigger systems. The PDP-11 does not seem to have overcome this weakness, although it appears that more complex systems are being built successfully with the PDP-11 than with its predecessors, the PDP-8 and PDP-15. The problems faced by computer designers can usually be attributed to one of two causes: inexperience or second-systemitis Before the PDP-11, there was no UNIX. Before the PDP-11, there was no C, this is the computer that C was designed on. If you want to know why the classical C int is 16 bits wide, it's because of the PDP-11. UNIX bought us ideas such as pipes, everything is a file, and interactive computing. UNIX, which had arrived at Berkley in 1974 aboard a tape carried by Ken Thompson, would evolve into the west coast flavoured Berkley Systems Distribution. Berkeley UNIX had been ported to the VAX by the start of the 1980's and was thriving as the counter cultural alternative to DEC's own VMS operating system. Berkeley UNIX spawned a new generation of hackers who would go on to form companies like Sun micro systems, and languages like Self, which lead directly to the development of Java. UNIX was ported to a bewildering array of computer systems during the 80's and the fallout from the UNIX wars gave us the various BSD operating systems who continue to this day. The article, and the papers it is summarizing, contain a lot more than we could possibly dig into even if we dedicated the entire show to the topic *** News Roundup Two-factor authentication SSH with Duo in FreeBSD 11 (https://www.teachnix.com/2017/11/29/configuring-two-factor-authentication-on-freebsd-with-duo/) This setup uses an SSH key as the first factor of authentication. Please watch Part 1 on setting up SSH keys and how to scp it to your server. Video guide (https://www.youtube.com/watch?v=E5EuvF-iaV0) Register for a free account at Duo.com Install the Duo package on your FreeBSD server pkg install -y duo Log into the Duo site > Applications > Protect an Application > Search for Unix application > Protect this Application This will generate the keys we need to configure Duo. Edit the Duo config file using the course notes template vi /usr/local/etc/pam_duo.conf Example config [duo] ; Duo integration key ikey = Integration key goes here ; Duo secret key skey = Secret key goes here ; Duo API host host = API hostname goes here Change the permissions of the Duo config file. If the permissions are not correct then the service will not function properly. chmod 600 /usr/local/etc/pam_duo.conf Edit the SSHD config file using the course notes template vi /etc/ssh/sshd_config Example config ListenAddress 0.0.0.0 Port 22 PasswordAuthentication no UsePAM yes ChallengeResponseAuthentication yes UseDNS no PermitRootLogin yes AuthenticationMethods publickey,keyboard-interactive Edit PAM to configure SSHD for Duo using the course notes template Example config ``` # auth auth sufficient pamopie.so nowarn nofakeprompts auth requisite pamopieaccess.so nowarn allowlocal auth required /usr/local/lib/security/pamduo.so # session # session optional pamssh.so wantagent session required pam_permit.so # password # password sufficient pamkrb5.so nowarn tryfirstpass password required pamunix.so nowarn tryfirstpass ``` Restart the sshd service service sshd restart SSH into your FreeBSD server and follow the link it outputs to enroll your phone with Duo. ssh server.example.com SSH into your server again ssh server.example.com Choose your preferred method and it should log you into your server. FreeBSD 2017 Release Engineering Recap (https://www.freebsdfoundation.org/blog/2017-release-engineering-recap/) This past year was undoubtedly a rather busy and successful year for the Release Engineering Team. Throughout the year, development snapshot builds for FreeBSD-CURRENT and supported FreeBSD-STABLE branches were continually provided. In addition, work to package the base system using pkg(8) continued throughout the year and remains ongoing. The FreeBSD Release Engineering Team worked on the FreeBSD 11.1-RELEASE, with the code slush starting mid-May. The FreeBSD 11.1-RELEASE cycle stayed on schedule, with the final release build starting July 21, and the final release announcement following on July 25, building upon the stability and reliability of 11.0-RELEASE. Milestones during the 11.1-RELEASE cycle can be found on the 11.1 schedule page (https://www.freebsd.org/releases/11.1R/schedule.html). The final announcement is available here (https://www.freebsd.org/releases/11.1R/announce.html). The FreeBSD Release Engineering Team started the FreeBSD 10.4-RELEASE cycle, led by Marius Strobl. The FreeBSD 10.4-RELEASE cycle continued on schedule, with the only adjustments to the schedule being the addition of BETA4 and the removal of RC3. FreeBSD 10.4-RELEASE builds upon the stability and reliability of FreeBSD 10.3-RELEASE, and is planned to be the final release from the stable/10 branch. Milestones during the 10.4-RELEASE cycle can be found on the 10.4 schedule page (https://www.freebsd.org/releases/10.4R/schedule.html). The final announcement is available here (https://www.freebsd.org/releases/10.4R/announce.html). In addition to these releases, support for additional arm single-board computer images were added, notably Raspberry Pi 3 and Pine64. Additionally, release-related documentation effective 12.0-RELEASE and later has been moved from the base system repository to the documentation repository, making it possible to update related documentation as necessary post-release. Additionally, the FreeBSD Release Engineering article in the Project Handbook had been rewritten to outline current practices used by the Release Engineering Team. For more information on the procedures and processes the FreeBSD Release Engineering Team follows, the new article is available here and continually updated as procedures change. Finally, following the availability of FreeBSD 11.1-RELEASE, Glen Barber attended the September Developer Summit hosted at vBSDCon in Reston, VA, USA, where he gave a brief talk comprising of several points relating directly to the 11.1-RELEASE cycle. In particular, some of the points covered included what he felt went well during the release cycle, what did not go as well as it could have, and what we, as a Project, could do better to improve the release process. The slides from the talk are available in the FreeBSD Wiki. During the question and answer time following the talk, some questions asked included: Q: Should developers use the ‘Relnotes' tag in the Subversion commit template more loosely, at risk of an increase in false positives. A: When asked when the tag in the template was initially added, the answer would have been “no”, however in hindsight it is easier to sift through the false positives, than to comb through months or years of commit logs. Q: What issues are present preventing moving release-related documentation to the documentation repository? A: There were some rendering issues last time it was investigated, but it is really nothing more than taking the time to fix those issues. (Note, that since this talk, the migration of the documentation in question had moved.) Q: Does it make sense to extend the timeframe between milestone builds during a release cycle from one week to two weeks, to allow more time for testing, for example, RC1 versus RC2? A: No. It would extend the length of the release cycle with no real benefit between milestones since as we draw nearer to the end of a given release cycle, the number of changes to that code base significantly reduce. FLIMP - GIMP Exploit on FreeBSD (https://flimp.fuzzing-project.org) In 2014, when starting the Fuzzing Project (https://fuzzing-project.org/), Hanno Böck did some primitive fuzzing on GIMP and reported two bugs. They weren't fixed and were forgotten in the public bug tracker. Recently Tobias Stöckmann found one of these bugs (https://bugzilla.gnome.org/show_bug.cgi?id=739133) (CVE-2017-17785) and figured out that it's easy to exploit. What kind of bug is that? It's a classic heap buffer overflow in the FLIC parser. FLIC is a file format for animations and was introduced by Autodesk Animator. How does the exploit work? Tobias has created a detailed writeup (https://flimp.fuzzing-project.org/exploit.html). The exploit doesn't work for me! We figured out it's unreliable and the memory addresses are depending on many circumstances. The exploit ZIP comes with two variations using different memory addresses. Try both of them. We also noticed putting the files in a subdirectory sometimes made the exploit work. Anything more to tell about the GIMP? There's a wide variety of graphics formats. GIMP tries to support many of them, including many legacy formats that nobody is using any more today. While this has obvious advantages - you can access the old images you may find on a backup CD from 1995 - it comes with risks. Support for many obscure file formats means many parsers that hardly anyone ever looks at. So... what about the other parsers? The second bug (https://bugzilla.gnome.org/show_bug.cgi?id=739134) (CVE-2017-17786), which is a simple overread, was in the TGA parser. Furthermore we found buffer overreads in the XCF parser (https://bugzilla.gnome.org/show_bug.cgi?id=790783) (CVE-2017-17788), the Gimp Brush (GBR) parser (https://bugzilla.gnome.org/show_bug.cgi?id=790784) (CVE-2017-17784) and the Paint Shop Pro (PSP) parser (https://bugzilla.gnome.org/show_bug.cgi?id=790849) (CVE-2017-17789). We found another Heap buffer overflow (https://bugzilla.gnome.org/show_bug.cgi?id=790849) in the Paint Shop Pro parser (CVE-2017-17787) which is probably also exploitable. In other words: The GIMP import parsers are full of memory safety bugs. What should happen? First of all obviously all known memory safety bugs should be fixed. Furthermore we believe the way GIMP plugins work is not ideal for security testing. The plug-ins are separate executables, however they can't be executed on their own, as they communicate with the main GIMP process. Ideally either these plug-ins should be changed in a way that allows running them directly from the command line or - even better - they should be turned into libraries. The latter would also have the advantage of making the parser code useable for other software projects. Finally it might be a good idea to sandbox the import parsers. Dell FS12-NV7 Review – Bargain FreeBSD/ZFS box (http://blog.frankleonhardt.com/2017/dell-fs12-nv7-review-bargain-freebsdzfs-box/) It seems just about everyone selling refurbished data centre kit has a load of Dell FS12-NV7's to flog. Dell FS-what? You won't find them in the Dell catalogue, that's for sure. They look a bit like C2100s of some vintage, and they have a lot in common. But on closer inspection they're obviously a “special” for an important customer. Given the number of them knocking around, it's obviously a customer with big data, centres stuffed full of servers with a lot of processing to do. Here's a hint: It's not Google or Amazon. So, should you be buying a weirdo box with no documentation whatsoever? I'd say yes, definitely. If you're interests are anything like mine. In a 2U box you can get twin 4-core CPUs and 64Gb of RAM for £150 or less. What's not to like? Ah yes, the complete lack of documentation. Over the next few weeks I intend to cover that. And to start off this is my first PC review for nearly twenty years. As I mentioned, it's a 2U full length heavy metal box on rails. On the back there are the usual I/O ports: a 9-way RS-232, VGA, two 1Gb Ethernet, two USB2 and a PS/2 keyboard and mouse. The front is taken up by twelve 3.5″ hard drive bays, with the status lights and power button on one of the mounting ears to make room. Unlike other Dell servers, all the connections are on the back, only. So, in summary, you're getting a lot for your money if its the kind of thing you want. It's ideal as a high-performance Unix box with plenty of drive bays (preferably running BSD and ZFS). In this configuration it really shifts. Major bang-per-buck. Another idea I've had is using it for a flight simulator. That's a lot of RAM and processors for the money. If you forego the SAS controllers in the PCIe slots and dump in a decent graphics card and sound board, it's hard to see what's could be better (and you get jet engine sound effects without a speaker). So who should buy one of these? BSD geeks is the obvious answer. With a bit of tweaking they're a dream. It can build-absolutely-everything in 20-30 minutes. For storage you can put fast SAS drives in and it goes like the wind, even at 3Gb bandwidth per drive. I don't know if it works with FreeNAS but I can't see why not – I'm using mostly FreeBSD 11.1 and the generic kernel is fine. And if you want to run a load of weird operating systems (like Windows XP) in VM format, it seems to work very well with the Xen hypervisor and Dom0 under FreeBSD. Or CentOS if you prefer. So I shall end this review in true PCW style: Pros: Cheap Lots of CPUs, Lots of RAM Lots of HD slots Great for BSD/ZFS or VMs Cons: Noisy no AES-NI SAS needs upgrading Limited PCI slots As I've mentioned, the noise and SAS are easy and relatively cheap to fix, and thanks to BitCoin miners, even the PCI slot problem can be sorted. I'll talk about this in a later post. Beastie Bits Reflections on Hackathons (https://undeadly.org/cgi?action=article;sid=20171126090055) 7-Part Video Crash Course on SaltStack For FreeBSD (https://www.youtube.com/watch?v=HijG0hWebZk&list=PL5yV8umka8YQOr1wm719In5LITdGzQMOF) The LLVM Thread Sanitizer has been ported to NetBSD (https://blog.netbsd.org/tnf/entry/the_llvm_thread_sanitizer_has) The First Unix Port (1998) (http://bitsavers.informatik.uni-stuttgart.de/bits/Interdata/32bit/unix/univWollongong_v6/miller.pdf) arm64 platform now officially supported [and has syspatch(8)] (https://undeadly.org/cgi?action=article;sid=20171208082238) BSDCan 2018 Call for Participation (https://www.freebsdfoundation.org/news-and-events/call-for-papers/bsdcan-2018-call-for-participation/) AsiaBSDCon 2018 Call for Papers (https://www.freebsdfoundation.org/news-and-events/call-for-papers/asiabsdcon-2018-call-for-papers/) *** Feedback/Questions Shawn - DragonFlyBSD vagrant images (http://dpaste.com/3PRPJHG#wrap) Ben - undermydesk (http://dpaste.com/0AZ32ZB#wrap) Ken - Conferences (http://dpaste.com/3E8FQC6#wrap) Ben - ssh keys (http://dpaste.com/0E4538Q#wrap) SSH Chaining (https://www.bsdnow.tv/tutorials/ssh-chaining) ***
We take a look at two-faced Oracle, cover a FAMP installation, how Netflix works the complex stuff, and show you who the patron of yak shaving is. This episode was brought to you by Headlines Why is Oracle so two-faced over open source? (https://www.theregister.co.uk/2017/10/12/oracle_must_grow_up_on_open_source/) Oracle loves open source. Except when the database giant hates open source. Which, according to its recent lobbying of the US federal government, seems to be "most of the time". Yes, Oracle has recently joined the Cloud Native Computing Foundation (CNCF) to up its support for open-source Kubernetes and, yes, it has long supported (and contributed to) Linux. And, yes, Oracle has even gone so far as to (finally) open up Java development by putting it under a foundation's stewardship. Yet this same, seemingly open Oracle has actively hammered the US government to consider that "there is no math that can justify open source from a cost perspective as the cost of support plus the opportunity cost of forgoing features, functions, automation and security overwhelm any presumed cost savings." That punch to the face was delivered in a letter to Christopher Liddell, a former Microsoft CFO and now director of Trump's American Technology Council, by Kenneth Glueck, Oracle senior vice president. The US government had courted input on its IT modernisation programme. Others writing back to Liddell included AT&T, Cisco, Microsoft and VMware. In other words, based on its letter, what Oracle wants us to believe is that open source leads to greater costs and poorly secured, limply featured software. Nor is Oracle content to leave it there, also arguing that open source is exactly how the private sector does not function, seemingly forgetting that most of the leading infrastructure, big data, and mobile software today is open source. Details! Rather than take this counterproductive detour into self-serving silliness, Oracle would do better to follow Microsoft's path. Microsoft, too, used to Janus-face its way through open source, simultaneously supporting and bashing it. Only under chief executive Satya Nadella's reign did Microsoft realise it's OK to fully embrace open source, and its financial results have loved the commitment. Oracle has much to learn, and emulate, in Microsoft's approach. I love you, you're perfect. Now change Oracle has never been particularly warm and fuzzy about open source. As founder Larry Ellison might put it, Oracle is a profit-seeking corporation, not a peace-loving charity. To the extent that Oracle embraces open source, therefore it does so for financial reward, just like every other corporation. Few, however, are as blunt as Oracle about this fact of corporate open-source life. As Ellison told the Financial Times back in 2006: "If an open-source product gets good enough, we'll simply take it. So the great thing about open source is nobody owns it – a company like Oracle is free to take it for nothing, include it in our products and charge for support, and that's what we'll do. "So it is not disruptive at all – you have to find places to add value. Once open source gets good enough, competing with it would be insane... We don't have to fight open source, we have to exploit open source." "Exploit" sounds about right. While Oracle doesn't crack the top-10 corporate contributors to the Linux kernel, it does register a respectable number 12, which helps it influence the platform enough to feel comfortable building its IaaS offering on Linux (and Xen for virtualisation). Oracle has also managed to continue growing MySQL's clout in the industry while improving it as a product and business. As for Kubernetes, Oracle's decision to join the CNCF also came with P&L strings attached. "CNCF technologies such as Kubernetes, Prometheus, gRPC and OpenTracing are critical parts of both our own and our customers' development toolchains," said Mark Cavage, vice president of software development at Oracle. One can argue that Oracle has figured out the exploitation angle reasonably well. This, however, refers to the right kind of exploitation, the kind that even free software activist Richard Stallman can love (or, at least, tolerate). But when it comes to government lobbying, Oracle looks a lot more like Mr Hyde than Dr Jekyll. Lies, damned lies, and Oracle lobbying The current US president has many problems (OK, many, many problems), but his decision to follow the Obama administration's support for IT modernisation is commendable. Most recently, the Trump White House asked for feedback on how best to continue improving government IT. Oracle's response is high comedy in many respects. As TechDirt's Mike Masnick summarises, Oracle's "latest crusade is against open-source technology being used by the federal government – and against the government hiring people out of Silicon Valley to help create more modern systems. Instead, Oracle would apparently prefer the government just give it lots of money." Oracle is very good at making lots of money. As such, its request for even more isn't too surprising. What is surprising is the brazenness of its position. As Masnick opines: "The sheer contempt found in Oracle's submission on IT modernization is pretty stunning." Why? Because Oracle contradicts much that it publicly states in other forums about open source and innovation. More than this, Oracle contradicts much of what we now know is essential to competitive differentiation in an increasingly software and data-driven world. Take, for example, Oracle's contention that "significant IT development expertise is not... central to successful modernization efforts". What? In our "software is eating the world" existence Oracle clearly believes that CIOs are buyers, not doers: "The most important skill set of CIOs today is to critically compete and evaluate commercial alternatives to capture the benefits of innovation conducted at scale, and then to manage the implementation of those technologies efficiently." While there is some truth to Oracle's claim – every project shouldn't be a custom one-off that must be supported forever – it's crazy to think that a CIO – government or otherwise – is doing their job effectively by simply shovelling cash into vendors' bank accounts. Indeed, as Masnick points out: "If it weren't for Oracle's failures, there might not even be a USDS [the US Digital Service created in 2014 to modernise federal IT]. USDS really grew out of the emergency hiring of some top-notch internet engineers in response to the Healthcare.gov rollout debacle. And if you don't recall, a big part of that debacle was blamed on Oracle's technology." In short, blindly giving money to Oracle and other big vendors is the opposite of IT modernisation. In its letter to Liddell, Oracle proceeded to make the fantastic (by which I mean "silly and false") claim that "the fact is that the use of open-source software has been declining rapidly in the private sector". What?!? This is so incredibly untrue that Oracle should score points for being willing to say it out loud. Take a stroll through the most prominent software in big data (Hadoop, Spark, Kafka, etc.), mobile (Android), application development (Kubernetes, Docker), machine learning/AI (TensorFlow, MxNet), and compare it to Oracle's statement. One conclusion must be that Oracle believes its CIO audience is incredibly stupid. Oracle then tells a half-truth by declaring: "There is no math that can justify open source from a cost perspective." How so? Because "the cost of support plus the opportunity cost of forgoing features, functions, automation and security overwhelm any presumed cost savings." Which I guess is why Oracle doesn't use any open source like Linux, Kubernetes, etc. in its services. Oops. The Vendor Formerly Known As Satan The thing is, Oracle doesn't need to do this and, for its own good, shouldn't do this. After all, we already know how this plays out. We need only look at what happened with Microsoft. Remember when Microsoft wanted us to "get the facts" about Linux? Now it's a big-time contributor to Linux. Remember when it told us open source was anti-American and a cancer? Now it aggressively contributes to a huge variety of open-source projects, some of them homegrown in Redmond, and tells the world that "Microsoft loves open source." Of course, Microsoft loves open source for the same reason any corporation does: it drives revenue as developers look to build applications filled with open-source components on Azure. There's nothing wrong with that. Would Microsoft prefer government IT to purchase SQL Server instead of open-source-licensed PostgreSQL? Sure. But look for a single line in its response to the Trump executive order that signals "open source is bad". You won't find it. Why? Because Microsoft understands that open source is a friend, not foe, and has learned how to monetise it. Microsoft, in short, is no longer conflicted about open source. It can compete at the product level while embracing open source at the project level, which helps fuel its overall product and business strategy. Oracle isn't there yet, and is still stuck where Microsoft was a decade ago. It's time to grow up, Oracle. For a company that builds great software and understands that it increasingly needs to depend on open source to build that software, it's disingenuous at best to lobby the US government to put the freeze on open source. Oracle needs to learn from Microsoft, stop worrying and love the open-source bomb. It was a key ingredient in Microsoft's resurgence. Maybe it could help Oracle get a cloud clue, too. Install FAMP on FreeBSD (https://www.linuxsecrets.com/home/3164-install-famp-on-freebsd) The acronym FAMP refers to a set of free open source applications which are commonly used in Web server environments called Apache, MySQL and PHP on the FreeBSD operating system, which provides a server stack that provides web services, database and PHP. Prerequisites sudo Installed and working - Please read Apache PHP5 or PHP7 MySQL or MariaDB Install your favorite editor, ours is vi Note: You don't need to upgrade FreeBSD but make sure all patches have been installed and your port tree is up-2-date if you plan to update by ports. Install Ports portsnap fetch You must use sudo for each indivdual command during installations. Please see link above for installing sudo. Searching Available Apache Versions to Install pkg search apache Install Apache To install Apache 2.4 using pkg. The apache 2.4 user account managing Apache is www in FreeBSD. pkg install apache24 Confirmation yes prompt and hit y for yes to install Apache 2.4 This installs Apache and its dependencies. Enable Apache use sysrc to update services to be started at boot time, Command below adds "apache24enable="YES" to the /etc/rc.conf file. For sysrc commands please read ```sysrc apache24enable=yes Start Apache service apache24 start``` Visit web address by accessing your server's public IP address in your web browser How To find Your Server's Public IP Address If you do not know what your server's public IP address is, there are a number of ways that you can find it. Usually, this is the address you use to connect to your server through SSH. ifconfig vtnet0 | grep "inet " | awk '{ print $2 }' Now that you have the public IP address, you may use it in your web browser's address bar to access your web server. Install MySQL Now that we have our web server up and running, it is time to install MySQL, the relational database management system. The MySQL server will organize and provide access to databases where our server can store information. Install MySQL 5.7 using pkg by typing pkg install mysql57-server Enter y at the confirmation prompt. This installs the MySQL server and client packages. To enable MySQL server as a service, add mysqlenable="YES" to the /etc/rc.conf file. This sysrc command will do just that ```sysrc mysqlenable=yes Now start the MySQL server service mysql-server start Now run the security script that will remove some dangerous defaults and slightly restrict access to your database system. mysqlsecureinstallation``` Answer all questions to secure your newly installed MySQL database. Enter current password for root (enter for none): [RETURN] Your database system is now set up and we can move on. Install PHP5 or PHP70 pkg search php70 Install PHP70 you would do the following by typing pkg install php70-mysqli mod_php70 Note: In these instructions we are using php5.7 not php7.0. We will be coming out with php7.0 instructions with FPM. PHP is the component of our setup that will process code to display dynamic content. It can run scripts, connect to MySQL databases to get information, and hand the processed content over to the web server to display. We're going to install the modphp, php-mysql, and php-mysqli packages. To install PHP 5.7 with pkg, run this command ```pkg install modphp56 php56-mysql php56-mysqli Copy sample PHP configuration file into place. cp /usr/local/etc/php.ini-production /usr/local/etc/php.ini Regenerate the system's cached information about your installed executable files rehash``` Before using PHP, you must configure it to work with Apache. Install PHP Modules (Optional) To enhance the functionality of PHP, we can optionally install some additional modules. To see the available options for PHP 5.6 modules and libraries, you can type this into your system pkg search php56 Get more information about each module you can look at the long description of the package by typing pkg search -f apache24 Optional Install Example pkg install php56-calendar Configure Apache to Use PHP Module Open the Apache configuration file vim /usr/local/etc/apache24/Includes/php.conf DirectoryIndex index.php index.html Next, we will configure Apache to process requested PHP files with the PHP processor. Add these lines to the end of the file: SetHandler application/x-httpd-php SetHandler application/x-httpd-php-source Now restart Apache to put the changes into effect service apache24 restart Test PHP Processing By default, the DocumentRoot is set to /usr/local/www/apache24/data. We can create the info.php file under that location by typing vim /usr/local/www/apache24/data/info.php Add following line to info.php and save it. Details on info.php info.php file gives you information about your server from the perspective of PHP. It' useful for debugging and to ensure that your settings are being applied correctly. If this was successful, then your PHP is working as expected. You probably want to remove info.php after testing because it could actually give information about your server to unauthorized users. Remove file by typing rm /usr/local/www/apache24/data/info.php Note: Make sure Apache / meaning the root of Apache is owned by user which should have been created during the Apache install is the owner of the /usr/local/www structure. That explains FAMP on FreeBSD. IXsystems IXsystems TrueNAS X10 Torture Test & Fail Over Systems In Action with the ZFS File System (https://www.youtube.com/watch?v=GG_NvKuh530) How Netflix works: what happens every time you hit Play (https://medium.com/refraction-tech-everything/how-netflix-works-the-hugely-simplified-complex-stuff-that-happens-every-time-you-hit-play-3a40c9be254b) Not long ago, House of Cards came back for the fifth season, finally ending a long wait for binge watchers across the world who are interested in an American politician's ruthless ascendance to presidency. For them, kicking off a marathon is as simple as reaching out for your device or remote, opening the Netflix app and hitting Play. Simple, fast and instantly gratifying. What isn't as simple is what goes into running Netflix, a service that streams around 250 million hours of video per day to around 98 million paying subscribers in 190 countries. At this scale, providing quality entertainment in a matter of a few seconds to every user is no joke. And as much as it means building top-notch infrastructure at a scale no other Internet service has done before, it also means that a lot of participants in the experience have to be negotiated with and kept satiated?—?from production companies supplying the content, to internet providers dealing with the network traffic Netflix brings upon them. This is, in short and in the most layman terms, how Netflix works. Let us just try to understand how Netflix is structured on the technological side with a simple example. Netflix literally ushered in a revolution around ten years ago by rewriting the applications that run the entire service to fit into a microservices architecture?—?which means that each application, or microservice's code and resources are its very own. It will not share any of it with any other app by nature. And when two applications do need to talk to each other, they use an application programming interface (API)?—?a tightly-controlled set of rules that both programs can handle. Developers can now make many changes, small or huge, to each application as long as they ensure that it plays well with the API. And since the one program knows the other's API properly, no change will break the exchange of information. Netflix estimates that it uses around 700 microservices to control each of the many parts of what makes up the entire Netflix service: one microservice stores what all shows you watched, one deducts the monthly fee from your credit card, one provides your device with the correct video files that it can play, one takes a look at your watching history and uses algorithms to guess a list of movies that you will like, and one will provide the names and images of these movies to be shown in a list on the main menu. And that's the tip of the iceberg. Netflix engineers can make changes to any part of the application and can introduce new changes rapidly while ensuring that nothing else in the entire service breaks down. They made a courageous decision to get rid of maintaining their own servers and move all of their stuff to the cloud?—?i.e. run everything on the servers of someone else who dealt with maintaining the hardware while Netflix engineers wrote hundreds of programs and deployed it on the servers rapidly. The someone else they chose for their cloud-based infrastructure is Amazon Web Services (AWS). Netflix works on thousands of devices, and each of them play a different format of video and sound files. Another set of AWS servers take this original film file, and convert it into hundreds of files, each meant to play the entire show or film on a particular type of device and a particular screen size or video quality. One file will work exclusively on the iPad, one on a full HD Android phone, one on a Sony TV that can play 4K video and Dolby sound, one on a Windows computer, and so on. Even more of these files can be made with varying video qualities so that they are easier to load on a poor network connection. This is a process known as transcoding. A special piece of code is also added to these files to lock them with what is called digital rights management or DRM?—?a technological measure which prevents piracy of films. The Netflix app or website determines what particular device you are using to watch, and fetches the exact file for that show meant to specially play on your particular device, with a particular video quality based on how fast your internet is at that moment. Here, instead of relying on AWS servers, they install their very own around the world. But it has only one purpose?—?to store content smartly and deliver it to users. Netflix strikes deals with internet service providers and provides them the red box you saw above at no cost. ISPs install these along with their servers. These Open Connect boxes download the Netflix library for their region from the main servers in the US?—?if there are multiple of them, each will rather store content that is more popular with Netflix users in a region to prioritise speed. So a rarely watched film might take time to load more than a Stranger Things episode. Now, when you will connect to Netflix, the closest Open Connect box to you will deliver the content you need, thus videos load faster than if your Netflix app tried to load it from the main servers in the US. In a nutshell… This is what happens when you hit that Play button: Hundreds of microservices, or tiny independent programs, work together to make one large Netflix service. Content legally acquired or licensed is converted into a size that fits your screen, and protected from being copied. Servers across the world make a copy of it and store it so that the closest one to you delivers it at max quality and speed. When you select a show, your Netflix app cherry picks which of these servers will it load the video from> You are now gripped by Frank Underwood's chilling tactics, given depression by BoJack Horseman's rollercoaster life, tickled by Dev in Master of None and made phobic to the future of technology by the stories in Black Mirror. And your lifespan decreases as your binge watching turns you into a couch potato. It looked so simple before, right? News Roundup Moving FreshPorts (http://dan.langille.org/2017/11/15/moving-freshports/) Today I moved the FreshPorts website from one server to another. My goal is for nobody to notice. In preparation for this move, I have: DNS TTL reduced to 60s Posted to Twitter Updated the status page Put the website put in offline mode: What was missed I turned off commit processing on the new server, but I did not do this on the old server. I should have: sudo svc -d /var/service/freshports That stops processing of incoming commits. No data is lost, but it keeps the two databases at the same spot in history. Commit processing could continue during the database dumping, but that does not affect the dump, which will be consistent regardless. The offline code Here is the basic stuff I used to put the website into offline mode. The main points are: header(“HTTP/1.1 503 Service Unavailable”); ErrorDocument 404 /index.php I move the DocumentRoot to a new directory, containing only index.php. Every error invokes index.php, which returns a 503 code. The dump The database dump just started (Sun Nov 5 17:07:22 UTC 2017). root@pg96:~ # /usr/bin/time pg_dump -h 206.127.23.226 -Fc -U dan freshports.org > freshports.org.9.6.dump That should take about 30 minutes. I have set a timer to remind me. Total time was: 1464.82 real 1324.96 user 37.22 sys The MD5 is: MD5 (freshports.org.9.6.dump) = 5249b45a93332b8344c9ce01245a05d5 It is now: Sun Nov 5 17:34:07 UTC 2017 The rsync The rsync should take about 10-20 minutes. I have already done an rsync of yesterday's dump file. The rsync today should copy over only the deltas (i.e. differences). The rsync started at about Sun Nov 5 17:36:05 UTC 2017 That took 2m9.091s The MD5 matches. The restore The restore should take about 30 minutes. I ran this test yesterday. It is now Sun Nov 5 17:40:03 UTC 2017. $ createdb -T template0 -E SQL_ASCII freshports.testing $ time pg_restore -j 16 -d freshports.testing freshports.org.9.6.dump Done. real 25m21.108s user 1m57.508s sys 0m15.172s It is now Sun Nov 5 18:06:22 UTC 2017. Insert break here About here, I took a 30 minute break to run an errand. It was worth it. Changing DNS I'm ready to change DNS now. It is Sun Nov 5 19:49:20 EST 2017 Done. And nearly immediately, traffic started. How many misses? During this process, XXXXX requests were declined: $ grep -c '" 503 ' /usr/websites/log/freshports.org-access.log XXXXX That's it, we're done Total elapsed time: 1 hour 48 minutes. There are still a number of things to follow up on, but that was the transfers. The new FreshPorts Server (http://dan.langille.org/2017/11/17/x8dtu-3/) *** Using bhyve on top of CEPH (https://lists.freebsd.org/pipermail/freebsd-virtualization/2017-November/005876.html) Hi, Just an info point. I'm preparing for a lecture tomorrow, and thought why not do an actual demo.... Like to be friends with Murphy :) So after I started the cluster: 5 jails with 7 OSDs This what I manually needed to do to boot a memory stick Start een Bhyve instance rbd --dest-pool rbddata --no-progress import memstick.img memstick rbd-ggate map rbddata/memstick ggate-devvice is available on /dev/ggate1 kldload vmm kldload nmdm kldload iftap kldload ifbridge kldload cpuctl sysctl net.link.tap.uponopen=1 ifconfig bridge0 create ifconfig bridge0 addm em0 up ifconfig ifconfig tap11 create ifconfig bridge0 addm tap11 ifconfig tap11 up load the GGate disk in bhyve bhyveload -c /dev/nmdm11A -m 2G -d /dev/ggate1 FB11 and boot a single from it. bhyve -H -P -A -c 1 -m 2G -l com1,/dev/nmdm11A -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap11 -s 4,ahci-hd,/dev/ggate1 FB11 & bhyvectl --vm=FB11 --get-stats Connect to the VM cu -l /dev/nmdm11B And that'll give you a bhyve VM running on an RBD image over ggate. In the installer I tested reading from the bootdisk: root@:/ # dd if=/dev/ada0 of=/dev/null bs=32M 21+1 records in 21+1 records out 734077952 bytes transferred in 5.306260 secs (138341865 bytes/sec) which is a nice 138Mb/sec. Hope the demonstration does work out tomorrow. --WjW *** Donald Knuth - The Patron Saint of Yak Shaves (http://yakshav.es/the-patron-saint-of-yakshaves/) Excerpts: In 2015, I gave a talk in which I called Donald Knuth the Patron Saint of Yak Shaves. The reason is that Donald Knuth achieved the most perfect and long-running yak shave: TeX. I figured this is worth repeating. How to achieve the ultimate Yak Shave The ultimate yak shave is the combination of improbable circumstance, the privilege to be able to shave at your hearts will and the will to follow things through to the end. Here's the way it was achieved with TeX. The recount is purely mine, inaccurate and obviously there for fun. I'll avoid the most boring facts that everyone always tells, such as why Knuth's checks have their own Wikipedia page. Community Shaving is Best Shaving Since the release of TeX, the community has been busy working on using it as a platform. If you ever downloaded the full TeX distribution, please bear in mind that you are downloading the amassed work of over 40 years, to make sure that each and every TeX document ever written builds. We're talking about documents here. But mostly, two big projects sprung out of that. The first is LaTeX by Leslie Lamport. Lamport is a very productive researcher, famous for research in formal methods through TLA+ and also known laying groundwork for many distributed algorithms. LaTeX is based on the idea of separating presentation and content. It is based around the idea of document classes, which then describe the way a certain document is laid out. Think Markdown, just much more complex. The second is ConTeXt, which is far more focused on fine grained layout control. The Moral of the Story Whenever you feel like “can't we just replace this whole thing, it can't be so hard” when handling TeX, don't forget how many years of work and especially knowledge were poured into that system. Typesetting isn't the most popular knowledge around programmers. Especially see it in the context of the space it is in: they can't remove legacy. Ever. That would break documents. TeX is also not a programming language. It might resemble one, but mostly, it should be approached as a typesetting system first. A lot of it's confusing lingo gets much better then. It's not programming lingo. By approaching TeX with an understanding for its history, a lot of things can be learned from it. And yes, a replacement would be great, but it would take ages. In any case, I hope I thoroughly convinced you why Donald Knuth is the Patron Saint of Yak Shaves. Extra Credits This comes out of a enjoyable discussion with [Arne from Lambda Island](https://lambdaisland.com/https://lambdaisland.com/, who listened and said “you should totally turn this into a talk”. Vincent's trip to EuroBSDCon 2017 (http://www.vincentdelft.be/post/post_20171016) My euroBSDCon 2017 Posted on 2017-10-16 09:43:00 from Vincent in Open Bsd Let me just share my feedback on those 2 days spent in Paris for the EuroBSDCon. My 1st BSDCon. I'm not a developer, contributor, ... Do not expect to improve your skills with OpenBSD with this text :-) I know, we are on October 16th, and the EuroBSDCon of Paris was 3 weeks ago :( I'm not quick !!! Sorry for that Arrival at 10h, I'm too late for the start of the key note. The few persons behind a desk welcome me by talking in Dutch, mainly because of my name. Indeed, Delft is a city in Netherlands, but also a well known university. I inform them that I'm from Belgium, and the discussion moves to the fact the Fosdem is located in Brussels. I receive my nice T-shirt white and blue, a bit like the marine T-shirts, but with the nice EuroBSDCon logo. I'm asking where are the different rooms reserved for the BSD event. We have 1 big on the 1st floor, 1 medium 1 level below, and 2 smalls 1 level above. All are really easy to access. In this entrance we have 4 or 5 tables with some persons representing their company. Those are mainly the big sponsors of the event providing details about their activity and business. I discuss a little bit with StormShield and Gandi. On other tables people are selling BSD t-shirts, and they will quickly be sold. "Is it done yet ?" The never ending story of pkg tools In the last Fosdem, I've already hear Antoine and Baptiste presenting the OpenBSD and FreeBSD battle, I decide to listen Marc Espie in the medium room called Karnak. Marc explains that he has rewritten completely the pkg_add command. He explains that, at contrario with other elements of OpenBSD, the packages tools must be backward compatible and stable on a longer period than 12 months (the support period for OpenBSD). On the funny side, he explains that he has his best idea inside his bath. Hackathons are also used to validate some ideas with other OpenBSD developers. All in all, he explains that the most time consuming part is to imagine a good solution. Coding it is quite straightforward. He adds that better an idea is, shorter the implementation will be. A Tale of six motherboards, three BSDs and coreboot After the lunch I decide to listen the talk about Coreboot. Indeed, 1 or 2 years ago I had listened the Libreboot project at Fosdem. Since they did several references to Coreboot, it's a perfect occasion to listen more carefully to this project. Piotr and Katazyba Kubaj explains us how to boot a machine without the native Bios. Indeed Coreboot can replace the bios, and de facto avoid several binaries imposed by the vendor. They explain that some motherboards are supporting their code. But they also show how difficult it is to flash a Bios and replace it by Coreboot. They even have destroyed a motherboard during the installation. Apparently because the power supply they were using was not stable enough with the 3v. It's really amazing to see that open source developers can go, by themselves, to such deep technical level. State of the DragonFly's graphics stack After this Coreboot talk, I decide to stay in the room to follow the presentation of Fran?ois Tigeot. Fran?ois is now one of the core developer of DrangonflyBSD, an amazing BSD system having his own filesystem called Hammer. Hammer offers several amazing features like snapshots, checksum data integrity, deduplication, ... Francois has spent his last years to integrate the video drivers developed for Linux inside DrangonflyBSD. He explains that instead of adapting this code for the video card to the kernel API of DrangonflyBSD, he has "simply" build an intermediate layer between the kernel of DragonflyBSD and the video drivers. This is not said in the talk, but this effort is very impressive. Indeed, this is more or less a linux emulator inside DragonflyBSD. Francois explains that he has started with Intel video driver (drm/i915), but now he is able to run drm/radeon quite well, but also drm/amdgpu and drm/nouveau. Discovering OpenBSD on AWS Then I move to the small room at the upper level to follow a presentation made by Laurent Bernaille on OpenBSD and AWS. First Laurent explains that he is re-using the work done by Antoine Jacoutot concerning the integration of OpenBSD inside AWS. But on top of that he has integrated several other Open Source solutions allowing him to build OpenBSD machines very quickly with one command. Moreover those machines will have the network config, the required packages, ... On top of the slides presented, he shows us, in a real demo, how this system works. Amazing presentation which shows that, by putting the correct tools together, a machine builds and configure other machines in one go. OpenBSD Testing Infrastructure Behind bluhm.genua.de Here Jan Klemkow explains us that he has setup a lab where he is able to run different OpenBSD architectures. The system has been designed to be able to install, on demand, a certain version of OpenBSD on the different available machines. On top of that a regression test script can be triggered. This provides reports showing what is working and what is not more working on the different machines. If I've well understood, Jan is willing to provide such lab to the core developers of OpenBSD in order to allow them to validate easily and quickly their code. Some more effort is needed to reach this goal, but with what exists today, Jan and his colleague are quite close. Since his company is using OpenBSD business, to his eyes this system is a "tit for tat" to the OpenBSD community. French story on cybercrime Then comes the second keynote of the day in the big auditorium. This talk is performed by the colonel of french gendarmerie. Mr Freyssinet, who is head of the Cyber crimes unit inside the Gendarmerie. Mr Freyssinet explains that the "bad guys" are more and more volatile across countries, and more and more organized. The small hacker in his room, alone, is no more the reality. As a consequence the different national police investigators are collaborating more inside an organization called Interpol. What is amazing in his talk is that Mr Freyssinet talks about "Crime as a service". Indeed, more and more hackers are selling their services to some "bad and temporary organizations". Social event It's now time for the famous social event on the river: la Seine. The organizers ask us to go, by small groups, to a station. There is a walk of 15 minutes inside Paris. Hopefully the weather is perfect. To identify them clearly several organizers takes a "beastie fork" in their hands and walk on the sidewalk generating some amazing reactions from some citizens and toursits. Some of them recognize the Freebsd logo and ask us some details. Amazing :-) We walk on small and big sidewalks until a small stair going under the street. There, we have a train station a bit like a metro station. 3 stations later they ask us to go out. We walk few minutes and come in front of a boat having a double deck: one inside, with nice tables and chairs and one on the roof. But the crew ask us to go up, on the second deck. There, we are welcome with a glass of wine. The tour Eiffel is just at few 100 meters from us. Every hour the Eiffel tower is blinking for 5 minutes with thousands of small lights. Brilliant :-) We see also the "statue de la libertee" (the small one) which is on a small island in the middle of the river. During the whole night the bar will be open with drinks and some appetizers, snacks, ... Such walking diner is perfect to talk with many different persons. I've discussed with several persons just using BSD, they are not, like me, deep and specialized developers. One was from Switzerland, another one from Austria, and another one from Netherlands. But I've also followed a discussion with Theo de Raadt, several persons of the FreeBSD foundation. Some are very technical guys, other just users, like me. But all with the same passion for one of the BSD system. Amazing evening. OpenBSD's small steps towards DTrace (a tale about DDB and CTF) On the second day, I decide to sleep enough in order to have enough resources to drive back to my home (3 hours by car). So I miss the 1st presentations, and arrive at the event around 10h30. Lot of persons are already present. Some faces are less "fresh" than others. I decide to listen to Dtrace in OpenBSD. After 10 minutes I am so lost into those too technical explainations, that I decide to open and look at my PC. My OpenBSD laptop is rarely leaving my home, so I've never had the need to have a screen locking system. In a crowded environment, this is better. So I was looking for a simple solution. I've looked at how to use xlock. I've combined it with the /ets/apm/suspend script, ... Always very easy to use OpenBSD :-) The OpenBSD web stack Then I decide to follow the presentation of Michael W Lucas. Well know person for his different books about "Absolute OpenBSD", Relayd", ... Michael talks about the httpd daemon inside OpenBSD. But he also present his integration with Carp, Relayd, PF, FastCGI, the rules based on LUA regexp (opposed to perl regexp), ... For sure he emphasis on the security aspect of those tools: privilege separation, chroot, ... OpenSMTPD, current state of affairs Then I follow the presentation of Gilles Chehade about the OpenSMTPD project. Amazing presentation that, on top of the technical challenges, shows how to manage such project across the years. Gilles is working on OpenSMTPD since 2007, thus 10 years !!!. He explains the different decisions they took to make the software as simple as possible to use, but as secure as possible, too: privilege separation, chroot, pledge, random malloc, ? . The development starts on BSD systems, but once quite well known they received lot of contributions from Linux developers. Hoisting: lessons learned integrating pledge into 500 programs After a small break, I decide to listen to Theo de Raadt, the founder of OpenBSD. In his own style, with trekking boots, shorts, backpack. Theo starts by saying that Pledge is the outcome of nightmares. Theo explains that the book called "Hacking blind" presenting the BROP has worried him since few years. That's why he developed Pledge as a tool killing a process as soon as possible when there is an unforeseen behavior of this program. For example, with Pledge a program which can only write to disk will be immediately killed if he tries to reach network. By implementing Pledge in the +-500 programs present in the "base", OpenBSD is becoming more secured and more robust. Conclusion My first EuroBSDCon was a great, interesting and cool event. I've discussed with several BSD enthusiasts. I'm using OpenBSD since 2010, but I'm not a developer, so I was worried to be "lost" in the middle of experts. In fact it was not the case. At EuroBSDCon you have many different type of enthusiasts BSD's users. What is nice with the EuroBSDCon is that the organizers foresee everything for you. You just have to sit and listen. They foresee even how to spend, in a funny and very cool attitude, the evening of Saturday. > The small draw back is that all of this has a cost. In my case the whole weekend cost me a bit more than 500euro. Based on what I've learned, what I've saw this is very acceptable price. Nearly all presentations I saw give me a valuable input for my daily job. For sure, the total price is also linked to my personal choice: hotel, parking. And I'm surely biased because I'm used to go to the Fosdem in Brussels which cost nothing (entrance) and is approximately 45 minutes of my home. But Fosdem is not the same atmosphere and presentations are less linked to my daily job. I do not regret my trip to EuroBSDCon and will surely plan other ones. Beastie Bits Important munitions lawyering (https://www.jwz.org/blog/2017/10/important-munitions-lawyering/) AsiaBSDCon 2018 CFP is now open, until December 15th (https://2018.asiabsdcon.org/) ZSTD Compression for ZFS by Allan Jude (https://www.youtube.com/watch?v=hWnWEitDPlM&feature=share) NetBSD on Allwinner SoCs Update (https://blog.netbsd.org/tnf/entry/netbsd_on_allwinner_socs_update) *** Feedback/Questions Tim - Creating Multi Boot USB sticks (http://dpaste.com/0FKTJK3#wrap) Nomen - ZFS Questions (http://dpaste.com/1HY5MFB) JJ - Questions (http://dpaste.com/3ZGNSK9#wrap) Lars - Hardening Diffie-Hellman (http://dpaste.com/3TRXXN4) ***
Allan reports on his trip to BSD Taiwan, new versions of Lumina and GhostBSD are here, a bunch of OpenBSD p2k17 hackathon reports. This episode was brought to you by Headlines Allan's Trip Report from BSD Taiwan (https://bsdtw.org/) BSD TW and Taiwan in general was a fun and interesting experience I arrived Thursday night and took the high speed train to Taipei main station, and then got on the Red line subway to the venue. The dorm rooms were on par with BSDCan, except the mattress was better. I spent Friday with a number of other FreeBSD developers doing touristy things. We went to Taipei 101, the world's tallest building from 2004 - 2010. It also features the world's fastest elevator (2004 - 2016), traveling at 60.6 km/h and transporting passengers from the 5th to 89th floor in 37 seconds. We also got to see the “tuned mass damper”, a 660 tonne steel pendulum suspended between the 92nd and 87th floors. This device resists the swaying of the building caused by high winds. There are interesting videos on display beside the damper, of its reaction during recent typhoons and earthquakes. The Taipei 101 building sits just 200 meters from a major fault line. Then we had excellent dumplings for lunch After walking around the city for a few more hours, we retired to a pub to escape the heat of the sunny Friday afternoon. Then came the best part of each day in Taipei, dinner! We continued our efforts to cause a nation wide shortage of dumplings Special thanks to Scott Tsai (https://twitter.com/scottttw) who took detailed notes for each of the presentations Saturday marked the start of the conference: Arun Thomas provided background and then a rundown of what is happening with the RISC-V architecture. Notes (https://docs.google.com/document/d/1yrnhNTHaMDr4DG-iviXN0O9NES9Lmlc7sWVQhnios6g/edit#heading=h.kcm1n3yzl35q) George Neville-Neil talked about using DTrace in distributed systems as an in-depth auditing system (who did what to whom and when). Notes (https://docs.google.com/document/d/1qut6tMVF8NesrGHd6bydLDN-aKBdXMgHx8Vp3_iGKjQ/edit#heading=h.qdghsgk1bgtl) Baptiste Daroussin presented Poudrière image, an extension of everyone's favourite package building system, to build custom images of FreeBSD. There was discussion of making this generate ZFS based images as well, making it mesh very well with my talk the next day. Notes (https://docs.google.com/document/d/1LceXj8IWJeTRHp9KzOYy8tpM00Fzt7fSN0Gw83B9COE/edit#heading=h.incfzi6bnzxr) Brooks Davis presented his work on an API design for a replacement for mmap. It started with a history of address space management in the BSD family of operating systems going all the way back to the beginning. This overview of the feature and how it evolved filled in many gaps for me, and showed why the newer work would be beneficial. The motivation for the work includes further extensions to support the CHERI hardware platform. Notes (https://docs.google.com/document/d/1LceXj8IWJeTRHp9KzOYy8tpM00Fzt7fSN0Gw83B9COE/edit#heading=h.incfzi6bnzxr) Johannes M Dieterich gave an interesting presentation about using FreeBSD and GPU acceleration for high performance computing. One of the slides showed that amd64 has taken almost the entire market for the top 500 super computers, and that linux dominates the list, with only a few remaining non-linux systems. Sadly, at the supercomputing conference the next week, it was announced that linux has achieved 100% saturation of the top 500 super computers list. Johannes detailed the available tools, what ports are missing, what changes should be made to the base system (mostly OpenMP), and generally what FreeBSD needs to do to become a player in the supercomputer OS market. Johannes' perspective is interesting, as he is a computational chemist, not a computer scientist. Those interested in improving the numerical libraries and GPU acceleration frameworks on FreeBSD should join the ports team. Notes (https://docs.google.com/document/d/1uaJiqtPk8WetST6_GnQwIV49bj790qx7ToY2BHC9zO4/edit#heading=h.nvsz1n6w3gyq) The final talk of the day was Peter Grehan, who spoke about how graphics support in bhyve came to be. He provided a history of how the feature evolved, and where it stands today. Notes (https://docs.google.com/document/d/1LqJQJUwdUwWZ0n5KwCH1vNI8jiWGJlI1j0It3mERN80/edit#heading=h.sgeixwgz7bjs) Afterwards, we traveled as a group to a large restaurant for dinner. There was even Mongolian Vodka, provided by Ganbold Tsagaankhuu of the FreeBSD project. Sunday: The first talk of the day Sunday was mine. I presented “ZFS: Advanced Integration”, mostly talking about how boot environments work, and the new libbe and be(1) tools that my GSoC student Kyle Kneitinger created to manage them. I talked about how they can be used for laptop and developer systems, but also how boot environments can be used to replace nanobsd for appliances (as already done in FreeNAS and pfSense). I also presented about zfsbootcfg (zfs nextboot), and some future extensions to it to make it even more useful in appliance type workloads. I also provided a rundown of new developments out of the ZFS developer summit, two weeks previous. Notes (https://docs.google.com/document/d/1Blh3Dulf0O91A0mwv34UnIgxRZaS_0FU2lZ41KRQoOU/edit#heading=h.gypim387e8hy) Theo de Raadt presented “Mitigations and other real Security Features”, and made his case for changing to a ‘fail closed' mode of interoperability. Computer's cannot actually self heal, so lets stop pretending that they can. Notes (https://docs.google.com/document/d/1fFHzlxJjbHPsV9t_Uh3PXZnXmkapAK5RkJsfaHki7kc/edit#heading=h.192e4lmbl70c) Ruslan Bukin talked about doing the port of FreeBSD for RISC-V and writing the Device Drivers. Ruslan walked through the process step by step, leading members of the audience to suggest he turn it into a developer's handbook article, explaining how to do the initial bringup on new hardware. Ruslan also showed off a FreeBSD/MIPS board he designed himself and had manufactured in China. Notes (https://docs.google.com/document/d/1kRhRr3O3lQ-0dS0kYF0oh_S0_zFufEwrdFjG1QLyk8Y/edit#heading=h.293mameym7w1) Mariusz Zaborski presented Case studies on sandboxing the base system with Capsicum. He discussed the challenges encountered as existing programs are modified to sandbox them, and recent advancements in the debugging tools available during that process. Mariusz also discussed the Casper service at length, including the features that are planned for 2018 and onwards. Notes (https://docs.google.com/document/d/1_0BpAE1jGr94taUlgLfSWlJOYU5II9o7Y3ol0ym1eZQ/edit#heading=h.xm9mh7dh6bay) The final presentation of the day was Mark Johnston on Memory Management Improvements in FreeBSD 12.0. This talk provided a very nice overview of the memory management system in FreeBSD, and then detailed some of the recent improvements. Notes (https://docs.google.com/document/d/1gFQXxsHM66GQGMO4-yoeFRTcmOP4NK_ujVFHIQJi82U/edit#heading=h.uirc9jyyti7w) The conference wrapped up with the Work-in-Progress session, including updates on: multi-device-at-once GELI attach, MP-safe networking on NetBSD, pkgsrc, NetBSD in general, BSD on Microsoft Azure, Mothra (send-pr for bugzilla), BSDMizer a machine learning compiler optimizer, Hyperledger Sawtooth (blockchain), and finally VIMAGE and pf testing on FreeBSD. Notes (https://docs.google.com/document/d/1miHZEPrqrpCTh8JONmUKWDPYUmTuG2lbsVrWDtekvLc/edit#heading=h.orhedpjis5po) Group Photo (https://pbs.twimg.com/media/DOh1txnVoAAFKAa.jpg:large) BSDTW was a great conference. They are still considering if it should be an annual thing, trade off every 2nd year with AsiaBSDCon, or something else. In order to continue, BSD Taiwan requires more organizers and volunteers. They have regular meetups in Taipei if you are interested in getting involved. *** Lumina 1.4.0 released (https://lumina-desktop.org/version-1-4-0-released/) The Lumina Theme Engine (and associated configuration utility) The Lumina theme engine is a new component of the “core” desktop, and provides enhanced theming capabilities for the desktop as well as all Qt5 applications. While it started out life as a fork of the “qt5ct” utility, it quickly grew all sorts of new features and functionality such as system-defined color profiles, modular theme components, and built-in editors/creators for all components. The backend of this engine is a standardized theme plugin for the Qt5 toolkit, so that all Qt5 applications will now present a unified appearance (if the application does not enforce a specific appearance/theme of it's own). Users of the Lumina desktop will automatically have this plugin enabled: no special action is required. Please note that the older desktop theme system for Lumina has been rendered obsolete by the new engine, but a settings-conversion path has already been implemented which should transition your current settings to the new engine the first time you login to Lumina 1.4.0. Custom themes for the older system may not be converted though, but it is trivial to copy/paste any custom stylesheets from the old system into the editor for the new theme engine to register/re-apply them as desired. Lumina-Themes Repository I also want to give a shout-out to the trueos/lumina-themes github repository contributors. All of the wallpapers in the 1.4.0 screenshots I posted come from that package, and they are working on making more wallpapers, color palettes, and desktop styles for use with the Lumina Theme Engine. If your operating system does not currently provide a package for lumina-themes, I highly recommend that you make one as soon as possible! The Lumina PDF Viewer (lumina-pdf) This is a new, stand-alone desktop utility for viewing/printing/presenting PDF documents. It uses the poppler-qt5 library in the backend for rendering the document, but uses multi-threading in many ways (such as to speed up the loading of pages) to give the user a nice, streamlined utility for viewing PDF documents. There is also built-in presentation functionality which allows users to easily cast the document to a separate screen without mucking about in system menus or configuration utilities. Lumina PDF Viewer (1.4.0) Important Packaging Changes One significant change of note for people who are packaging Lumina for their particular operating system is that the minimum supported versions of Qt for Lumina have been changed with this release: lumina-core: Qt 5.4+ lumina-mediaplayer: Qt 5.7+ Everything else: Qt 5.2+ Of course, using the latest version of the Qt5 libraries is always recommended. When packaging for Linux distributions, the theme engine also requires the availability of some of the “-dev” packages for Qt itself when compiling the theme plugin. For additional information (specifically regarding Ubuntu builds), please take a look at a recent ticket on the Lumina repository. + The new lumina-pdf utility requires the availability of the “poppler-qt5” library. The includes for this library on Ubuntu 17.10 were found to be installed outside of the normal include directories, so a special rule for it was added to our OS-Detect file in the Lumina source tree. If your particular operating system also places the the poppler include files in a non-standard place, please patch that file or send us the information and we can add more special rules for your particular OS. Other Changes of Note (in no particular order) lumina-config: Add a new page for changing audio theme (login, logout, low battery) Add option to replace fluxbox with some other WM (with appropriate warnings) Have the “themes” page redirect to launching the Lumina theme engine configuration utility. start-lumina-desktop: Auto-detect the active X11 displays and create a new display for the Lumina session (prevent conflict with prior graphical sessions). Add a process-failure counter & restart mechanism. This is particularly useful for restarting Fluxbox from time to time (such as after any monitor addition/removal) lumina-xconfig: Restart fluxbox after making any monitor changes with xrandr. This ensures a more reliable session. Implement a new 2D monitor layout mechanism. This allows for the placement of monitors anywhere in the X/Y plane, with simplification buttons for auto-tiling the monitors in each dimension based on their current location. Add the ability to save/load monitor profiles. Distinguish between the “default” monitor arrangement and the “current” monitor arrangement. Allow the user to set the current arrangement as the new default. lumina-desktop: Completely revamp the icon loading mechanisms so it should auto-update when the theme changes. Speed up the initialization of the desktop quite a bit. Prevent loading/probing files in the “/net/” path for existence (assume they exist in the interest of providing shortcuts). On FreeBSD, these are special paths that actually pause the calling process in order to mount/load a network share before resuming the process, and can cause significant “hangs” in the desktop process. Add the ability to take a directory as a target for the wallpaper. This will open/probe the directory for any existing image files that it can use as a wallpaper and randomly select one. Remove the popup dialog prompting about system updates, and replace it with new “Restart (with updates)” buttons on the appropriate menus/windows instead. If no wallpapers selection is provided, try to use the “lumina-nature” wallpaper directory as the default, otherwise fall back on the original default wallpaper if the “lumina-themes” package is not installed. lumina-open: Make the *.desktop parsing a bit more flexible regarding quoted strings where there should not be any. If selecting which application to use, only overwrite the user-default app if the option is explicitly selected. lumina-fileinfo: Significant cleanup of this utility. Now it can be reliably used for creating/registering XDG application shortcuts. Add a whole host of new ZFS integrations: If a ZFS dataset is being examined, show all the ZFS properties for that dataset. If the file being examined exists within ZFS snapshots, show all the snapshots of the file lumina-fm: Significant use of additional multi-threading. Makes the loading of directories much faster (particularly ones with image files which need thumbnails) Add detection/warning when running as root user. Also add an option to launch a new instance of lumina-fm as the root user. [FreeBSD/TrueOS] Fix up the detection of the “External Devices” list to also list available devices for the autofs system. Fix up some drag and drop functionality. Expose the creation, extraction, and insertion of files into archives (requires lumina-archiver at runtime) Expand the “Open With” option into a menu of application suggestions in addition to the “Other” option which runs “lumina-open” to find an application. Provide an option to set the desktop wallpaper to the selected image file(s). (If the running desktop session is Lumina). lumina-mediaplayer: Enable the ability to playback local video files. (NOTE: If Qt5 is set to use the gstreamer multimedia backend, make sure you have the “GL” plugin installed for smooth video playback). lumina-archiver: Add CLI flags for auto-archive and auto-extract. This allows for programmatic/scriptable interactions with archives. That is not mentioning all of the little bugfixes, performance tweaks, and more that are also included in this release. *** The strongest KASLR, ever? (https://blog.netbsd.org/tnf/entry/the_strongest_kaslr_ever) Re: amd64: kernel aslr support (https://mail-index.netbsd.org/tech-kern/2017/11/14/msg022594.html) So, I did it. Now the kernel sections are split in sub-blocks, and are all randomized independently. See my drawing [1]. What it means in practice, is that Kernel ASLR is much more difficult to defeat: a cache attack will at most allow you to know that a given range is mapped as executable for example, but you don't know which sub-block of .text it is; a kernel pointer leak will at most allow you to reconstruct the layout of one sub-block, but you don't know the layout and address of the remaining blocks, and there can be many. The size and number of these blocks is controlled by the split-by-file parameter in Makefile.amd64. Right now it is set to 2MB, which produces a kernel with ~23 allocatable (ie useful at runtime) sections, which is a third of the total number supported (BTSPACENSEGS = 64). I will probably reduce this parameter a bit in the future, to 1.5MB, or even 1MB. All of that leaves us with about the most advanced KASLR implementation available out there. There are ways to improve it even more, but you'll have to wait a few weeks for that. If you want to try it out you need to make sure you have the latest versions of GENERICKASLR / prekern / bootloader. The instructions are still here, and haven't changed. Initial design As I said in the previous episode, I added in October a Kernel ASLR implementation in NetBSD for 64bit x86 CPUs. This implementation would randomize the location of the kernel in virtual memory as one block: a random VA would be chosen, and the kernel ELF sections would be mapped contiguously starting from there. This design had several drawbacks: one leak, or one successful cache attack, could be enough to reconstruct the layout of the entire kernel and defeat KASLR. NetBSD's new KASLR design significantly improves this situation. New design In the new design, each kernel ELF section is randomized independently. That is to say, the base addresses of .text, .rodata, .data and .bss are not correlated. KASLR is already at this stage more difficult to defeat, since you would need a leak or cache attack on each of the kernel sections in order to reconstruct the in-memory kernel layout. Then, starting from there, several techniques are used to strengthen the implementation even more. Sub-blocks The kernel ELF sections are themselves split in sub-blocks of approximately 1MB. The kernel therefore goes from having: { .text .rodata .data .bss } to having { .text .text.0 .text.1 ... .text.i .rodata .rodata.0 ... .rodata.j ... .data ...etc } As of today, this produces a kernel with ~33 sections, each of which is mapped at a random address and in a random order. This implies that there can be dozens of .text segments. Therefore, even if you are able to conduct a cache attack and determine that a given range of memory is mapped as executable, you don't know which sub-block of .text it is. If you manage to obtain a kernel pointer via a leak, you can at most guess the address of the section it finds itself in, but you don't know the layout of the remaining 32 sections. In other words, defeating this KASLR implementation is much more complicated than in the initial design. Higher entropy Each section is put in a 2MB-sized physical memory chunk. Given that the sections are 1MB in size, this leaves half of the 2MB chunk unused. Once in control, the prekern shifts the section within the chunk using a random offset, aligned to the ELF alignment constraint. This offset has a maximum value of 1MB, so that once shifted the section still resides in its initial 2MB chunk: The prekern then maps these 2MB physical chunks at random virtual addresses; but addresses aligned to 2MB. For example, the two sections in Fig. A will be mapped at two distinct VAs: There is a reason the sections are shifted in memory: it offers higher entropy. If we consider a .text.i section with a 64byte ELF alignment constraint, and give a look at the number of possibilities for the location of the section in memory: The prekern shifts the 1MB section in its 2MB chunk, with an offset aligned to 64 bytes. So there are (2MB-1MB)/(64B)=214 possibilities for the offset. Then, the prekern uses a 2MB-sized 2MB-aligned range of VA, chosen in a 2GB window. So there are (2GB-2MB)/(2MB)=210-1 possibilities for the VA. Therefore, there are 214x(210-1)˜224 possible locations for the section. As a comparison with other systems: OS # of possibilities Linux 2^6 MacOS 2^8 Windows 2^13 NetBSD 2^24 Of course, we are talking about one .text.i section here; the sections that will be mapped afterwards will have fewer location possibilities because some slots will be already occupied. However, this does not alter the fact that the resulting entropy is still higher than that of the other implementations. Note also that several sections have an alignment constraint smaller than 64 bytes, and that in such cases the entropy is even higher. Large pages There is also a reason we chose to use 2MB-aligned 2MB-sized ranges of VAs: when the kernel is in control and initializes itself, it can now use large pages to map the physical 2MB chunks. This greatly improves memory access performance at the CPU level. Countermeasures against TLB cache attacks With the memory shift explained above, randomness is therefore enforced at both the physical and virtual levels: the address of the first page of a section does not equal the address of the section itself anymore. It has, as a side effect, an interesting property: it can mostly mitigate TLB cache attacks. Such attacks operate at the virtual-page level; they will allow you to know that a given large page is mapped as executable, but you don't know where exactly within that page the section actually begins. Strong? This KASLR implementation, which splits the kernel in dozens of sub-blocks, randomizes them independently, while at the same time allowing for higher entropy in a way that offers large page support and some countermeasures against TLB cache attacks, appears to be the most advanced KASLR implementation available publicly as of today. Feel free to prove me wrong, I would be happy to know! WIP Even if it is in a functional state, this implementation is still a work in progress, and some of the issues mentioned in the previous blog post haven't been addressed yet. But feel free to test it and report any issue you encounter. Instructions on how to use this implementation can still be found in the previous blog post, and haven't changed since. See you in the next episode! News Roundup GhostBSD 11.1 Finally Ready and Available! (http://www.ghostbsd.org/11.1_release_announcement) Screenshots (https://imgur.com/a/Mu8xk) After a year of development, testing, debugging and working on our software package repository, we are pleased to announce the release of GhostBSD 11.1 is now available on 64-bit(amd64) architecture with MATE and XFCE Desktop on direct and torrent download. With 11.1 we drop 32-bit i386 supports, and we currently maintain our software packages repository for more stability. What's new on GhostBSD 11.1 GhostBSD software repository Support VMware Workstation Guest Features New UFS full disk mirroring option on the installer New UFS full disk MBR and GPT option on the installer New UFS full disk swap size option on the installer Whisker Menu as default Application menu on XFCE All software developed by GhostBSD is now getting updated ZFS configuration for disk What has been fixed on 11.1? Fix XFCE sound plugin Installer ZFS configuration file setting Installer ZFS setup appears to be incomplete The installer was not listing ZFS disk correctly. The installer The partition list was not deleted when pressing back XFCE and MATE shutdown/suspend/hibernate randomly missing Clicking 'GhostBSD Bugs' item in the Main menu -> 'System Tools' brings up 'Server not found' page XFCE installation - incorrect keyboard layout Locale setting not filling correctly Update Station tray icon The image checksum's, hybrid ISO(DVD, USB) images are available at GhostBSD (http://www.ghostbsd.org/download). *** p2k17 Hackathon Reports p2k17 Hackathon Report: Matthias Kilian on xpdf, haskell, and more (https://undeadly.org/cgi?action=article;sid=20171107034258) p2k17 Hackathon Report: Herzliche grusse vom Berlin (espie@ on mandoc, misc packages progress) (https://undeadly.org/cgi?action=article;sid=20171107185122) p2k17 Hackathon Report: Paul Irofti (pirofti@) on hotplugd(8), math ports, xhci(4) and other kernel advancements (https://undeadly.org/cgi?action=article;sid=20171107225258) p2k17 Hackathon report: Jeremy Evans on ruby progress, postgresql and webdriver work (https://undeadly.org/cgi?action=article;sid=20171108072117) p2k17 Hackathon report: Christian Weisgerber on random devices, build failures and gettext (https://undeadly.org/cgi?action=article;sid=20171109171447) p2k17 Hackathon report: Sebastian Reitenbach on Puppet progress (https://undeadly.org/cgi?action=article;sid=20171110124645) p2k17 Hackathon Report: Anthony J. Bentley on firmware, games and securing pkg_add runs (https://undeadly.org/cgi?action=article;sid=20171110124656) p2k17 Hackathon Report: Landry Breuil on Mozilla things and much more (https://undeadly.org/cgi?action=article;sid=20171113091807) p2k17 Hackathon report: Florian Obser on network stack progress, kernel relinking and more (https://undeadly.org/cgi?action=article;sid=20171113235334) p2k17 Hackathon report: Antoine Jacoutot on ports+packages progress (https://undeadly.org/cgi?action=article;sid=20171120075903) *** TrueOS Talks Tech and Open Source at Pellissippi State (https://www.trueos.org/blog/trueos-talks-tech-open-source-pellissippi-state/) Ken Moore of the TrueOS project presented a talk to the AITP group at Pellissippi State today entitled “It's A Unix(-like) system? An Introduction to TrueOS and Open source”. Joshua Smith of the TrueOS project was also in attendance. We were happy to see a good attendance of about 40 individuals that came to hear more about TrueOS and how we continue to innovate along with the FreeBSD project. Many good questions were raised about development, snapshots, cryptocurrency, and cyber-security. We've included a copy of the slides if you'd like to have a look at the talk on open source. We'd like to offer a sincere thanks to everyone who attended and offer an extended invitation for you to join us at our KnoxBUG group on October 30th @ the iXsystems offices! We hope to see you soon! Open Source Talk – Slideshare PDF (https://web.trueos.org/wp-content/uploads/2017/10/Open-Source-Talk.pdf) KnoxBug - Lumina Rising : Challenging Desktop Orthodoxy (http://knoxbug.org/content/octobers-talk-available-youtube) Ken gave his talk about the new Lumina 2.0 Window Manager that he gave at Ohio LinuxFest 2017 KnoxBUG October 2017 (https://youtu.be/w3ZrqxLTnIU) (OLF 2017) Lumina Rising: Challenging Desktop Orthodoxy (https://www.slideshare.net/beanpole135/olf-2017-lumina-rising-challenging-desktop-orthodoxy) *** Official OpenBSD 6.2 CD set - the only one to be made! (https://undeadly.org/cgi?action=article;sid=20171118190325) Our dear friend Bob Beck (beck@) writes: So, again this release the tradition of making Theo do art has continued! Up for sale by auction to the highest bidder on Ebay is the only OpenBSD 6.2 CD set to be produced. The case and CD's feature the 6.2 artwork, custom drawn and signed by Theo. All proceeds to support OpenBSD Go have a look at the auction As with previous OpenBSD auctions, if you are not the successful bidder, we would like to encourage you to donate the equivalent of you highest bid to the project. The Auction (https://www.ebay.ca/itm/Official-OpenBSD-6-2-CD-Set/253265944606) *** Beastie Bits HAMMER2 userspace on Linux (http://lists.dragonflybsd.org/pipermail/users/2017-October/313646.html) OpenBSD Porting Workshop (now changed to January 3, 2018) (http://www.nycbug.org/index.cgi?action=view&id=10655) Matt Ahrens on when Native Encryption for ZFS will land (https://twitter.com/mahrens1/status/921204908094775296) The first successful build of OpenBSD base system (http://nanxiao.me/en/the-first-successful-build-of-openbsd-base-system/) KnoxBug November Meeting (https://www.meetup.com/KnoxBUG-BSD-Linux-and-FOSS-Users-Unite/events/245291204/) Absolute FreeBSD, 3rd Edition, pre-orders available (https://www.michaelwlucas.com/os/af3e) Feedback/Questions Jon - Jails and Networking (http://dpaste.com/2BEW0HB#wrap) Nathan - bhyve Provisioning (http://dpaste.com/1GHSYJS#wrap) Lian - OpenSSL jumping the Shark (http://dpaste.com/18P8D8C#wrap) Kim - Suggestions (http://dpaste.com/1VE0K9E#wrap) ***
We recap EuroBSDcon in Paris, tell the story behind a pf PR, and show you how to do screencasting with OpenBSD. This episode was brought to you by Headlines Recap of EuroBSDcon 2017 in Paris, France (https://2017.eurobsdcon.org) EuroBSDcon was held in Paris, France this year, which drew record numbers this year. With over 300 attendees, it was the largest BSD event I have ever attended, and I was encouraged by the higher than expected number of first time attendees. The FreeBSD Foundation held a board meeting on Wednesday afternoon with the members who were in Paris. Topics included future conferences (including a conference kit we can mail to people who want to represent FreeBSD) and planning for next year. The FreeBSD Devsummit started on Thursday at the beautiful Mozilla Office in Paris. After registering and picking up our conference bag, everyone gathered for a morning coffee with lots of handshaking and greeting. We then gathered in the next room which had a podium with microphone, screens as well as tables and chairs. After developers sat down, Benedict opened the devsummit with a small quiz about France for developers to win a Mogics Power Bagel (https://www.mogics.com/?page_id=3824). 45 developers participated and DES won the item in the end. After introductions and collecting topics of interest from everyone, we started with the Work in Progress (WIP) session. The WIP session had different people present a topic they are working on in 7 minute timeslots. Topics ranged from FreeBSD Forwarding Performance, fast booting options, and a GELI patch under review to attach multiple providers. See their slides on the FreeBSD wiki (https://wiki.freebsd.org/DevSummit/201709). After lunch, the FreeBSD Foundation gave a general update on staff and funding, as well as a more focused presentation about our partnership with Intel. People were interested to hear what was done so far and asked a few questions to the Intel representative Glenn Weinberg. After lunch, developers worked quietly on their own projects. The mic remained open and occasionally, people would step forward and gave a short talk without slides or motivated a discussion of common interest. The day concluded with a dinner at a nice restaurant in Paris, which allowed to continue the discussions of the day. The second day of the devsummit began with a talk about the CAM-based SDIO stack by Ilya Bakulin. His work would allow access to wifi cards/modules on embedded boards like the Raspberry Pi Zero W and similar devices as many of these are using SDIO for data transfers. Next up was a discussion and Q&A session with the FreeBSD core team members who were there (missing only Benno Rice, Kris Moore, John Baldwin, and Baptiste Daroussin, the latter being busy with conference preparations). The new FCP (FreeBSD community proposals) were introduced for those who were not at BSDCan this year and the hows and whys about it. Allan and I were asked to describe our experiences as new members of core and we encouraged people to run for core when the next election happens. After a short break, Scott Long gave an overview of the work that's been started on NUMA (Non-Uniform Memory Architecture), what the goals of the project are and who is working on it. Before lunch, Christian Schwarz presented his work on zrepl, a new ZFS replication solution he developed using Go. This sparked interest in developers, a port was started (https://reviews.freebsd.org/D12462) and people suggested to Christian that he should submit his talk to AsiaBSDcon and BSDCan next year. Benedict had to leave before lunch was done to teach his Ansible tutorial (which was well attended) at the conference venue. There were organized dinners, for those two nights, quite a feat of organization to fit over 100 people into a restaurant and serve them quickly. On Saturday, there was a social event, a river cruise down the Seine. This took the form of a ‘standing' dinner, with a wide selection of appetizer type dishes, designed to get people to walk around and converse with many different people, rather than sit at a table with the same 6-8 people. I talked to a much larger group of people than I had managed to at the other dinners. I like having both dinner formats. We would also like to thank all of the BSDNow viewers who attended the conference and made the point of introducing themselves to us. It was nice to meet you all. The recordings of the live video stream from the conference are available immediately, so you can watch the raw versions of the talks now: Auditorium Keynote 1: Software Development in the Age of Heroes (https://youtu.be/4iR8g9-39LM?t=179) by Thomas Pornin (https://twitter.com/BearSSLnews) Tuning FreeBSD for routing and firewalling (https://youtu.be/4iR8g9-39LM?t=1660) by Olivier Cochard-Labbé (https://twitter.com/ocochardlabbe) My BSD sucks less than yours, Act I (https://youtu.be/4iR8g9-39LM?t=7040) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_) My BSD sucks less than yours, Act II (https://youtu.be/4iR8g9-39LM?t=14254) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_) Reproducible builds on NetBSD (https://youtu.be/4iR8g9-39LM?t=23351) by Christos Zoulas Your scheduler is not the problem (https://youtu.be/4iR8g9-39LM?t=26845) by Martin Pieuchot Keynote 2: A French story on cybercrime (https://youtu.be/4iR8g9-39LM?t=30540) by Éric Freyssinet (https://twitter.com/ericfreyss) Case studies of sandboxing base system with Capsicum (https://youtu.be/jqdHYEH_BQY?t=731) by Mariusz Zaborski (https://twitter.com/oshogbovx) OpenBSD's small steps towards DTrace (a tale about DDB and CTF) (https://youtu.be/jqdHYEH_BQY?t=6030) by Jasper Lievisse Adriaanse The Realities of DTrace on FreeBSD (https://youtu.be/jqdHYEH_BQY?t=13096) by George Neville-Neil (https://twitter.com/gvnn3) OpenSMTPD, current state of affairs (https://youtu.be/jqdHYEH_BQY?t=16818) by Gilles Chehade (https://twitter.com/PoolpOrg) Hoisting: lessons learned integrating pledge into 500 programs (https://youtu.be/jqdHYEH_BQY?t=21764) by Theo de Raadt Keynote 3: System Performance Analysis Methodologies (https://youtu.be/jqdHYEH_BQY?t=25463) by Brendan Gregg (https://twitter.com/brendangregg) Closing Session (https://youtu.be/jqdHYEH_BQY?t=29355) Karnak “Is it done yet ?” The never ending story of pkg tools (https://youtu.be/1hjzleqGRYk?t=71) by Marc Espie (https://twitter.com/espie_openbsd) A Tale of six motherboards, three BSDs and coreboot (https://youtu.be/1hjzleqGRYk?t=7498) by Piotr Kubaj and Katarzyna Kubaj State of the DragonFly's graphics stack (https://youtu.be/1hjzleqGRYk?t=11475) by François Tigeot From NanoBSD to ZFS and Jails – FreeBSD as a Hosting Platform, Revisited (https://youtu.be/1hjzleqGRYk?t=16227) by Patrick M. Hausen Bacula – nobody ever regretted making a backup (https://youtu.be/1hjzleqGRYk?t=20069) by Dan Langille (https://twitter.com/DLangille) Never Lose a Syslog Message (https://youtu.be/qX0BS4P65cQ?t=325) by Alexander Bluhm Running CloudABI applications on a FreeBSD-based Kubernetes cluster (https://youtu.be/qX0BS4P65cQ?t=5647) by Ed Schouten (https://twitter.com/EdSchouten) The OpenBSD web stack (https://youtu.be/qX0BS4P65cQ?t=13255) by Michael W. Lucas (https://twitter.com/mwlauthor) The LLDB Debugger on NetBSD (https://youtu.be/qX0BS4P65cQ?t=16835) by Kamil Rytarowski What's in store for NetBSD 8.0? (https://youtu.be/qX0BS4P65cQ?t=21583) by Alistair Crooks Louxor A Modern Replacement for BSD spell(1) (https://youtu.be/6Nen6a1Xl7I?t=156) by Abhinav Upadhyay (https://twitter.com/abhi9u) Portable Hotplugging: NetBSD's uvm_hotplug(9) API development (https://youtu.be/6Nen6a1Xl7I?t=5874) by Cherry G. Mathew Hardening pkgsrc (https://youtu.be/6Nen6a1Xl7I?t=9343) by Pierre Pronchery (https://twitter.com/khorben) Discovering OpenBSD on AWS (https://youtu.be/6Nen6a1Xl7I?t=14874) by Laurent Bernaille (https://twitter.com/lbernail) OpenBSD Testing Infrastructure Behind bluhm.genua.de (https://youtu.be/6Nen6a1Xl7I?t=18639) by Jan Klemkow The school of hard knocks – PT1 (https://youtu.be/8wuW8lfsVGc?t=276) by Sevan Janiyan (https://twitter.com/sevanjaniyan) 7 years of maintaining firefox, and still looking ahead (https://youtu.be/8wuW8lfsVGc?t=5321) by Landry Breuil Branch VPN solution based on OpenBSD, OSPF, RDomains and Ansible (https://youtu.be/8wuW8lfsVGc?t=12385) by Remi Locherer Running BSD on AWS (https://youtu.be/8wuW8lfsVGc?t=15983) by Julien Simon and Nicolas David Getting started with OpenBSD device driver development (https://youtu.be/8wuW8lfsVGc?t=21491) by Stefan Sperling A huge thanks to the organizers, program committee, and sponsors of EuroBSDCon. Next year, EuroBSDcon will be in Bucharest, Romania. *** The story of PR 219251 (https://www.sigsegv.be//blog/freebsd/PR219251) The actual story I wanted Kristof to tell, the pf bug he fixed at the Essen Hackathon earlier this summer. As I threatened to do in my previous post, I'm going to talk about PR 219251 for a bit. The bug report dates from only a few months ago, but the first report (that I can remeber) actually came from Shawn Webb on Twitter, of all places Despite there being a stacktrace it took quite a while (nearly 6 months in fact) before I figured this one out. It took Reshad Patuck managing to distill the problem down to a small-ish test script to make real progress on this. His testcase meant that I could get core dumps and experiment. It also provided valuable clues because it could be tweaked to see what elements were required to trigger the panic. This test script starts a (vnet) jail, adds an epair interface to it, sets up pf in the jail, and then reloads the pf rules on the host. Interestingly the panic does not seem to occur if that last step is not included. Obviously not the desired behaviour, but it seems strange. The instances of pf in the jails are supposed to be separate. We try to fetch a counter value here, but instead we dereference a bad pointer. There's two here, so already we need more information. Inspection of the core dump reveals that the state pointer is valid, and contains sane information. The rule pointer (rule.ptr) points to a sensible location, but the data is mostly 0xdeadc0de. This is the memory allocator being helpful (in debug mode) and writing garbage over freed memory, to make use-after-free bugs like this one easier to find. In other words: the rule has been free()d while there was still a state pointing to it. Somehow we have a state (describing a connection pf knows about) which points to a rule which no longer exists. The core dump also shows that the problem always occurs with states and rules in the default vnet (i.e. the host pf instance), not one of the pf instances in one of the vnet jails. That matches with the observation that the test script does not trigger the panic unless we also reload the rules on the host. Great, we know what's wrong, but now we need to work out how we can get into this state. At this point we're going to have to learn something about how rules and states get cleaned up in pf. Don't worry if you had no idea, because before this bug I didn't either. The states keep a pointer to the rule they match, so when rules are changed (or removed) we can't just delete them. States get cleaned up when connections are closed or they time out. This means we have to keep old rules around until the states that use them expire. When rules are removed pfunlinkrule() adds then to the Vpfunlinkedrules list (more on that funny V prefix later). From time to time the pf purge thread will run over all states and mark the rules that are used by a state. Once that's done for all states we know that all rules that are not marked as in-use can be removed (because none of the states use it). That can be a lot of work if we've got a lot of states, so pfpurgethread() breaks that up into smaller chuncks, iterating only part of the state table on every run. We iterate over all of our virtual pf instances (VNETFOREACH()), check if it's active (for FreeBSD-EN-17.08, where we've seen this code before) and then check the expired states with pfpurgeexpiredstates(). We start at state 'idx' and only process a certain number (determined by the PFTMINTERVAL setting) states. The pfpurgeexpiredstates() function returns a new idx value to tell us how far we got. So, remember when I mentioned the odd V_ prefix? Those are per-vnet variables. They work a bit like thread-local variables. Each vnet (virtual network stack) keeps its state separate from the others, and the V_ variables use a pointer that's changed whenever we change the currently active vnet (say with CURVNETSET() or CURVNETRESTORE()). That's tracked in the 'curvnet' variable. In other words: there are as many Vpfvnetactive variables as there are vnets: number of vnet jails plus one (for the host system). Why is that relevant here? Note that idx is not a per-vnet variable, but we handle multiple pf instances here. We run through all of them in fact. That means that we end up checking the first X states in the first vnet, then check the second X states in the second vnet, the third X states in the third and so on and so on. That of course means that we think we've run through all of the states in a vnet while we really only checked some of them. So when pfpurgeunlinkedrules() runs it can end up free()ing rules that actually are still in use because pfpurgethread() skipped over the state(s) that actually used the rule. The problem only happened if we reloaded rules in the host, because the active ruleset is never free()d, even if there are no states pointing to the rule. That explains the panic, and the fix is actually quite straightforward: idx needs to be a per-vnet variable, Vpfpurge_idx, and then the problem is gone. As is often the case, the solution to a fairly hard problem turns out to be really simple. As you might expect, finding the problem takes a lot more work that fixing it Thanks to Kristof for writing up this detailed post explaining how the problem was found, and what caused it. *** vBSDcon 2017: BSD at Work (https://www.ixsystems.com/blog/vbsdcon-2017-dexter/) The third biennial vBSDcon hosted by Verisign took place September 7th through 9th with the FreeBSD Developer Summit taking place the first day. vBSDcon and iXsystems' MeetBSD event have been alternating between the East and West coasts of the U.S.A. and these two events play vital roles in reaching Washington, DC-area and Bay Area/Silicon Valley audiences. Where MeetBSD serves many BSD Vendors, vBSDcon attracts a unique government and security industry demographic that isn't found anywhere else. Conference time and travel budgets are always limited and bringing these events to their attendees is a much-appreciated service provided by their hosts. The vBSDcon FreeBSD DevSummit had a strong focus on OpenZFS, the build system and networking with the FreeBSD 12 wish list of features in mind. How to best incorporate the steady flow of new OpenZFS features into FreeBSD such as dataset-level encryption was of particular interest. This feature from a GNU/Linux-based storage vendor is tribute to the growth of the OpenZFS community which is vital in light of the recent “Death of Solaris and ZFS” at Oracle. There has never been more demand for OpenZFS on FreeBSD and the Oracle news further confirms our collective responsibility to meet that demand. The official conference opened with my talk on “Isolated BSD Build Environments” in which I explained how the bhyve hypervisor can be used to effortlessly tour FreeBSD 5.0-onward and build specific source releases on demand to trace regressions to their offending commit. I was followed by a FreeNAS user who made the good point that FreeNAS is an exemplary “entry vector” into Unix and Enterprise Storage fundamentals, given that many of the vectors our generation had are gone. Where many of us discovered Unix and the Internet via console terminals at school or work, smart phones are only delivering the Internet without the Unix. With some irony, both iOS and Android are Unix-based yet offer few opportunities for their users to learn and leverage their Unix environments. The next two talks were The History and Future of Core Dumps in FreeBSD by Sam Gwydir and Using pkgsrc for multi-platform deployments in heterogeneous environments by G. Clifford Williams. I strongly recommend that anyone wanting to speak at AsiaBSDCon read Sam's accompanying paper on core dumps because I consider it the perfect AsiaBSDCon topic and his execution is excellent. Core dumps are one of those things you rarely think about until they are a DROP EVERYTHING! priority. G. Clifford's talk was about what I consider a near-perfect BSD project: pkgsrc, the portable BSD package manager. I put it up there with OpenSSH and mandoc as projects that have provided significant value to other Open Source operating systems. G. Clifford's real-world experiences are perfectly inline with vBSDcon's goal to be more production-oriented than other BSDCons. Of the other talks, any and all Dtrace talks are always appreciated and George Neville-Neil's did not disappoint. He based it on his experiences with the Teach BSD project which is bringing FreeBSD-based computer science education to schools around the world. The security-related talks by John-Mark Gurney, Dean Freeman and Michael Shirk also represented vBSDcon's consideration of the local community and made a convincing point that the BSDs should make concerted efforts to qualify for Common Criteria, FIPS, and other Government security requirements. While some security experts will scoff at these, they are critical to the adoption of BSD-based products by government agencies. BSD Now hosts Allan Jude and Benedict Reuschling hosted an OpenZFS BoF and Ansible talk respectively and I hosted a bhyve hypervisor BoF. The Hallway Track and food at vBSDcon were excellent and both culminated with an after-dinner dramatic reading of Michael W. Lucas' latest book that raised money for the FreeBSD Foundation. A great time was had by all and it was wonderful to see everyone! News Roundup FreeBSD 10.4-RC2 Available (https://lists.freebsd.org/pipermail/freebsd-stable/2017-September/087848.html) FreeBSD 10.4 will be released soon, this is the last chance to find bugs before the official release is cut. Noteworthy Changes Since 10.4-RC1: Given that the amd64 disc1 image was overflowing, more of the base components installed into the disc1 (live) file systems had to be disabled. Most notably, this removed the compiler toolchain from the disc1 images. All disabled tools are still available with the dvd1 images, though. The aesni(4) driver now no longer shares a single FPU context across multiple sessions in multiple threads, addressing problems seen when employing aesni(4) for ipsec(4). Support for netmap(4) by the ixgbe(4) driver has been brought into line with the netmap(4) API present in stable/10. Also, ixgbe(4) now correctly handles VFs in its netmap(4) support again instead of treating these as PFs. During the creation of amd64 and i386 VM images, etcupdate(8) and mergemaster(8) databases now are bootstrapped, akin to what happens along the extraction of base.txz as part of a new installation via bsdinstall(8). This change allows for both of these tools to work out-of-box on the VM images and avoids errors seen when upgrading these images via freebsd-update(8). If you are still on the stable/10 branch, you should test upgrading to 10.4, and make sure there are no problems with your workload Additional testing specifically of the features that have changed since 10.4-BETA1 would also be most helpful This will be the last release from the stable/10 branch *** OpenBSD changes of note 628 (https://www.tedunangst.com/flak/post/openbsd-changes-of-note-628) EuroBSDCon in two weeks. Be sure to attend early and often. Many and various documentation improvements for libcrypto. New man pages, rewrites, expanded bugs sections, and more. Only allow upward migration in vmd. There's a README for the syspatch build system if you want to run your own. Move the kernel relinking code from /etc/rc into a seperate script usable by syspatch. Kernel patches can now be reduced to just the necessary files. Make the callers of sogetopt() responsible for allocating memory. Now allocation and free occur in the same place. Use waitpid() instead of wait() in most programs to avoid accidentally collecting the wrong child. Have cu call isatty() before making assumptions. Switch mandoc rendering of mathematical symbols and greek letters from trying to imitate the characters' graphical shapes, which resulted in unintelligible renderings in many cases, to transliterations conveying the characters' meanings. Update libexpat to 2.2.4. Fix copying partial UTF-8 characters. Sigh, here we go again. Work around bug in F5's handling of the supported elliptic curves extension. RFC 4492 only defines elliptic_curves for ClientHello. However, F5 is sending it in ServerHello. We need to skip over it since our TLS extension parsing code is now more strict. After a first install, run syspatch -c to check for patches. If SMAP is present, clear PSL_AC on kernel entry and interrupt so that only the code in copy{in,out}* that need it run with it set. Panic if it's set on entry to trap() or syscall(). Prompted by Maxime Villard's NetBSD work. Errata. New drivers for arm: rktemp, mvpinctrl, mvmpic, mvneta, mvmdio, mvpxa, rkiic, rkpmic. No need to exec rm from within mandoc. We know there's exactly one file and directory to remove. Similarly with running cmp. Revert to Mesa 13.0.6 to hopefully address rendering issues a handful of people have reported with xpdf/fvwm on ivy bridge with modesetting driver. Rewrite ALPN extension using CBB/CBS and the new extension framework. Rewrite SRTP extension using CBB/CBS and the new extension framework. Revisit 2q queue sizes. Limit the hot queue to 1/20th the cache size up to a max of 4096 pages. Limit the warm and cold queues to half the cache. This allows us to more effectively notice re-interest in buffers instead of losing it in a large hot queue. Add glass console support for arm64. Probably not yet for your machine, though. Replace heaps of hand-written syscall stubs in ld.so with a simpler framework. 65535 is a valid port to listen on. When xinit starts an X server that listens only on UNIX socket, prefer DISPLAY=unix:0 rather than DISPLAY=:0. This will prevent applications from ever falling back to TCP if the UNIX socket connection fails (such as when the X server crashes). Reverted. Add -z and -Z options to apmd to auto suspend or hibernate when low on battery. Remove the original (pre-IETF) chacha20-poly1305 cipher suites. Add urng(4) which supports various USB RNG devices. Instead of adding one driver per device, start bundling them into a single driver. Remove old deactivated pledge path code. A replacement mechanism is being brewed. Fix a bug from the extension parsing rewrite. Always parse ALPN even if no callback has been installed to prevent leaving unprocessed data which leads to a decode error. Clarify what is meant by syslog priorities being ordered, since the numbers and priorities are backwards. Remove a stray setlocale() from ksh, eliminating a lot of extra statically linked code. Unremove some NPN symbols from libssl because ports software thinks they should be there for reasons. Fix saved stack location after resume. Somehow clang changed it. Resume works again on i386. Improve error messages in vmd and vmctl to be more informative. Stop building the miniroot installer for OMAP3 Beagleboards. It hasn't worked in over a year and nobody noticed. Have the callers of sosetopt() free the mbuf for symmetry. On octeon, let the kernel use the hardware FPU even if emulation is compiled in. It's faster. Fix support for 486DX CPUs by not calling cpuid. I used to own a 486. Now I don't. Merge some drm fixes from linux. Defer probing of floppy drives, eliminating delays during boot. Better handling of probes and beacons and timeouts and scans in wifi stack to avoid disconnects. Move mutex, condvar, and thread-specific data routes, pthreadonce, and pthreadexit from libpthread to libc, along with low-level bits to support them. Let's thread aware (but not actually threaded) code work with just libc. New POSIX xlocale implementation. Complete as long as you only use ASCII and UTF-8, as you should. Round and round it goes; when 6.2 stops, nobody knows. A peak at the future? *** Screencasting with OpenBSD (http://eradman.com/posts/screencasting.html) USB Audio Any USB microphone should appear as a new audio device. Here is the dmesg for my mic by ART: uaudio0 at uhub0 port 2 configuration 1 interface 0 "M-One USB" rev 1.10/0.01 addr 2 uaudio0: audio rev 1.00, 8 mixer controls audio1 at uaudio0 audioctl can read off all of the specific characterisitcs of this device $ audioctl -f /dev/audio1 | grep record mode=play,record record.rate=48000 record.channels=1 record.precision=16 record.bps=2 record.msb=1 record.encoding=slinear_le record.pause=0 record.active=0 record.block_size=1960 record.bytes=0 record.errors=0 Now test the recording from the second audio device using aucat(1) aucat -f rsnd/1 -o file.wav If the device also has a headset audio can be played through the same device. aucat -f rsnd/1 -i file.wav Screen Capture using Xvfb The rate at which a framebuffer for your video card is a feature of the hardware and software your using, and it's often very slow. x11vnc will print an estimate of the banwidth for the system your running. x11vnc ... 09/05/2012 22:23:45 fb read rate: 7 MB/sec This is about 4fps. We can do much better by using a virtual framebuffer. Here I'm setting up a new screen, setting the background color, starting cwm and an instance of xterm Xvfb :1 -screen 0 720x540x16 & DISPLAY=:1 xsetroot -solid steelblue & DISPLAY=:1 cwm & DISPLAY=:1 xterm +sb -fa Hermit -fs 14 & Much better! Now we're up around 20fps. x11vnc -display :1 & ... 11/05/2012 18:04:07 fb read rate: 168 MB/sec Make a connection to this virtual screen using raw encoding to eliminate time wasted on compression. vncviewer localhost -encodings raw A test recording with sound then looks like this ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 ~/out.avi Note: always stop the recording and playback using q, not Ctrl-C so that audio inputs are shut down properly. Screen Capture using Xephyr Xephyr is perhaps the easiest way to run X with a shadow framebuffer. This solution also avoids reading from the video card's RAM, so it's reasonably fast. Xephyr -ac -br -noreset -screen 800x600 :1 & DISPLAY=:1 xsetroot -solid steelblue & DISPLAY=:1 cwm & DISPLAY=:1 xrdb -load ~/.Xdefaults & DISPLAY=:1 xterm +sb -fa "Hermit" -fs 14 & Capture works in exactally the same way. This command tries to maintain 12fps. ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi To capture keyboard and mouse input press Ctrl then Shift. This is very handy for using navigating a window manager in the nested X session. Arranging Windows I have sometimes found it helpful to launch applications and arrange them in a specific way. This will open up a web browser listing the current directory and position windows using xdotool DISPLAY=:1 midori "file:///pwd" & sleep 2 DISPLAY=:1 xdotool search --name "xterm" windowmove 0 0 DISPLAY=:1 xdotool search --class "midori" windowmove 400 0 DISPLAY=:1 xdotool search --class "midori" windowsize 400 576 This will position the window precisely so that it appears to be in a tmux window on the right. Audio/Video Sync If you find that the audio is way out of sync with the video, you can ajust the start using the -ss before the audio input to specify the number of seconds to delay. My final recording command line, that delays the audio by 0.5 seconds, writing 12fps ffmpeg -ss 0.5 -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi Sharing a Terminal with tmux If you're trying to record a terminal session, tmux is able to share a session. In this way a recording of an X framebuffer can be taken without even using the screen. Start by creating the session. tmux -2 -S /tmp/tmux0 Then on the remote side connect on the same socket tmux -2 -S /tmp/tmux0 attach Taking Screenshots Grabbing a screenshots on Xvfb server is easily accomplished with ImageMagick's import command DISPLAY=:1 import -window root screenshot.png Audio Processing and Video Transcoding The first step is to ensure that the clip begins and ends where you'd like it to. The following will make a copy of the recording starting at time 00:00 and ending at 09:45 ffmpeg -i interactive-sql.avi -vcodec copy -acodec copy -ss 00:00:00 -t 00:09:45 interactive-sql-trimmed.avi mv interactive-sql-trimmed.avi interactive-sql.avi Setting the gain correctly is very important with an analog mixer, but if you're using a USB mic there may not be a gain option; simply record using it's built-in settings and then adjust the levels afterwards using a utility such as normalize. First extact the audio as a raw PCM file and then run normalize ffmpeg -i interactive-sql.avi -c:a copy -vn audio.wav normalize audio.wav Next merge the audio back in again ffmpeg -i interactive-sql.avi -i audio.wav -map 0:0 -map 1:0 -c copy interactive-sql-normalized.avi The final step is to compress the screencast for distribution. Encoding to VP8/Vorbis is easy: ffmpeg -i interactive-sql-normalized.avi -c:v libvpx -b:v 1M -c:a libvorbis -q:a 6 interactive-sql.webm H.264/AAC is tricky. For most video players the color space needs to be set to yuv420p. The -movflags puts the index data at the beginning of the file to enable streaming/partial content requests over HTTP: ffmpeg -y -i interactive-sql-normalized.avi -c:v libx264 -preset slow -crf 14 -pix_fmt yuv420p -movflags +faststart -c:a aac -q:a 6 interactive-sql.mp4 TrueOS @ Ohio Linuxfest '17! (https://www.trueos.org/blog/trueos-ohio-linuxfest-17/) Dru Lavigne and Ken Moore are both giving presentations on Saturday the 30th. Sit in and hear about new developments for the Lumina and FreeNAS projects. Ken is offering Lumina Rising: Challenging Desktop Orthodoxy at 10:15 am in Franklin A. Hear his thoughts about the ideas propelling desktop environment development and how Lumina, especially Lumina 2, is seeking to offer a new model of desktop architecture. Elements discussed include session security, application dependencies, message handling, and operating system integration. Dru is talking about What's New in FreeNAS 11 at 2:00 pm in Franklin D. She'll be providing an overview of some of the new features added in FreeNAS 11.0, including: Alert Services Starting specific services at boot time AD Monitoring to ensure the AD service restarts if disconnected A preview of the new user interface support for S3-compatible storage and the bhyve hypervisor She's also giving a sneak peek of FreeNAS 11.1, which has some neat features: A complete rewrite of the Jails/Plugins system as FreeNAS moves from warden to iocage Writing new plugins with just a few lines of code A brand new asynchronous middleware API Who's going? Attending this year are: Dru Lavigne (dlavigne): Dru leads the technical documentation team at iX, and contributes heavily to open source documentation projects like FreeBSD, FreeNAS, and TrueOS. Ken Moore (beanpole134): Ken is the lead developer of Lumina and a core contributor to TrueOS. He also works on a number of other Qt5 projects for iXsystems. J.T. Pennington (q5sys): Some of you may be familiar with his work on BSDNow, but J.T. also contributes to the TrueOS, Lumina, and SysAdm projects, helping out with development and general bug squashing. *** Beastie Bits Lumina Development Preview: Theme Engine (https://www.trueos.org/blog/lumina-development-preview-theme-engine/) It's happening! Official retro Thinkpad lappy spotted in the wild (https://www.theregister.co.uk/2017/09/04/retro_thinkpad_spotted_in_the_wild/) LLVM libFuzzer and SafeStack ported to NetBSD (https://blog.netbsd.org/tnf/entry/llvm_libfuzzer_and_safestack_ported) Remaining 2017 FreeBSD Events (https://www.freebsdfoundation.org/news-and-events/event-calendar/2017-openzfs-developer-summit/) *** Feedback/Questions Andrew - BSD Teaching Material (http://dpaste.com/0YTT0VP) Seth - Switching to Tarsnap after Crashplan becomes no more (http://dpaste.com/1SK92ZX#wrap) Thomas - Native encryption in ZFS (http://dpaste.com/02KD5FX#wrap) Coding Cowboy - Coding Cowboy - Passwords and clipboards (http://dpaste.com/31K0E40#wrap) ***
In this episode, we take a look at the reimplementation of NetBSD using a Microkernel, check out what makes DHCP faster, and see what high-process count support for DragonflyBSD has to offer, and we answer the questions you've always wanted to ask us. This episode was brought to you by Headlines A Reimplementation Of Netbsd Using a Microkernel (http://theembeddedboard.review/a-reimplementation-of-netbsd-using-a-microkernel-part-1-of-2/) Minix author Andy Tanenbaum writes in Part 1 of a-reimplementation-of-netbsd-using-a-microkernel (http://theembeddedboard.review/a-reimplementation-of-netbsd-using-a-microkernel-part-1-of-2/) Based on the MINIX 3 microkernel, we have constructed a system that to the user looks a great deal like NetBSD. It uses pkgsrc, NetBSD headers and libraries, and passes over 80% of the KYUA tests). However, inside, the system is completely different. At the bottom is a small (about 13,000 lines of code) microkernel that handles interrupts, message passing, low-level scheduling, and hardware related details. Nearly all of the actual operating system, including memory management, the file system(s), paging, and all the device drivers run as user-mode processes protected by the MMU. As a consequence, failures or security issues in one component cannot spread to other ones. In some cases a failed component can be replaced automatically and on the fly, while the system is running, and without user processes noticing it. The talk will discuss the history, goals, technology, and status of the project. Research at the Vrije Universiteit has resulted in a reimplementation of NetBSD using a microkernel instead of the traditional monolithic kernel. To the user, the system looks a great deal like NetBSD (it passes over 80% of the KYUA tests). However, inside, the system is completely different. At the bottom is a small (about 13,000 lines of code) microkernel that handles interrupts, message passing, low-level scheduling, and hardware related details. Nearly all of the actual operating system, including memory management, the file system(s), paging, and all the device drivers run as user-mode processes protected by the MMU. As a consequence, failures or security issues in one component cannot spread to other ones. In some cases a failed component can be replaced automatically and on the fly, while the system is running. The latest work has been adding live update, making it possible to upgrade to a new version of the operating system WITHOUT a reboot and without running processes even noticing. No other operating system can do this. The system is built on MINIX 3, a derivative of the original MINIX system, which was intended for education. However, after the original author, Andrew Tanenbaum, received a 2 million euro grant from the Royal Netherlands Academy of Arts and Sciences and a 2.5 million euro grant from the European Research Council, the focus changed to building a highly reliable, secure, fault tolerant operating system, with an emphasis on embedded systems. The code is open source and can be downloaded from www.minix3.org. It runs on the x86 and ARM Cortex V8 (e.g., BeagleBones). Since 2007, the Website has been visited over 3 million times and the bootable image file has been downloaded over 600,000 times. The talk will discuss the history, goals, technology, and status of the project. Part 2 (http://theembeddedboard.review/a-reimplementation-of-netbsd-using-a-microkernel-part-2-of-2/) is also available. *** Rapid DHCP: Or, how do Macs get on the network so fast? (https://cafbit.com/post/rapid_dhcp_or_how_do/) One of life's minor annoyances is having to wait on my devices to connect to the network after I wake them from sleep. All too often, I'll open the lid on my EeePC netbook, enter a web address, and get the dreaded "This webpage is not available" message because the machine is still working on connecting to my Wi-Fi network. On some occasions, I have to twiddle my thumbs for as long as 10-15 seconds before the network is ready to be used. The frustrating thing is that I know it doesn't have to be this way. I know this because I have a Mac. When I open the lid of my MacBook Pro, it connects to the network nearly instantaneously. In fact, no matter how fast I am, the network comes up before I can even try to load a web page. My curiosity got the better of me, and I set out to investigate how Macs are able to connect to the network so quickly, and how the network connect time in other operating systems could be improved. I figure there are three main categories of time-consuming activities that occur during network initialization: Link establishment. This is the activity of establishing communication with the network's link layer. In the case of Wi-Fi, the radio must be powered on, the access point detected, and the optional encryption layer (e.g. WPA) established. After link establishment, the device is able to send and receive Ethernet frames on the network. Dynamic Host Configuration Protocol (DHCP). Through DHCP handshaking, the device negotiates an IP address for its use on the local IP network. A DHCP server is responsible for managing the IP addresses available for use on the network. Miscellaneous overhead. The operating system may perform any number of mundane tasks during the process of network initialization, including running scripts, looking up preconfigured network settings in a local database, launching programs, etc. My investigation thus far is primarily concerned with the DHCP phase, although the other two categories would be interesting to study in the future. I set up a packet capture environment with a spare wireless access point, and observed the network activity of a number of devices as they initialized their network connection. For a worst-case scenario, let's look at the network activity captured while an Android tablet is connecting: This tablet, presumably in the interest of "optimization", is initially skipping the DHCP discovery phase and immediately requesting its previous IP address. The only problem is this is a different network, so the DHCP server ignores these requests. After about 4.5 seconds, the tablet stubbornly tries again to request its old IP address. After another 4.5 seconds, it resigns itself to starting from scratch, and performs the DHCP discovery needed to obtain an IP address on the new network. In all fairness, this delay wouldn't be so bad if the device was connecting to the same network as it was previously using. However, notice that the tablet waits a full 1.13 seconds after link establishment to even think about starting the DHCP process. Engineering snappiness usually means finding lots of small opportunities to save a few milliseconds here and there, and someone definitely dropped the ball here. In contrast, let's look at the packet dump from the machine with the lightning-fast network initialization, and see if we can uncover the magic that is happening under the hood: The key to understanding the magic is the first three unicast ARP requests. It looks like Mac OS remembers certain information about not only the last connected network, but the last several networks. In particular, it must at least persist the following tuple for each of these networks: > 1. The Ethernet address of the DHCP server > 2. The IP address of the DHCP server > 3. Its own IP address, as assigned by the DHCP server During network initialization, the Mac transmits carefully crafted unicast ARP requests with this stored information. For each network in its memory, it attempts to send a request to the specific Ethernet address of the DHCP server for that network, in which it asks about the server's IP address, and requests that the server reply to the IP address which the Mac was formerly using on that network. Unless network hosts have been radically shuffled around, at most only one of these ARP requests will result in a response—the request corresponding to the current network, if the current network happens to be one of the remembered networks. This network recognition technique allows the Mac to very rapidly discover if it is connected to a known network. If the network is recognized (and presumably if the Mac knows that the DHCP lease is still active), it immediately and presumptuously configures its IP interface with the address it knows is good for this network. (Well, it does perform a self-ARP for good measure, but doesn't seem to wait more than 13ms for a response.) The DHCP handshaking process begins in the background by sending a DHCP request for its assumed IP address, but the network interface is available for use during the handshaking process. If the network was not recognized, I assume the Mac would know to begin the DHCP discovery phase, instead of sending blind requests for a former IP address as the Galaxy Tab does. The Mac's rapid network initialization can be credited to more than just the network recognition scheme. Judging by the use of ARP (which can be problematic to deal with in user-space) and the unusually regular transmission intervals (a reliable 1.0ms delay between each packet sent), I'm guessing that the Mac's DHCP client system is entirely implemented as tight kernel-mode code. The Mac began the IP interface initialization process a mere 10ms after link establishment, which is far faster than any other device I tested. Android devices such as the Galaxy Tab rely on the user-mode dhclient system (part of the dhcpcd package) dhcpcd program, which no doubt brings a lot of additional overhead such as loading the program, context switching, and perhaps even running scripts. The next step for some daring kernel hacker is to implement a similarly aggressive DHCP client system in the Linux kernel, so that I can enjoy fast sign-on speeds on my Android tablet, Android phone, and Ubuntu netbook. There already exists a minimal DHCP client implementation in the Linux kernel, but it lacks certain features such as configuring the DNS nameservers. Perhaps it wouldn't be too much work to extend this code to support network recognition and interface with a user-mode daemon to handle such auxillary configuration information received via DHCP. If I ever get a few spare cycles, maybe I'll even take a stab at it. You can also find other ways of optimizing the dhclient program and how it works in the dhclient tutorial on Calomel.org (https://calomel.org/dhclient.html). *** BSDCam Trip Report (https://www.freebsdfoundation.org/blog/bsdcam-2017-trip-report-michael-lucas/) Over the decades, FreeBSD development and coordination has shifted from being purely on-line to involving more and more in-person coordination and cooperation. The FreeBSD Foundation sponsors a devsummit right before BSDCan, EuroBSDCon, and AsiaBSDCon, so that developers traveling to the con can leverage their airfare and hammer out some problems. Yes, the Internet is great for coordination, but nothing beats a group of developers spending ten minutes together to sketch on a whiteboard and figuring out exactly how to make something bulletproof. In addition to the coordination efforts, though, conference devsummits are hierarchical. There's a rigid schedule, with topics decided in advance. Someone leads the session. Sessions can be highly informative, passionate arguments, or anything in between. BSDCam is… a little different. It's an invaluable part of the FreeBSD ecosystem. However, it's something that I wouldn't normally attend. But right now, is not normal. I'm writing a new edition of Absolute FreeBSD. To my astonishment, people have come to rely on this book when planning their deployments and operations. While I find this satisfying, it also increases the pressure on me to get things correct. When I wrote my first FreeBSD book back in 2000, a dozen mailing lists provided authoritative information on FreeBSD development. One person could read every one of those lists. Today, that's not possible—and the mailing lists are only one narrow aspect of the FreeBSD social system. Don't get me wrong—it's pretty easy to find out what people are doing and how the system works. But it's not that easy to find out what people will be doing and how the system will work. If this book is going to be future-proof, I needed to leave my cozy nest and venture into the wilds of Cambridge, England. Sadly, the BSDCam chair agreed with my logic, so I boarded an aluminum deathtrap—sorry, a “commercial airliner”—and found myself hurtled from Detroit to Heathrow. And one Wednesday morning, I made it to the William Gates building of Cambridge University, consciousness nailed to my body by a thankfully infinite stream of proper British tea. BSDCam attendance is invitation only, and the facilities can only handle fifty folks or so. You need to be actively working on FreeBSD to wrangle an invite. Developers attend from all over the world. Yet, there's no agenda. Robert Watson is the chair, but he doesn't decide on the conference topics. He goes around the room and asks everyone to introduce themselves, say what they're working on, and declare what they want to discuss during the conference. The topics of interest are tallied. The most popular topics get assigned time slots and one of the two big rooms. Folks interested in less popular topics are invited to claim one of the small breakout rooms. Then the real fun begins. I started by eavesdropping in the virtualization workshop. For two hours, people discussed FreeBSD's virtualization needs, strengths, and weaknesses. What needs help? What should this interface look like? What compatibility is important, and what isn't? By the end of the session, the couple dozen people had developed a reasonable consensus and, most importantly, some folks had added items to their to-do lists. Repeat for a dozen more topics. I got a good grip on what's really happening with security mitigation techniques, FreeBSD's cloud support, TCP/IP improvements, advances in teaching FreeBSD, and more. A BSDCan devsummit presentation on packaging the base system is informative, but eavesdropping on two dozen highly educated engineers arguing about how to nail down the final tidbits needed to make that a real thing is far more educational. To my surprise, I was able to provide useful feedback for some sessions. I speak at a lot of events outside of the FreeBSD world, and was able to share much of what I hear at Linux conferences. A tool that works well for an experienced developer doesn't necessarily work well for everyone. Every year, I leave BSDCan tired. I left BSDCam entirely exhausted. These intense, focused discussions stretched my brain. But, I have a really good idea where key parts of FreeBSD development are actually headed. This should help future-proof the new Absolute FreeBSD, as much as any computer book can be future-proof. Plus, BSDCam throws the most glorious conference dinner I've ever seen. I want to thank Robert Watson for his kind invitation, and the FreeBSD Foundation for helping defray the cost of this trip Interview - The BSDNow Crew As a kid, what did you dream of to become as an adult? JT: An Astronaut BR: I wanted to be a private detective, because of all the crime novels that I read back then. I didn't get far with it. However, I think the structured analysis skills (who did what, when, and such) help me in debugging and sysadmin work. AJ: Didn't think about it much How do you manage to stay organized day to day with so much things you're actively doing each day? (Day job, wife/girlfriend, conferences, hobbies, friends, etc.) JT: Who said I was organized? BR: A lot of stuff in my calendar as reminders, open browser tabs as “to read later” list. A few things like task switching when getting stuck helps. Also, focus on a single goal for the day, even though there will be distractions. Slowly, but steadily chip away at the things you're working on. Rather than to procrastinate and put things back to review later, get started early with easy things for a big task and then tackle the hard part. Often, things look totally chaotic and unmanageable, until you start working on them. AJ: I barely manage. Lots of Google Calendar reminders, and the entire wall of my office is covered in whiteboard sheet todo lists. I use pinboard.in to deal with finding and organizing bookmarks. Write things down, don't trust your memory. What hobbies outside of IT do you have? JT: I love photography, but I do that Professional part time, so I'm not sure if that counts as a hobby anymore. I guess it'd have to be working in the garage on my cars. BR: I do Tai Chi to relax once a week in a group, but can also do it alone, pretty much everywhere. Way too much Youtube watching and browsing the web. I did play some games before studying at the university and I'm still proud that I could control it to the bare minimum not to impact my studies. A few “lapses” from time to time, revisiting the old classics since the newer stuff won't run on my machines anyway. Holiday time is pretty much spent for BSD conferences and events, this is where I can relax and talk with like-minded people from around the world, which is fascinating. Plus, it gets me to various places and countries I never would have dared to visit on my own. AJ: I play a few video games, and I like to ski, although I don't go very often as most of my vacation time is spent hanging out with my BSD friends at various conferences How do you relax? JT: What is this word ‘relax' and what does it mean? BR: My Tai Chi plays a big part in it I guess. I really calms you and the constant stream of thoughts for a while. It also gives you better clarity of what's important in life. Watching movies, sleeping long. AJ: Usually watching TV or Movies. Although I have taken to doing most of my TV watching on my exercise bike now, but it is still mentally relaxing If FreeBSD didn't exist, which BSD flavour would you use? Why? JT: I use TrueOS, but if FreeBSD didn't exist, that project might not either… so… My other choice would be HardenedBSD, but since it's also based on FreeBSD I'm in the same dillema. BR: I once installed NetBSD to see what It can do. If FreeBSD wouldn't exist, I would probably try my luck with it. OpenBSD is also appealing, but I've never installed it. AJ: When I started using FreeBSD in 2000, the only other BSD I had heard of at the time was OpenBSD. If FreeBSD wasn't around, I don't think the world would look like it does, so it is hard to speculate. If any of the BSD's weren't around and you had to use Linux, which camp would belong to? (Redhat, SUSE, Debian, Ubuntu, Gentoo?) JT: I learned Linux in the mid 90s using Slackware, which I used consistently up until the mid 2000s, when I joined the PuppyLinux community and eventually became a developer (FYI, Puppy was/is/can be based on Slackware -- its complicated). So I'd go back to using either Slackware or PuppyLinux. BR: I tried various Linux distributions until I landed at Debian. I used is pretty extensively as my desktop OS at home, building custom kernels and packages to install them until I discovered FreeBSD. I ran both side by side for a few months for learning until one day I figured out that I had not booted Debian in a while, so I switched completely. AJ: The first Linux I played with was Slackware, and it is the most BSD like, but the bits of Linux I learned in school were Redhat and so I can somewhat wrap my head around it, although now that they are changing everything to systemd, all of that old knowledge is more harmful than useful. Are you still finding yourself in need to use Windows/Mac OS? Why? JT: I work part time as a professional Photographer, so I do use Windows for my photography work. While I can do everything I need to do in Linux, it comes down to being pragmatic about my time. What takes me several hours to accomplish in Linux I can accomplish in 20 minutes on Windows. BR: I was a long time Windows-only user before my Unix days. But back when Vista was about to come out and I needed a new laptop, my choice was basically learning to cope with Vistas awful features or learn MacOS X. I did the latter, it increased my productivity since it's really a good Unix desktop experience (at least, back then). I only have to use Windows at work from time to time as I manage our Windows Terminal server, which keeps the exposure low enough and I only connect to it to use a certain app not available for the Mac or the BSDs. AJ: I still use Windows to play games, for a lot of video conferencing, and to produce BSD Now. Some of it could be done on BSD but not as easily. I have promised myself that I will switch to 100% BSD rather than upgrade to Windows 10, so we'll see how that goes. Please describe your home networking setup. Router type, router OS, router hardware, network segmentation, wifi apparatus(es), other devices connected, and anything else that might be interesting about your home network. BR: Very simple and boring: Apple Airport Express base station and an AVM FritzBox for DNS, DHCP, and the link to my provider. A long network cable to my desktop machine. That I use less and less often. I just bought an RPI 3 for some home use in the future to replace it. Mostly my brother's and my Macbook Pro's are connected, our phones and the iPad of my mother. AJ: I have a E3-1220 v3 (dual 3.1ghz + HT) with 8 GB of ram, and 4x Intel gigabit server NICs as my router, and it runs vanilla FreeBSD (usually some snapshot of -current). I have 4 different VLANs, Home, Office, DMZ, and Guest WiFi. WiFi is served via a tiny USB powered device I bought in Tokyo years ago, it serves 3 different SSIDs, one for each VLAN except the DMZ. There are ethernet jacks in every room wired for 10 gigabit, although the only machines with 10 gigabit are my main workstation, file server, and some machines in the server rack. There are 3 switches, one for the house (in the laundry room), one for the rack, and one for 10gig stuff. There is a rack in the basement spare bedroom, it has 7 servers in it, mostly storage for live replicas of customer data for my company. How do guys manage to get your work done on FreeBSD desktops? What do you do when you need to a Linux or Windows app that isn't ported, or working? I've made several attempts to switch to FreeBSD, but each attempt failed because of tools not being available (e.g. Zoom, Dropbox, TeamViewer, Crashplan) or broken (e.g. VirtualBox). BR: I use VIrtualBox for everything that is not natively available or Windows-only. Unfortunately, that means no modern games. I mostly do work in the shell when I'm on FreeBSD and when it has to be a graphical application, then I use Fluxbox as the DE. I want to get work done, not look at fancy eye-candy that get's boring after a while. Deactivated the same stuff on my mac due to the same reason. I look for alternative software online, but my needs are relatively easy to satisfy as I'm not doing video editing/rendering and such. AJ: I generally find that I don't need these apps. I use Firefox, Thunderbird, OpenSSH, Quassel, KomodoEdit, and a few other apps, so my needs are not very demanding. It is annoying when packages are broken, but I usually work around this with boot environments, and being able to just roll back to a version that worked for a few days until the problem is solved. I do still have access to a windows machine for the odd time I need specific VPN software or access to Dell/HP etc out-of-band management tools. Which desktop environments are your favorite, and why? For example, I like i3, Xfce, and I'm drawn to Lumina's ethos, but so far always seem to end up back on Xfc because of its ease of use, flexibility, and dashing good looks. JT: As a Lumina Desktop developer, I think my preference is obvious. ;) I am also a long timeOpenBox user, so I have a soft place in my heart for that as well. BR: I use Fluxbox when I need to work with a lot of windows or an application demands X11. KDE and others are too memory heavy for me and I rarely use even 20% of the features they provide. AJ: I was a long time KDE user, but I have adopted Lumina. I find it fast, and that it gets out of my way and lets me do what I want. It had some annoyances early on, but I've nagged the developers into making it work for me. Which command-line shells do you prefer, why, and how (if at all) have you customised the environment or prompt? BR: I use zsh, but without all the fancy stuff you can find online. It might make you more productive, yes. But again, I try to keep things simple. I'm slowly learning tmux and want to work more in it in the future. I sometimes look at other BSD people's laptops and am amazed at what they do with window-management in tmux. My prompt looks like this: bcr@Voyager:~> 20:20 17-08-17 Put this in your .zshrc to get the same result: PROMPT='%n@%m:%~>' RPROMPT='%T %D' AJ: I started using tcsh early on, because it was the shell on the first box I had access to, and because one of the first things I read in “BSD Hacks” was how to enable ‘typo correction”, which made my life a lot better especially on dial up in the early days. My shell prompt looks like this: allan@CA-TOR1-02:/usr/home/allan% What is one thing (or more) missing in FreeBSD you would import from another project or community? Could be tech, process, etc. JT: AUFS from Linux BR: Nohup from Illumos where you can detach an already running process and put it in the background. I often forget that and I'm not in tmux when that happens, so I can see myself use that feature a lot. AJ: Zones (more complete Jails) from IllumOS how do you manage your time to learn about and work on FreeBSD? Does your work/employment enable what you do, or are your contributions mainly done in private time? JT: These days I'm mostly learning things I need for work, so it just falls into something I'm doing while working on work projects. BR: We have a lot of time during the semester holidays to learn on our own, it's part of the idea of being in a university to keep yourself updated, at least for me. Especially in the fast moving world of IT. I also read a lot in my free time. My interests can shift sometimes, but then I devour everything I can find on the topic. Can be a bit excessive, but has gotten me where I am now and I still need a lot to learn (and want to). Since I work with FreeBSD at work (my owndoing), I can try out many things there. AJ: My work means a spend a lot of time working with FreeBSD, but not that much time working ON it. My contributions are mostly done outside of work, but as I own the company I do get more flexibility to take time off for conferences and other FreeBSD related stuff. we know we can bribe Michael W Lucas with gelato (good gelato that is), but what can we use to bribe you guys? Like when I want to have Allan to work on fixing a bug which prevents me from running ZFS on this fancy rock64 board? BR: Desserts of various kinds. AJ: I am probably not the right person to look at your rock64 board. Most people in the project have taken to bribing me with chocolate. In general, my todo list is so long, the best way is a trade, you take this task and I'll take that task. Is your daily mobile device iOS, Android, Windows Mobile, or other? Why? JT: These days I'm using Android on my Blackberry Priv, but until recently I was still a heavy user of Sailfish OS. I would use SailfishOS everyday, if I could find a phone with a keyboard that I could run it on. BR: iOS on the iPhone 7 currently. Never used an Android phone, saw it on other people's devices and what they can do with it (much more). But the infrequent security updates (if any at all) keep me away from it. AJ: I have a Google Nexus 6 (Android 7.1). I wanted the ‘pure' Android experience, and I had been happy with my previous Nexus S. I don't run a custom OS/ROM or anything because I use the phone to verify that video streams work on an ‘average users device'. I am displeased that support for my device will end soon. I am not sure what device I will get next, but it definitely won't be an iPhone. News Roundup Beta Update - Request for (more) Testing (http://undeadly.org/cgi?action=article&sid=20170808065718&mode=flat&count=30) https://beta.undeadly.org/ has received an update. The most significant changes include: The site has been given a less antiquated "look". (As the topic icons have been eliminated, we are no longer seeking help with those graphics.) The site now uses a moderate amount of semantic HTML5. Several bugs in the HTML fragment validator (used for submissions and comments) have been fixed. To avoid generating invalid HTML, submission content which fails validation is no longer displayed in submission/comment previews. Plain text submissions are converted to HTML in a more useful fashion. (Instead of just converting each EOL to , the converter now generates proper paragraphs and interprets two or more consecutive EOLs as indicating a paragraph break.) The redevelopment remains a work-in-progress. Many thanks to those who have contributed! As before, constructive feedback would be appreciated. Of particular interest are reports of bugs in behaviour (for example, in the HTML validator or in authentication) that would preclude the adoption of the current code for the main site. High-process-count support added to master (http://lists.dragonflybsd.org/pipermail/users/2017-August/313552.html) We've fixed a number of bottlenecks that can develop when the number of user processes runs into the tens of thousands or higher. One thing led to another and I said to myself, "gee, we have a 6-digit PID, might as well make it work to a million!". With the commits made today, master can support at least 900,000 processes with just a kern.maxproc setting in /boot/loader.conf, assuming the machine has the memory to handle it. And, in fact, as today's machines start to ratchet up there in both memory capacity and core count, with fast storage (NVMe) and fast networking (10GigE and higher), even in consumer boxes, this is actually something that one might want to do. With AMD's threadripper and EPYC chips now out, the IntelAMD cpu wars are back on! Boasting up to 32 cores (64 threads) per socket and two sockets on EPYC, terabytes of ram, and motherboards with dual 10GigE built-in, the reality is that these numbers are already achievable in a useful manner. In anycase, I've tested these changes on a dual-socket xeon. I can in-fact start 900,000 processes. They don't get a whole lot of cpu and running 'ps' would be painful, but it works and the system is still responsive from the shell with all of that going on. xeon126# uptime 1:42PM up 9 mins, 3 users, load averages: 890407.00, 549381.40, 254199.55 In fact, judging from the memory use, these minimal test processes only eat around 60KB each. 900,000 of them ate only 55GB on a 128GB machine. So even a million processes is not out of the question, depending on the cpu requirements for those processes. Today's modern machines can be stuffed with enormous amounts of memory. Of course, our PIDs are currently limited to 6 digits, so a million is kinda the upper limit in terms of discrete user processes (verses pthreads which are less restricted). I'd rather not go to 7 digits (yet). CFT: Driver for generic MS Windows 7/8/10 - compatible USB HID multi-touch touchscreens (https://lists.freebsd.org/pipermail/freebsd-current/2017-August/066783.html) Following patch [1] adds support for generic MS Windows 7/8/10 - compatible USB HID multi-touch touchscreens via evdev protocol. It is intended to be a native replacement of hid-multitouch.c driver found in Linux distributions and multimedia/webcamd port. Patch is made for 12-CURRENT and most probably can be applied to recent 11-STABLE and 11.1-RELEASE (not tested) How to test" 1. Apply patch [1] 2. To compile this driver into the kernel, place the following lines into your kernel configuration file: device wmt device usb device evdev Alternatively, to load the driver as a module at boot time, place the following line in loader.conf(5): wmt_load="YES" 3. Install x11-drivers/xf86-input-evdev or x11-drivers/xf86-input-libinput port 4. Tell XOrg to use evdev or libinput driver for the device: ``` Section "ServerLayout" InputDevice "TouchScreen0" "SendCoreEvents" EndSection Section "InputDevice" Identifier "TouchScreen0" Driver "evdev" # Driver "libinput" Option "Device" "/dev/input/eventXXX" EndSection ``` Exact value of "/dev/input/eventXXX" can be obtained with evemu-record utility from devel/evemu. Note1: Currently, driver does not support pens or touchpads. Note2: wmt.ko should be kld-loaded before uhid driver to take precedence over it! Otherwise uhid can be kld-unloaded after loading of wmt. wmt review: https://reviews.freebsd.org/D12017 Raw diff: https://reviews.freebsd.org/D12017.diff *** Beastie Bits BSDMag Programing Languages Infographic (https://bsdmag.org/programm_history/) t2k17 Hackathon Report: Bob Beck on buffer cache tweaks, libressl and pledge progress (http://undeadly.org/cgi?action=article&sid=20170815171854) New FreeBSD Journal (https://www.freebsdfoundation.org/past-issues/resource-control/) NetBSD machines at Open Source Conference 2017 Kyoto (http://mail-index.netbsd.org/netbsd-advocacy/2017/08/10/msg000744.html) *** Feedback/Questions Dan - HDD question (http://dpaste.com/3H6TDJV) Benjamin - scrub of death (http://dpaste.com/10F086V) Jason - Router Opinion (http://dpaste.com/2D9102K) Sohrab - Thanks (http://dpaste.com/1XYYTWF) ***
In this episode, we clear up the myth about scrub of death, look at Wayland and Weston on FreeBSD, Intel QuickAssist is here, and we check out OpenSMTP on OpenBSD. This episode was brought to you by Headlines Matt Ahrens answers questions about the “Scrub of Death” In working on the breakdown of that ZFS article last week, Matt Ahrens contacted me and provided some answers he has given to questions in the past, allowing me to answer them using HIS exact words. “ZFS has an operation, called SCRUB, that is used to check all data in the pool and recover any data that is incorrect. However, if a bug which make errors on the pool persist (for example, a system with bad non-ecc RAM) then SCRUB can cause damage to a pool instead of recover it. I heard it called the “SCRUB of death” somewhere. Therefore, as far as I understand, using SCRUB without ECC memory is dangerous.” > I don't believe that is accurate. What is the proposed mechanism by which scrub can corrupt a lot of data, with non-ECC memory? > ZFS repairs bad data by writing known good data to the bad location on disk. The checksum of the data has to verify correctly for it to be considered "good". An undetected memory error could change the in-memory checksum or data, causing ZFS to incorrectly think that the data on disk doesn't match the checksum. In that case, ZFS would attempt to repair the data by first re-reading the same offset on disk, and then reading from any other available copies of the data (e.g. mirrors, ditto blocks, or RAIDZ reconstruction). If any of these attempts results in data that matches the checksum, then the data will be written on top of the (supposed) bad data. If the data was actually good, then overwriting it with the same good data doesn't hurt anything. > Let's look at what will happen with 3 types of errors with non-ECC memory: > 1. Rare, random errors (e.g. particle strikes - say, less than one error per GB per second). If ZFS finds data that matches the checksum, then we know that we have the correct data (at least at that point in time, with probability 1-1/2^256). If there are a lot of memory errors happening at a high rate, or if the in-memory checksum was corrupt, then ZFS won't be able to find a good copy of the data , so it won't do a repair write. It's possible that the correctly-checksummed data is later corrupted in memory, before the repair write. However, the window of vulnerability is very very small - on the order of milliseconds between when the checksum is verified, and when the write to disk completes. It is implausible that this tiny window of memory vulnerability would be hit repeatedly. > 2. Memory that pretty much never does the right thing. (e.g. huge rate of particle strikes, all memory always reads 0, etc). In this case, critical parts of kernel memory (e.g. instructions) will be immediately corrupted, causing the system to panic and not be able to boot again. > 3. One or a few memory locations have "stuck bits", which always read 0 (or always read 1). This is the scenario discussed in the message which (I believe) originally started the "Scrub of Death" myth: https://forums.freenas.org/index.php?threads/ecc-vs-non-ecc-ram-and-zfs.15449/ This assumes that we read in some data from disk to a memory location with a stuck bit, "correct" that same bad memory location by overwriting the memory with the correct data, and then we write the bad memory location to disk. However, ZFS doesn't do that. (It seems the author thinks that ZFS uses parity, which it only does when using RAID-Z. Even with RAID-Z, we also verify the checksum, and we don't overwrite the bad memory location.) > Here's what ZFS will actually do in this scenario: If ZFS reads data from disk into a memory location with a stuck bit, it will detect a checksum mismatch and try to find a good copy of the data to repair the "bad" disk. ZFS will allocate a new, different memory location to read a 2nd copy of the data, e.g. from the other side of a mirror (this happens near the end of dslscanscrub_cb()). If the new memory location also has a stuck bit, then its checksum will also fail, so we won't use it to repair the "bad" disk. If the checksum of the 2nd copy of the data is correct, then we will write it to the "bad" disk. This write is unnecessary, because the "bad" disk is not really bad, but it is overwriting the good data with the same good data. > I believe that this misunderstanding stems from the idea that ZFS fixes bad data by overwriting it in place with good data. In reality, ZFS overwrites the location on disk, using a different memory location for each read from disk. The "Scrub of Death" myth assumes that ZFS overwrites the location in memory, which it doesn't do. > In summary, there's no plausible scenario where ZFS would amplify a small number of memory errors, causing a "scrub of death". Additionally, compared to other filesystems, ZFS checksums provide some additional protection against bad memory. “Is it true that ZFS verifies the checksum of every block on every read from disk?” > Yes “And if that block is incorrect, that ZFS will repair it?” > Yes “If yes, is it possible set options or flag for change that behavior? For example, I would like for ZFS to verify checksums during any read, but not change anything and only report about issues if it appears. Is it possible?” > There isn't any built-in flag for doing that. It wouldn't be hard to add one though. If you just wanted to verify data, without attempting to correct it, you could read or scan the data with the pool was imported read-only “If using a mirror, when a file is read, is it fully read and verified from both sides of the mirror?” > No, for performance purposes, each block is read from only one side of the mirror (assuming there is no checksum error). “What is the difference between a scrub and copying every file to /dev/null?” > That won't check all copies of the file (e.g. it won't check both sides of the mirror). *** Wayland, and Weston, and FreeBSD - Oh My! (https://euroquis.nl/bobulate/?p=1617) KDE's CI system for FreeBSD (that is, what upstream runs to continuously test KDE git code on the FreeBSD platform) is missing some bits and failing some tests because of Wayland. Or rather, because FreeBSD now has Wayland, but not Qt5-Wayland, and no Weston either (the reference implementation of a Wayland compositor). Today I went hunting for the bits and pieces needed to make that happen. Fortunately, all the heavy lifting has already been done: there is a Weston port prepared and there was a Qt5-Wayland port well-hidden in the Area51 plasma5/ branch. I have taken the liberty of pulling them into the Area51 repository as branch qtwayland. That way we can nudge Weston forward, and/or push Qt5-Wayland in separately. Nicest from a testing perspective is probably doing both at the same time. I picked a random “Hello World” Wayland tutorial and also built a minimal Qt program (using QMessageBox::question, my favorite function to hate right now, because of its i18n characteristics). Then, setting XDGRUNTIMEDIR to /tmp/xdg, I could start Weston (as an X11 client), wayland-hello (as a Wayland client, displaying in Weston) and qt-hello (as either an X11 client, or as a Wayland client). So this gives users of Area51 (while shuffling branches, granted) a modern desktop and modern display capabilities. Oh my! It will take a few days for this to trickle up and/or down so that the CI can benefit and we can make sure that KWin's tests all work on FreeBSD, but it's another good step towards tight CI and another small step towards KDE Plasma 5 on the desktop on FreeBSD. pkgsrcCon 2017 report (https://blog.netbsd.org/tnf/entry/pkgsrccon_2017_report) This years pkgsrcCon returned to London once again. It was last held in London back in 2014. The 2014 con was the first pkgsrcCon I attended, I had been working on Darwin/PowerPC fixes for some months and presented on the progress I'd made with a 12" G4 PowerBook. I took away a G4 Mac Mini that day to help spare the PowerBook for use and dedicate a machine for build and testing. The offer of PowerPC hardware donations was repeated at this years con, thanks to jperkin@ who showed up with a backpack full of Mac Minis (more on that later). Since 2014 we have held cons in Berlin (2015) & Krakow (2016). In Krakow we had talks about a wide range of projects over 2 days, from Haiku Ports to Common Lisp to midipix (building native PE binaries for Windows) and back to the BSDs. I was very pleased to continue the theme of a diverse program this year. Aside from pkgsrc and NetBSD, we had talks about FreeBSD, OpenBSD, Slackware Linux, and Plan 9. Things began with a pub gathering on the Friday for the pre-con social, we hung out and chatted till almost midnight on a wide range of topics, such as supporting a system using NFS on MS-DOS, the origins of pdksh, corporate IT, culture and many other topics. On parting I was asked about the starting time on Saturday as there was some conflicting information. I learnt that the registration email had stated a later start than I had scheduled for & advertised on the website, by 30 minutes. Lesson learnt: register for your own event! Not a problem, I still needed to setup a webpage for the live video stream, I could do both when I got back. With some trimming here and there I had a new schedule, I posted that to the pkgsrcCon website and moved to trying to setup a basic web page which contained a snippet of javascript to play a live video stream from Scale Engine. 2+ hours later, it was pointed out that the XSS protection headers on pkgsrc.org breaks the functionality. Thanks to jmcneill@ for debugging and providing a working page. Saturday started off with Giovanni Bechis speaking about pledge in OpenBSD and adding support to various packages in their ports tree, alnsn@ then spoke about installing packages from a repo hosted on the Tor network. After a quick coffee break we were back to hear Charles Forsyth speak about how Plan 9 and Inferno dealt with portability, building software and the problem which are avoided by the environment there. This was followed by a very energetic rant by David Spencer from the Slackbuilds project on packaging 3rd party software. Slackbuilds is a packaging system for Slackware Linux, which was inspired by FreeBSD ports. For the first slot after lunch, agc@ gave a talk on the early history of pkgsrc followed by Thomas Merkel on using vagrant to test pkgsrc changes with ease, locally, using vagrant. khorben@ covered his work on adding security to pkgsrc and bsiegert@ covered the benefits of performing our bulk builds in the cloud and the challenges we currently face. My talk was about some topics and ideas which had inspired me or caught my attention, and how it could maybe apply to my work.The title of the talk was taken from the name of Andrew Weatherall's Saint Etienne remix, possibly referring to two different styles of track (dub & vocal) merged into one or something else. I meant it in terms of applicability of thoughts and ideas. After me, agc@ gave a second talk on the evolution of the Netflix Open Connect appliance which runs FreeBSD and Vsevolod Stakhov wrapped up the day with a talk about the technical implementation details of the successor to pkgtools in FreeBSD, called pkg, and how it could be of benefit for pkgsrc. For day 2 we gathered for a hack day at the London Hack Space. I had burn't some some CD of the most recent macppc builds of NetBSD 8.0BETA and -current to install and upgrade Mac Minis. I setup the donated G4 minis for everyone in a dual-boot configuration and moved on to taking apart my MacBook Air to inspect the wifi adapter as I wanted to replace it with something which works on FreeBSD. It was not clear from the ifixit teardown photos of cards size, it seemed like a normal mini-PCIe card but it turned out to be far smaller. Thomas had also had the same card in his and we are not alone. Thomas has started putting together a driver for the Broadcom card, the project is still in its early days and lacks support for encrypted networks but hopefully it will appear on review.freebsd.org in the future. weidi@ worked on fixing SunOS bugs in various packages and later in the night we setup a NetBSD/macppc bulk build environment together on his Mac Mini. Thomas setup an OpenGrock instance to index the source code of all the software available for packaging in pkgsrc. This helps make the evaluation of changes easier and the scope of impact a little quicker without having to run through a potentially lengthy bulk build with a change in mind to realise the impact. bsiegert@ cleared his ticket and email backlog for pkgsrc and alnsn@ got NetBSD/evbmips64-eb booting on his EdgeRouter Lite. On Monday we reconvened at the Hack Space again and worked some more. I started putting together the talks page with the details from Saturday and the the slides which I had received, in preparation for the videos which would come later in the week. By 3pm pkgsrcCon was over. I was pretty exhausted but really pleased to have had a few days of techie fun. Many thanks to The NetBSD Foundation for purchasing a camera to use for streaming the event and a speedy response all round by the board. The Open Source Specialist Group at BCS, The Chartered Institute for IT and the London Hack Space for hosting us. Scale Engine for providing streaming facility. weidi@ for hosting the recorded videos. Allan Jude for pointers, Jared McNeill for debugging, NYCBUG and Patrick McEvoy for tips on streaming, the attendees and speakers. This year we had speakers from USA, Italy, Germany and London E2. Looking forward to pkgsrcCon 2018! The videos and slides are available here (http://www.pkgsrc.org/pkgsrcCon/2017/talks.html) and the Internet Archive (http://archive.org/details/pkgsrcCon-2017). News Roundup QuickAssist Driver for FreeBSD is here and pfSense Support Coming (https://www.servethehome.com/quickassist-driver-freebsd-pfsupport-coming/) This week we have something that STH readers will be excited about. Before I started writing for STH, I was a reader and had been longing for QuickAssist support ever since STH's first Rangeley article over three and a half years ago. It was clear from the get-go that Rangeley was going to be the preeminent firewall appliance platform of its day. The scope of products that were impacted by the Intel Atom C2000 series bug showed us it was indeed. For my personal firewalls, I use pfSense on that Rangeley platform so I have been waiting to use QuickAssist with my hardware for almost an entire product generation. + New Hardware and QuickAssist Incoming to pfSense (Finally) pfSense (and a few other firewalls) are based on FreeBSD. FreeBSD tends to lag driver support behind mainstream Linux but it is popular for embedded security appliances. While STH is the only site to have done QuickAssist benchmarks for OpenSSL and IPSec VPNs pre-Skylake, we expect more platforms to use it now that the new Intel Xeon Scalable Processor Family is out. With the Xeon Scalable platforms, the “Lewisburg” PCH has QuickAssist options of up to 100Gbps, or 2.5x faster than the previous generation add-in cards we tested (40Gbps.) We now have more and better hardware for QAT, but we were still devoid of a viable FreeBSD QAT driver from Intel. That has changed. Our Intel Xeon Scalable Processor Family (Skylake-SP) Launch Coverage Central has been the focus of the STH team's attention this week. There was another important update from Intel that got buried, a publicly available Intel QuickAssist driver for FreeBSD. You can find the driver on 01.org here dated July 12, 2017. Drivers are great, but we still need support to be enabled in the OS and at the application layer. Patrick forwarded me this tweet from Jim Thompson (lead at Netgate the company behind pfSense): The Netgate team has been a key company pushing QuickAssist appliances in the market, usually based on Linux. To see that QAT is coming to FreeBSD and that they were working to integrate into “pfSense soon” is more than welcome. For STH readers, get ready. It appears to be actually and finally happening. QuickAssist on FreeBSD and pfSense OpenBSD on the Huawei MateBook X (https://jcs.org/2017/07/14/matebook) The Huawei MateBook X is a high-quality 13" ultra-thin laptop with a fanless Core i5 processor. It is obviously biting the design of the Apple 12" MacBook, but it does have some notable improvements such as a slightly larger screen, a more usable keyboard with adequate key travel, and 2 USB-C ports. It also uses more standard PC components than the MacBook, such as a PS/2-connected keyboard, removable m.2 WiFi card, etc., so its OpenBSD compatibility is quite good. In contrast to the Xiaomi Mi Air, the MateBook is actually sold (2) in the US and comes with a full warranty and much higher build quality (though at twice the price). It is offered in the US in a "space gray" color for the Core i5 model and a gold color for the Core i7. The fanless Core i5 processor feels snappy and doesn't get warm during normal usage on OpenBSD. Doing a make -j4 build at full CPU speed does cause the laptop to get warm, though the palmrest maintains a usable temperature. The chassis is all aluminum and has excellent rigidity in the keyboard area. The 13.0" 2160x1440 glossy IPS "Gorilla glass" screen has a very small bezel and its hinge is properly weighted to allow opening the lid with one hand. There is no wobble in the screen when open, even when jostling the desk that the laptop sits on. It has a reported brightness of 350 nits. I did not experience any of the UEFI boot variable problems that I did with the Xiaomi, and the MateBook booted quickly into OpenBSD after re-initializing the GPT table during installation. OpenSMTPD under OpenBSD with SSL/VirtualUsers/Dovecot (https://blog.cagedmonster.net/opensmtpd-under-openbsd-with-ssl-virtualusers-dovecot/) During the 2013 AsiaBSDCon, the team of OpenBSD presented its mail solution named OpenSMTPD. Developed by the OpenBSD team, we find the so much appreciated philosophy of its developers : security, simplicity / clarity and advanced features. Basic configuration : OpenSMTPD is installed by default, we can immediately start with a simple configuration. > We listen on our interfaces, we specify the path of our aliases file so we can manage redirections. > Mails will be delivered for the domain cagedmonster.net to mbox (the local users mailbox), same for the aliases. > Finally, we accept to relay local mails exclusively. > We can now enable smtpd at system startup and start the daemon. Advanced configuration including TLS : You can use SSL with : A self-signed certificate (which will not be trusted) or a certificate generated by a trusted authority. LetsEncrypt uses Certbot to generated your certificate. You can check this page for further informations. Let's focus on the first. Generation of the certificate : We fix the permissions : We edit the config file : > We have a mail server with SSL, it's time to configure our IMAP server, Dovecot, and manage the creation of virtual users. Dovecot setup, and creation of Virtual Users : We will use the package system of OpenBSD, so please check the configuration of your /etc/pkg.conf file. Enable the service at system startup : Setup the Virtual Users structure : Adding the passwd table for smtpd : Modification of the OpenSMTPD configuration : We declare the files used for our Virtual Accounts, we include SSL, and we configure mails delivery via the Dovecot lmtp socket. We'll create our user lina@cagedmonster.net and set its password. Configure SSL Configure dovecot.conf Configure mail.con Configure login.conf : Make sure that the value of openfiles-cur in /etc/login.conf is equal or superior of 1000 ! Starting Dovecot *** OpenSMTPD and Dovecot under OpenBSD with MySQL support and SPAMD (https://blog.cagedmonster.net/opensmtpd-and-dovecot-under-openbsd-with-mysql-support-and-spamd/) This article is the continuation of my previous tutorial OpenSMTPD under OpenBSD with SSL/VirtualUsers/Dovecot. We'll use the same configuration and add some features so we can : Use our domains, aliases, virtual users with a MySQL database (MariaDB under OpenBSD). Deploy SPAMD with OpenSMTPD for a strong antispam solution. + Setup of the MySQL support for OpenSMTPD & Dovecot + We create our SQL database named « smtpd » + We create our SQL user « opensmtpd » we give him the privileges on our SQL database and we set its password + We create the structure of our SQL database + We generate our password with Blowfish (remember it's OpenBSD !) for our users + We create our tables and we include our datas + We push everything to our database + Time to configure OpenSMTPD + We create our mysql.conf file and configure it + Configuration of Dovecot.conf + Configuration of auth-sql.conf.ext + Configuration of dovecot-sql.conf.ext + Restart our services OpenSMTPD & SPAMD : SPAMD is a service simulating a fake SMTP server and relying on strict compliance with RFC to determine whether the server delivering a mail is a spammer or not. + Configuration of SPAMD : + Enable SPAMD & SPAMLOGD at system startup : + Configuration of SPAMD flags + Configuration of PacketFilter + Configuration of SPAMD + Start SPAMD & SPAMLOGD Running a TOR relay on FreeBSD (https://networkingbsdblog.wordpress.com/2017/07/14/freebsd-tor-relay-using-priveledge-seperation/) There are 2 main steps to getting a TOR relay working on FreeBSD: Installing and configuring Tor Using an edge router to do port translation In my case I wanted TOR to run it's services on ports 80 and 443 but any port under 1024 requires root access in UNIX systems. +So I used port mapping on my router to map the ports. +Begin by installing TOR and ARM from: /usr/ports/security/tor/ /usr/ports/security/arm/ Arm is the Anonymizing Relay Monitor: https://www.torproject.org/projects/arm.html.en It provides useful monitoring graph and can be used to configure the torrc file. Next step edit the torrc file (see Blog article for the edit) It is handy to add the following lines to /etc/services so you can more easily modify your pf configuration. torproxy 9050/tcp #torsocks torOR 9090/tcp #torOR torDIR 9099/tcp #torDIR To allow TOR services my pf.conf has the following lines: # interfaces lan_if=”re0″ wifi_if=”wlan0″ interfaces=”{wlan0,re0}” tcp_services = “{ ssh torproxy torOR torDIR }” # options set block-policy drop set loginterface $lan_if # pass on lo set skip on lo scrub in on $lan_if all fragment reassemble # NAT nat on $lan_if from $wifi_if:network to !($lan_if) -> ($lan_if) block all antispoof for $interfaces #In NAT pass in log on $wifi_if inet pass out all keep state #ICMP pass out log inet proto icmp from any to any keep state pass in log quick inet proto icmp from any to any keep state #SSH pass in inet proto tcp to $lan_if port ssh pass in inet proto tcp to $wifi_if port ssh #TCP Services on Server pass in inet proto tcp to $interfaces port $tcp_services keep state The finally part is mapping the ports as follows: TOR directory port: LANIP:9099 —> WANIP:80 TOR router port: LANIP:9090 —-> WANIP:443 Now enable TOR: $ sudo echo “tor_enable=YES” >> /etc/rc.conf Start TOR: $ sudo service tor start *** Beastie Bits OpenBSD as a “Desktop” (Laptop) (http://unixseclab.com/index.php/2017/06/12/openbsd-as-a-desktop-laptop/) Sascha Wildner has updated ACPICA in DragonFly to Intel's version 20170629 (http://lists.dragonflybsd.org/pipermail/commits/2017-July/625997.html) Dport, Rust, and updates for DragonFlyBSD (https://www.dragonflydigest.com/2017/07/18/19991.html) OPNsense 17.7 RC1 released (https://opnsense.org/opnsense-17-7-rc1/) Unix's mysterious && and || (http://www.networkworld.com/article/3205148/linux/unix-s-mysterious-andand-and.html#tk.rss_unixasasecondlanguage) The Commute Deck : A Homebrew Unix terminal for tight places (http://boingboing.net/2017/06/16/cyberspace-is-everting.html) FreeBSD 11.1-RC3 now available (https://lists.freebsd.org/pipermail/freebsd-stable/2017-July/087407.html) Installing DragonFlyBSD with ORCA when you're totally blind (http://lists.dragonflybsd.org/pipermail/users/2017-July/313528.html) Who says FreeBSD can't look good (http://imgur.com/gallery/dc1pu) Pratik Vyas adds the ability to do paused VM migrations for VMM (http://undeadly.org/cgi?action=article&sid=20170716160129) Feedback/Questions Hrvoje - OpenBSD MP Networking (http://dpaste.com/0EXV173#wrap) Goran - debuggers (http://dpaste.com/1N853NG#wrap) Abhinav - man-k (http://dpaste.com/1JXQY5E#wrap) Liam - university setup (http://dpaste.com/01ERMEQ#wrap)
This week on BSDNow, reports from AsiaBSDcon, TrueOS and FreeBSD news, Optimizing IllumOS Kernel, your questions and more. This episode was brought to you by Headlines AsiaBSDcon Reports and Reviews () AsiaBSDcon schedule (https://2017.asiabsdcon.org/program.html.en) Schedule and slides from the 4th bhyvecon (http://bhyvecon.org/) Michael Dexter's trip report on the iXsystems blog (https://www.ixsystems.com/blog/ixsystems-attends-asiabsdcon-2017) NetBSD AsiaBSDcon booth report (http://mail-index.netbsd.org/netbsd-advocacy/2017/03/13/msg000729.html) *** TrueOS Community Guidelines are here! (https://www.trueos.org/blog/trueos-community-guidelines/) TrueOS has published its new Community Guidelines The TrueOS Project has existed for over ten years. Until now, there was no formally defined process for interested individuals in the TrueOS community to earn contributor status as an active committer to this long-standing project. The current core TrueOS developers (Kris Moore, Ken Moore, and Joe Maloney) want to provide the community more opportunities to directly impact the TrueOS Project, and wish to formalize the process for interested people to gain full commit access to the TrueOS repositories. These describe what is expected of community members and committers They also describe the process of getting commit access to the TrueOS repo: Previously, Kris directly handed out commit bits. Now, the Core developers have provided a small list of requirements for gaining a TrueOS commit bit: Create five or more pull requests in a TrueOS Project repository within a single six month period. Stay active in the TrueOS community through at least one of the available community channels (Gitter, Discourse, IRC, etc.). Request commit access from the core developers via core@trueos.org OR Core developers contact you concerning commit access. Pull requests can be any contribution to the project, from minor documentation tweaks to creating full utilities. At the end of every month, the core developers review the commit logs, removing elements that break the Project or deviate too far from its intended purpose. Additionally, outstanding pull requests with no active dissension are immediately merged, if possible. For example, a user submits a pull request which adds a little-used OpenRC script. No one from the community comments on the request or otherwise argues against its inclusion, resulting in an automatic merge at the end of the month. In this manner, solid contributions are routinely added to the project and never left in a state of “limbo”. The page also describes the perks of being a TrueOS committer: Contributors to the TrueOS Project enjoy a number of benefits, including: A personal TrueOS email alias: @trueos.org Full access for managing TrueOS issues on GitHub. Regular meetings with the core developers and other contributors. Access to private chat channels with the core developers. Recognition as part of an online Who's Who of TrueOS developers. The eternal gratitude of the core developers of TrueOS. A warm, fuzzy feeling. Intel Donates 250.000 $ to the FreeBSD Foundation (https://www.freebsdfoundation.org/news-and-events/latest-news/new-uranium-level-donation-and-collaborative-partnership-with-intel/) More details about the deal: Systems Thinking: Intel and the FreeBSD Project (https://www.freebsdfoundation.org/blog/systems-thinking-intel-and-the-freebsd-project/) Intel will be more actively engaging with the FreeBSD Foundation and the FreeBSD Project to deliver more timely support for Intel products and technologies in FreeBSD. Intel has contributed code to FreeBSD for individual device drivers (i.e. NICs) in the past, but is now seeking a more holistic “systems thinking” approach. Intel Blog Post (https://01.org/blogs/imad/2017/intel-increases-support-freebsd-project) We will work closely with the FreeBSD Foundation to ensure the drivers, tools, and applications needed on Intel® SSD-based storage appliances are available to the community. This collaboration will also provide timely support for future Intel® 3D XPoint™ products. Thank you very much, Intel! *** Applied FreeBSD: Basic iSCSI (https://globalengineer.wordpress.com/2017/03/05/applied-freebsd-basic-iscsi/) iSCSI is often touted as a low-cost replacement for fibre-channel (FC) Storage Area Networks (SANs). Instead of having to setup a separate fibre-channel network for the SAN, or invest in the infrastructure to run Fibre-Channel over Ethernet (FCoE), iSCSI runs on top of standard TCP/IP. This means that the same network equipment used for routing user data on a network could be utilized for the storage as well. This article will cover a very basic setup where a FreeBSD server is configured as an iSCSI Target, and another FreeBSD server is configured as the iSCSI Initiator. The iSCSI Target will export a single disk drive, and the initiator will create a filesystem on this disk and mount it locally. Advanced topics, such as multipath, ZFS storage pools, failover controllers, etc. are not covered. The real magic is the /etc/ctl.conf file, which contains all of the information necessary for ctld to share disk drives on the network. Check out the man page for /etc/ctl.conf for more details; below is the configuration file that I created for this test setup. Note that on a system that has never had iSCSI configured, there will be no existing configuration file, so go ahead and create it. Then, enable ctld and start it: sysrc ctld_enable=”YES” service ctld start You can use the ctladm command to see what is going on: root@bsdtarget:/dev # ctladm lunlist (7:0:0/0): Fixed Direct Access SPC-4 SCSI device (7:0:1/1): Fixed Direct Access SPC-4 SCSI device root@bsdtarget:/dev # ctladm devlist LUN Backend Size (Blocks) BS Serial Number Device ID 0 block 10485760 512 MYSERIAL 0 MYDEVID 0 1 block 10485760 512 MYSERIAL 1 MYDEVID 1 Now, let's configure the client side: In order for a FreeBSD host to become an iSCSI Initiator, the iscsd daemon needs to be started. sysrc iscsid_enable=”YES” service iscsid start Next, the iSCSI Initiator can manually connect to the iSCSI target using the iscsictl tool. While setting up a new iSCSI session, this is probably the best option. Once you are sure the configuration is correct, add the configuration to the /etc/iscsi.conf file (see man page for this file). For iscsictl, pass the IP address of the target as well as the iSCSI IQN for the session: + iscsictl -A -p 192.168.22.128 -t iqn.2017-02.lab.testing:basictarget You should now have a new device (check dmesg), in this case, da1 The guide them walks through partitioning the disk, and laying down a UFS file system, and mounting it This it walks through how to disconnect iscsi, incase you don't want it anymore This all looked nice and easy, and it works very well. Now lets see what happens when you try to mount the iSCSI from Windows Ok, that wasn't so bad. Now, instead of sharing an entire space disk on the host via iSCSI, share a zvol. Now your windows machine can be backed by ZFS. All of your problems are solved. Interview - Philipp Buehler - pbuehler@sysfive.com (mailto:pbuehler@sysfive.com) Technical Lead at SysFive, and Former OpenBSD Committer News Roundup Half a dozen new features in mandoc -T html (http://undeadly.org/cgi?action=article&sid=20170316080827) mandoc (http://man.openbsd.org/mandoc.1)'s HTML output mode got some new features Even though mdoc(7) is a semantic markup language, traditionally none of the semantic annotations were communicated to the reader. [...] Now, at least in -T html output mode, you can see the semantic function of marked-up words by hovering your mouse over them. In terminal output modes, we have the ctags(1)-like internal search facility built around the less(1) tag jump (:t) feature for quite some time now. We now have a similar feature in -T html output mode. To jump to (almost) the same places in the text, go to the address bar of the browser, type a hash mark ('#') after the URI, then the name of the option, command, variable, error code etc. you want to jump to, and hit enter. Check out the full report by Ingo Schwarze (schwarze@) and try out these new features *** Optimizing IllumOS Kernel Crypto (http://zfs-create.blogspot.com/2014/05/optimizing-illumos-kernel-crypto.html) Sašo Kiselkov, of ZFS fame, looked into the performance of the OpenSolaris kernel crypto framework and found it lacking. The article also spends a few minutes on the different modes and how they work. Recently I've had some motivation to look into the KCF on Illumos and discovered that, unbeknownst to me, we already had an AES-NI implementation that was automatically enabled when running on Intel and AMD CPUs with AES-NI support. This work was done back in 2010 by Dan Anderson.This was great news, so I set out to test the performance in Illumos in a VM on my Mac with a Core i5 3210M (2.5GHz normal, 3.1GHz turbo). The initial tests of “what the hardware can do” were done in OpenSSL So now comes the test for the KCF. I wrote a quick'n'dirty crypto test module that just performed a bunch of encryption operations and timed the results. KCF got around 100 MB/s for each algorithm, except half that for AES-GCM OpenSSL had done over 3000 MB/s for CTR mode, 500 MB/s for CBC, and 1000 MB/s for GCM What the hell is that?! This is just plain unacceptable. Obviously we must have hit some nasty performance snag somewhere, because this is comical. And sure enough, we did. When looking around in the AES-NI implementation I came across this bit in aes_intel.s that performed the CLTS instruction. This is a problem: 3.1.2 Instructions That Cause VM Exits ConditionallyCLTS. The CLTS instruction causes a VM exit if the bits in position 3 (corresponding to CR0.TS) are set in both the CR0 guest/host mask and the CR0 read shadow. The CLTS instruction signals to the CPU that we're about to use FPU registers (which is needed for AES-NI), which in VMware causes an exit into the hypervisor. And we've been doing it for every single AES block! Needless to say, performing the equivalent of a very expensive context switch every 16 bytes is going to hurt encryption performance a bit. The reason why the kernel is issuing CLTS is because for performance reasons, the kernel doesn't save and restore FPU register state on kernel thread context switches. So whenever we need to use FPU registers inside the kernel, we must disable kernel thread preemption via a call to kpreemptdisable() and kpreemptenable() and save and restore FPU register state manually. During this time, we cannot be descheduled (because if we were, some other thread might clobber our FPU registers), so if a thread does this for too long, it can lead to unexpected latency bubbles The solution was to restructure the AES and KCF block crypto implementations in such a way that we execute encryption in meaningfully small chunks. I opted for 32k bytes, for reasons which I'll explain below. Unfortunately, doing this restructuring work was a bit more complicated than one would imagine, since in the KCF the implementation of the AES encryption algorithm and the block cipher modes is separated into two separate modules that interact through an internal API, which wasn't really conducive to high performance (we'll get to that later). Anyway, having fixed the issue here and running the code at near native speed, this is what I get: AES-128/CTR: 439 MB/s AES-128/CBC: 483 MB/s AES-128/GCM: 252 MB/s Not disastrous anymore, but still, very, very bad. Of course, you've got keep in mind, the thing we're comparing it to, OpenSSL, is no slouch. It's got hand-written highly optimized inline assembly implementations of most of these encryption functions and their specific modes, for lots of platforms. That's a ton of code to maintain and optimize, but I'll be damned if I let this kind of performance gap persist. Fixing this, however, is not so trivial anymore. It pertains to how the KCF's block cipher mode API interacts with the cipher algorithms. It is beautifully designed and implemented in a fashion that creates minimum code duplication, but this also means that it's inherently inefficient. ECB, CBC and CTR gained the ability to pass an algorithm-specific "fastpath" implementation of the block cipher mode, because these functions benefit greatly from pipelining multiple cipher calls into a single place. ECB, CTR and CBC decryption benefit enormously from being able to exploit the wide XMM register file on Intel to perform encryption/decryption operations on 8 blocks at the same time in a non-interlocking manner. The performance gains here are on the order of 5-8x.CBC encryption benefits from not having to copy the previously encrypted ciphertext blocks into memory and back into registers to XOR them with the subsequent plaintext blocks, though here the gains are more modest, around 1.3-1.5x. After all of this work, this is how the results now look on Illumos, even inside of a VM: Algorithm/Mode 128k ops AES-128/CTR: 3121 MB/s AES-128/CBC: 691 MB/s AES-128/GCM: 1053 MB/s So the CTR and GCM speeds have actually caught up to OpenSSL, and CBC is actually faster than OpenSSL. On the decryption side of things, CBC decryption also jumped from 627 MB/s to 3011 MB/s. Seeing these performance numbers, you can see why I chose 32k for the operation size in between kernel preemption barriers. Even on the slowest hardware with AES-NI, we can expect at least 300-400 MB/s/core of throughput, so even in the worst case, we'll be hogging the CPU for at most ~0.1ms per run. Overall, we're even a little bit faster than OpenSSL in some tests, though that's probably down to us encrypting 128k blocks vs 8k in the "openssl speed" utility. Anyway, having fixed this monstrous atrocity of a performance bug, I can now finally get some sleep. To made these tests repeatable, and to ensure that the changes didn't break the crypto algorithms, Saso created a crypto_test kernel module. I have recently created a FreeBSD version of crypto_test.ko, for much the same purposes Initial performance on FreeBSD is not as bad, if you have the aesni.ko module loaded, but it is not up to speed with OpenSSL. You cannot directly compare to the benchmarks Saso did, because the CPUs are vastly different. Performance results (https://wiki.freebsd.org/OpenCryptoPerformance) I hope to do some more tests on a range of different sized CPUs in order to determine how the algorithms scale across different clock speeds. I also want to look at, or get help and have someone else look at, implementing some of the same optimizations that Saso did. It currently seems like there isn't a way to perform addition crypto operations in the same session without regenerating the key table. Processing additional buffers in an existing session might offer a number of optimizations for bulk operations, although in many cases, each block is encrypted with a different key and/or IV, so it might not be very useful. *** Brendan Gregg's special freeware tools for sysadmins (http://www.brendangregg.com/specials.html) These tools need to be in every (not so) serious sysadmins toolbox. Triple ROT13 encryption algorithm (beware: export restrictions may apply) /usr/bin/maybe, in case true and false don't provide too little choice... The bottom command lists you all the processes using the least CPU cycles. Check out the rest of the tools. You wrote similar tools and want us to cover them in the show? Send us an email to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** A look at 2038 (http://www.lieberbiber.de/2017/03/14/a-look-at-the-year-20362038-problems-and-time-proofness-in-various-systems/) I remember the Y2K problem quite vividly. The world was going crazy for years, paying insane amounts of money to experts to fix critical legacy systems, and there was a neverending stream of predictions from the media on how it's all going to fail. Most didn't even understand what the problem was, and I remember one magazine writing something like the following: Most systems store the current year as a two-digit value to save space. When the value rolls over on New Year's Eve 1999, those two digits will be “00”, and “00” means “halt operation” in the machine language of many central processing units. If you're in an elevator at this time, it will stop working and you may fall to your death. I still don't know why they thought a computer would suddenly interpret data as code, but people believed them. We could see a nearby hydropower plant from my parents' house, and we expected it to go up in flames as soon as the clock passed midnight, while at least two airplanes crashed in our garden at the same time. Then nothing happened. I think one of the most “severe” problems was the police not being able to open their car garages the next day because their RFID tokens had both a start and end date for validity, and the system clock had actually rolled over to 1900, so the tokens were “not yet valid”. That was 17 years ago. One of the reasons why Y2K wasn't as bad as it could have been is that many systems had never used the “two-digit-year” representation internally, but use some form of “timestamp” relative to a fixed date (the “epoch”). The actual problem with time and dates rolling over is that systems calculate timestamp differences all day. Since a timestamp derived from the system clock seemingly only increases with each query, it is very common to just calculate diff = now - before and never care about the fact that now could suddenly be lower than before because the system clock has rolled over. In this case diff is suddenly negative, and if other parts of the code make further use of the suddenly negative value, things can go horribly wrong. A good example was a bug in the generator control units (GCUs) aboard Boeing 787 “Dreamliner” aircrafts, discovered in 2015. An internal timestamp counter would overflow roughly 248 days after the system had been powered on, triggering a shut down to “safe mode”. The aircraft has four generator units, but if all were powered up at the same time, they would all fail at the same time. This sounds like an overflow caused by a signed 32-bit counter counting the number of centiseconds since boot, overflowing after 248.55 days, and luckily no airline had been using their Boing 787 models for such a long time between maintenance intervals. The “obvious” solution is to simply switch to 64-Bit values and call it day, which would push overflow dates far into the future (as long as you don't do it like the IBM S/370 mentioned before). But as we've learned from the Y2K problem, you have to assume that computer systems, computer software and stored data (which often contains timestamps in some form) will stay with us for much longer than we might think. The years 2036 and 2038 might be far in the future, but we have to assume that many of the things we make and sell today are going to be used and supported for more than just 19 years. Also many systems have to store dates which are far in the future. A 30 year mortgage taken out in 2008 could have already triggered the bug, and for some banks it supposedly did. sysgettimeofday() is one of the most used system calls on a generic Linux system and returns the current time in form of an UNIX timestamp (timet data type) plus fraction (susecondst data type). Many applications have to know the current time and date to do things, e.g. displaying it, using it in game timing loops, invalidating caches after their lifetime ends, perform an action after a specific moment has passed, etc. In a 32-Bit UNIX system, timet is usually defined as a signed 32-Bit Integer. When kernel, libraries and applications are compiled, the compiler will turn this assumption machine code and all components later have to match each other. So a 32-Bit Linux application or library still expects the kernel to return a 32-Bit value even if the kernel is running on a 64-Bit architecture and has 32-Bit compatibility. The same holds true for applications calling into libraries. This is a major problem, because there will be a lot of legacy software running in 2038. Systems which used an unsigned 32-Bit Integer for timet push the problem back to 2106, but I don't know about many of those. The developers of the GNU C library (glibc), the default standard C library for many GNU/Linux systems, have come up with a design for year 2038 proofness for their library. Besides the timet data type itself, a number of other data structures have fields based on timet or the combined struct timespec and struct timeval types. Many methods beside those intended for setting and querying the current time use timestamps 32-Bit Windows applications, or Windows applications defining _USE32BITTIMET, can be hit by the year 2038 problem too if they use the timet data type. The _time64t data type had been available since Visual C 7.1, but only Visual C 8 (default with Visual Studio 2015) expanded timet to 64 bits by default. The change will only be effective after a recompilation, legacy applications will continue to be affected. If you live in a 64-Bit world and use a 64-Bit kernel with 64-Bit only applications, you might think you can just ignore the problem. In such a constellation all instances of the standard time_t data type for system calls, libraries and applications are signed 64-Bit Integers which will overflow in around 292 billion years. But many data formats, file systems and network protocols still specify 32-Bit time fields, and you might have to read/write this data or talk to legacy systems after 2038. So solving the problem on your side alone is not enough. Then the article goes on to describe how all of this will break your file systems. Not to mention your databases and other file formats. Also see Theo De Raadt's EuroBSDCon 2013 Presentation (https://www.openbsd.org/papers/eurobsdcon_2013_time_t/mgp00001.html) *** Beastie Bits Michael Lucas: Get your name in “Absolute FreeBSD 3rd Edition” (https://blather.michaelwlucas.com/archives/2895) ZFS compressed ARC stats to top (https://svnweb.freebsd.org/base?view=revision&revision=r315435) Matthew Dillon discovered HAMMER was repeating itself when writing to disk. Fixing that issue doubled write speeds (https://www.dragonflydigest.com/2017/03/14/19452.html) TedU on Meaningful Short Names (http://www.tedunangst.com/flak/post/shrt-nms-fr-clrty) vBSDcon and EuroBSDcon Call for Papers are open (https://www.freebsdfoundation.org/blog/submit-your-work-vbsdcon-and-eurobsdcon-cfps-now-open/) Feedback/Questions Craig asks about BSD server management (http://pastebin.com/NMshpZ7n) Michael asks about jails as a router between networks (http://pastebin.com/UqRwMcRk) Todd asks about connecting jails (http://pastebin.com/i1ZD6eXN) Dave writes in with an interesting link (http://pastebin.com/QzW5c9wV) > applications crash more often due to errors than corruptions. In the case of corruption, a few applications (e.g., Log-Cabin, ZooKeeper) can use checksums and redundancy to recover, leading to a correct behavior; however, when the corruption is transformed into an error, these applications crash, resulting in reduced availability. ***
This week on BSDNow, Allan and I are in Tokyo for AsiaBSDCon, but not to worry, we have a full episode lined up and ready to go. Hackathon reports This episode was brought to you by Headlines OpenBSD A2k17 hackathon reports a2k17 hackathon report: Patrick Wildt on the arm64 port (http://undeadly.org/cgi?action=article&sid=20170131101827) a2k17 hackathon report: Antoine Jacoutot on syspatch, rc.d improvements and more (http://undeadly.org/cgi?action=article&sid=20170203232049) a2k17 hackathon report: Martin Pieuchot on NET_LOCK and much more (http://undeadly.org/cgi?action=article&sid=20170127154356) a2k17 hackathon report: Kenneth Westerback on the hidden wonders of the build system, the network stack and more (http://undeadly.org/cgi?action=article&sid=20170127031836) a2k17 hackathon report: Bob Beck on LibreSSL progress and more (http://undeadly.org/cgi?action=article&sid=20170125225403) *** NetBSD is now reproducible (https://blog.netbsd.org/tnf/entry/netbsd_fully_reproducible_builds) Christos Zoulas posts to the NetBSD blog that he has completed his project to make fully reproducible NetBSD builds for amd64 and sparc64 I have been working on and off for almost a year trying to get reproducible builds (the same source tree always builds an identical cdrom) on NetBSD. I did not think at the time it would take as long or be so difficult, so I did not keep a log of all the changes I needed to make. I was also not the only one working on this. Other NetBSD developers have been making improvements for the past 6 years. I would like to acknowledge the NetBSD build system (aka build.sh) which is a fully portable cross-build system. This build system has given us a head-start in the reproducible builds work. I would also like to acknowledge the work done by the Debian folks who have provided a platform to run, test and analyze reproducible builds. Special mention to the diffoscope tool that gives an excellent overview of what's different between binary files, by finding out what they are (and if they are containers what they contain) and then running the appropriate formatter and diff program to show what's different for each file. Finally other developers who have started, motivated and did a lot of work getting us here like Joerg Sonnenberger and Thomas Klausner for their work on reproducible builds, and Todd Vierling and Luke Mewburn for their work on build.sh. Some of the stumbling blocks that were overcome: Timestamps Date/time/author embedded in source files Timezone sensitive code Directory order / build order Non-sanitized data stored in files Symbolic links / paths General tool inconsistencies: including gcc profiling, the fact that GPT partition tables, are by definition, globally unique each time they are created, and the iso9660 standard calls for a timestamp with a timezone. Toolchain Build information / tunables / environment. NetBSD now has a knob ‘MKREPRO', if set to YES it sets a long list of variables to a consistent set of a values. The post walks through how these problems where solves Future Work: Vary more parameters and find more inconsistencies Verify that cross-building is reproducible Verify that unprivileged builds are reproducible Test on other platforms *** Features are faults redux (http://www.tedunangst.com/flak/post/features-are-faults-redux) From Ted Unangst Last week I gave a talk for the security class at Notre Dame based on features are faults but with some various commentary added. It was an exciting trip, with the opportunity to meet and talk with the computer vision group as well. Some other highlights include the Indiana skillet I had for breakfast, which came with pickles and was amazing, and explaining the many wonders of cvs to the Linux users group over lunch. After that came the talk, which went a little something like this. I got started with OpenBSD back about the same time I started college, although I had a slightly different perspective then. I was using OpenBSD because it included so many security features, therefore it must be the most secure system, right? For example, at some point I acquired a second computer. What's the first thing anybody does when they get a second computer? That's right, set up a kerberos domain. The idea that more is better was everywhere. This was also around the time that ipsec was getting its final touches, and everybody knew ipsec was going to be the most secure protocol ever because it had more options than any other secure transport. We'll revisit this in a bit. There's been a partial attitude adjustment since then, with more people recognizing that layering complexity doesn't result in more security. It's not an additive process. There's a whole talk there, about the perfect security that people can't or won't use. OpenBSD has definitely switched directions, including less code, not more. All the kerberos code was deleted a few years ago. Let's assume about one bug per 100 lines of code. That's probably on the low end. Now say your operating system has 100 million lines of code. If I've done the math correctly, that's literally a million bugs. So that's one reason to avoid adding features. But that's a solveable problem. If we pick the right language and the right compiler and the right tooling and with enough eyeballs and effort, we can fix all the bugs. We know how to build mostly correct software, we just don't care. As we add features to software, increasing its complexity, new unexpected behaviors start to emerge. What are the bounds? How many features can you add before craziness is inevitable? We can make some guesses. Less than a thousand for sure. Probably less than a hundred? Ten maybe? I'll argue the answer is quite possibly two. Interesting corollary is that it's impossible to have a program with exactly two features. Any program with two features has at least a third, but you don't know what it is My first example is a bug in the NetBSD ftp client. We had one feature, we added a second feature, and just like that we got a third misfeature (http://marc.info/?l=oss-security&m=141451507810253&w=2) Our story begins long ago. The origins of this bug are probably older than I am. In the dark times before the web, FTP sites used to be a pretty popular way of publishing files. You run an ftp client, connect to a remote site, and then you can browse the remote server somewhat like a local filesystem. List files, change directories, get files. Typically there would be a README file telling you what's what, but you don't need to download a copy to keep. Instead we can pipe the output to a program like more. Right there in the ftp client. No need to disconnect. Fast forward a few decades, and http is the new protocol of choice. http is a much less interactive protocol, but the ftp client has some handy features for batch downloads like progress bars, etc. So let's add http support to ftp. This works pretty well. Lots of code reused. http has one quirk however that ftp doesn't have. Redirects. The server can redirect the client to a different file. So now you're thinking, what happens if I download http://somefile and the server sends back 302 http://|reboot. ftp reconnects to the server, gets the 200, starts downloading and saves it to a file called |reboot. Except it doesn't. The function that saves files looks at the first character of the name and if it's a pipe, runs that command instead. And now you just rebooted your computer. Or worse. It's pretty obvious this is not the desired behavior, but where exactly did things go wrong? Arguably, all the pieces were working according to spec. In order to see this bug coming, you needed to know how the save function worked, you needed to know about redirects, and you needed to put all the implications together. The post then goes into a lot more detail about other issues. We just don't have time to cover it all today, but you should go read it, it is very enlightening What do we do about this? That's a tough question. It's much easier to poke fun at all the people who got things wrong. But we can try. My attitudes are shaped by experiences with the OpenBSD project, and I think we are doing a decent job of containing the complexity. Keep paring away at dependencies and reducing interactions. As a developer, saying “no” to all feature requests is actually very productive. It's so much faster than implementing the feature. Sometimes users complain, but I've often received later feedback from users that they'd come to appreciate the simplicity. There was a question about which of these vulnerabilities were found by researchers, as opposed to troublemakers. The answer was most, if not all of them, but it made me realize one additional point I hadn't mentioned. Unlike the prototypical buffer overflow vulnerability, exploiting features is very reliable. Exploiting something like shellshock or imagetragick requires no customized assembly and is independent of CPU, OS, version, stack alignment, malloc implementation, etc. Within about 24 hours of the initial release of shellshock, I had logs of people trying to exploit it. So unless you're on about a 12 hour patch cycle, you're going to have a bad time. reimplement zfsctl (.zfs) support (https://svnweb.freebsd.org/changeset/base/314048) avg@ (Andriy Gapon) has rewritten the .zfs support in FreeBSD The current code is written on top of GFS, a library with the generic support for writing filesystems, which was ported from Illumos. Because of significant differences between illumos VFS and FreeBSD VFS models, both the GFS and zfsctl code were heavily modified to work on FreeBSD. Nonetheless, they still contain quite a few ugly hacks and bugs. This is a reimplementation of the zfsctl code where the VFS-specific bits are written from scratch and only the code that interacts with the rest of ZFS is reused. Some ideas are picked from an independent work by Will (wca@) This work improves the overall quality of the ZFS port to FreeBSD The code that provides support for ZFS .zfs/ directory functionality has been reimplemented. It is no longer possible to create a snapshot by mkdir under .zfs/snapshot/. That should be the only user visible change. TIL: On IllumOS, you can create, rename, and destroy snapshots, by manipulating the virtual directories in the .zfs/snapshots directory. If enough people would find this feature useful, maybe it could be implemented (rm and rename have never existed on FreeBSD). At the same time, it seems like rather a lot of work, when the ZFS command line tools work so well. Although wca@ pointed out on IRC, it can be useful to be able to create a snapshot over NFS, or SMB. Interview - Konrad Witaszczyk - def@freebsd.org (mailto:def@freebsd.org) Encrypted Kernel Crash Dumps *** News Roundup PBKDF2 Performance improvements on FreeBSD (https://svnweb.freebsd.org/changeset/base/313962) Joe Pixton did some research (https://jbp.io/2015/08/11/pbkdf2-performance-matters/) and found that, because of the way the spec is written, most PBKDF2 implementations are 2x slower than they need to be. Since the PBKDF is used to derive a key, used for encryption, this poses a problem. The attacker can derive a key twice as fast as you can. On FreeBSD the PBKDF2 was configured to derive a SHA512-HMAC key that would take approximately 2 seconds to calculate. That is 2 seconds on one core. So an attacker can calculate the same key in 1 second, and use many cores. Luckily, 1 second is still a long time for each brute force guess. On modern CPUs with the fast algorithm, you can do about 500,000 iterations of PBKDF per second (per core). Until a recent change, OpenBSD used only 8192 iterations. It now uses a similar benchmark of ~2 seconds, and uses bcrypt instead of a SHA1-HMAC. Joe's research showed that the majority of implementations were done the ‘slow' way. Calculating the initial part of the outer round each iteration, instead of reusing the initial calculation over and over for each round. Joe submitted a match to FreeBSD to solve this problem. That patch was improved, and a test of tests were added by jmg@, but then work stalled I picked up the work, and fixed some merge conflicts in the patch that had cropped up based on work I had done that moved the HMAC code to a separate file. This work is now committed. With this change, all newly generated GELI keys will be approximately 2x as strong. Previously generated keys will take half as long to calculate, resulting in faster mounting of encrypted volumes. Users may choose to rekey, to generate a new key with the larger default number of iterations using the geli(8) setkey command. Security of existing data is not compromised, as ~1 second per brute force attempt is still a very high threshold. If you are interested in the topic, I recommend the video of Joe's presentation from the Passwords15 conference in Las Vegas *** Quick How-To: Updating a screenshot in the TrueOS Handbook (https://www.trueos.org/blog/quick-updating-screenshot-trueos-handbook/) Docs writers, might be time to pay attention. This week we have a good walk-through of adding / updating new screenshots to the TrueOS Sphinx Documentation. For those who have not looked in the past, TrueOS and FreeNAS both have fantastic docs by the team over at iXsystems using Sphinx as their doc engine. Often we get questions from users asking what “they can do to help” but don't necessarily have programming skills to apply. The good news is that using Sphinx is relatively easy, and after learning some minio rst syntax you can easily help fix, or even contribute to new sections of the TrueOS (Or FreeNAS) documentation. In this example, Tim takes us through the process of replacing an old out of date screenshot in the handbook with the latest hotness. Starting with a .png file, he then locates the old screenshot name and adds the updated version “lumina-e.png” to “lumina-f.png”. With the file added to the tree, the relevant section of .rst code can be adjusted and the sphinx build run to verify the output HTML looks correct. Using this method you can easily start to get involved with other aspects of documentation and next thing you know you'll be writing boot-loaders like Allan! *** Learn C Programming With 9 Excellent Open Source Books (https://www.ossblog.org/learn-c-programming-with-9-excellent-open-source-books/) Now that you've easily mastered all your documentation skills, you may be ready to take on a new challenge. (Come on, that boot-loader isn't going to write itself!) We wanted to point out some excellent resources to get you started on your journey into writing C. Before you think, “oh, more books to purchase”, wait there's good news. These are the top-9 open-source books that you can download in digital form free of charge. Now I bet we got your attention. We start the rundown with “The C Book”, by Mike Banahan, Declan Brady and Mark Doran, which will lay the groundwork with your introduction into the C language and concepts. Next up, if you are going to do anything, do it with style, so take a read through the “C Elements of Style” which will make you popular at all the parties. (We can't vouch for that statement) From here we have a book on using C to build your own minimal “lisp” interpreter, reference guides on GNU C and some other excellent introduction / mastery books to help round-out your programming skill set. Your C adventure awaits, hopefully these books can not only teach you good C, but also make you feel confident when looking at bits of the FreeBSD world or kernel with a proper foundation to back it up. *** Running a Linux VM on OpenBSD (http://eradman.com/posts/linuxvm-on-openbsd.html) Over the past few years we've talked a lot about Virtualization, Bhyve or OpenBSD's ‘vmm', but qemu hasn't gotten much attention. Today we have a blog post with details on how to deploy qemu to run Linux on top of an OpenBSD host system. The starts by showing us how to first provision the storage for qemu, using the handy ‘qemu-img' command, which in this example only creates a 4GB disk, you'll probably want more for real-world usage though. Next up the qemu command will be run, pay attention to the particular flags for network and memory setup. You'll probably want to bump it up past the recommended 256M of memory. Networking is always the fun part, as the author describes his intended setup I want OpenBSD and Debian to be able to obtain an IP via DHCP on their wired interfaces and I don't want external networking required for an NFS share to the VM. To accomplish this I need two interfaces since dhclient will erase any other IPv4 addresses already assigned. We can't assign an address directly to the bridge, but we can configure a virtual Ethernet device and add it. The setup for this portion involves touching a few more files, but isn't that painless. Some “pf” rules to enable NAT for and dhcpd setup to assign a “fixed” IP to the vm will get us going, along with some additional details on how to configure the networking for inside the debian VM. Once those steps are completed you should be able to mount NFS and share data from the host to the VM painlessly. Beastie Bits MacObserver: Interview with Open Source Developer & Former Apple Manager Jordan Hubbard (https://www.macobserver.com/podcasts/background-mode-jordan-hubbard/) 2016 Google Summer of Code Mentor Summit and MeetBSD Trip Report: Gavin Atkinson (https://www.freebsdfoundation.org/blog/2016-google-summer-of-code-mentor-summit-and-meetbsd-trip-report-gavin-atkinson/) Feedback/Questions Joe - BGP / Vultr Followup (http://pastebin.com/TNyHBYwT) Ryan Moreno asks about Laptops (http://pastebin.com/s4Ypezsz) ***
This week on BSDNow! We've got Netflix + FreeBSD news to discuss, always a crowd pleaser, that plus EuroBSDCon is just around the corner. Stick around for your place This episode was brought to you by Headlines Protecting Netflix Viewing Privacy at Scale, with FreeBSD (http://techblog.netflix.com/search/label/FreeBSD) This blog post from Netflix tells the story of how Netflix developed in-kernel TLS to speed up delivery of video via HTTPS Since the beginning of the Open Connect program we have significantly increased the efficiency of our OCAs - from delivering 8 Gbps of throughput from a single server in 2012 to over 90 Gbps from a single server in 2016. We contribute to this effort on the software side by optimizing every aspect of the software for our unique use case - in particular, focusing on the open source FreeBSD operating system and the NGINX web server that run on the OCAs. In the modern internet world, we have to focus not only on efficiency, but also security. There are many state-of-the-art security mechanisms in place at Netflix, including Transport Level Security (TLS) encryption of customer information, search queries, and other confidential data. We have always relied on pre-encoded Digital Rights Management (DRM) to secure our video streams. Over the past year, we've begun to use Secure HTTP (HTTP over TLS or HTTPS) to encrypt the transport of the video content as well. This helps protect member privacy, particularly when the network is insecure - ensuring that our members are safe from eavesdropping by anyone who might want to record their viewing habits. The goal is to ensure that your government, ISP, and wifi sniffing neighbour cannot tell which Netflix videos you are watching Netflix Open Connect serves over 125 million hours of content per day, all around the world. Given our scale, adding the overhead of TLS encryption calculations to our video stream transport had the potential to greatly reduce the efficiency of our global infrastructure. We evaluated available and applicable ciphers and decided to primarily use the Advanced Encryption Standard (AES) cipher in Galois/Counter Mode (GCM), available starting in TLS 1.2. We chose AES-GCM over the Cipher Block Chaining (CBC) method, which comes at a higher computational cost. The AES-GCM cipher algorithm encrypts and authenticates the message simultaneously - as opposed to AES-CBC, which requires an additional pass over the data to generate keyed-hash message authentication code (HMAC). CBC can still be used as a fallback for clients that cannot support the preferred method. All revisions of Open Connect Appliances also have Intel CPUs that support AES-NI, the extension to the x86 instruction set designed to improve encryption and decryption performance. We needed to determine the best implementation of AES-GCM with the AES-NI instruction set, so we investigated alternatives to OpenSSL, including BoringSSL and the Intel Intelligent Storage Acceleration Library (ISA-L). Netflix and NGINX had previously worked together to improve our HTTP client request and response time via the use of sendfile calls to perform a zero-copy data flow from storage (HDD or SSD) to network socket, keeping the data in the kernel memory address space and relieving some of the CPU burden. The Netflix team specifically added the ability to make the sendfile calls asynchronous - further reducing the data path and enabling more simultaneous connections. However, TLS functionality, which requires the data to be passed to the application layer, was incompatible with the sendfile approach. To retain the benefits of the sendfile model while adding TLS functionality, we designed a hybrid TLS scheme whereby session management stays in the application space, but the bulk encryption is inserted into the sendfile data pipeline in the kernel. This extends sendfile to support encrypting data for TLS/SSL connections. We tested the BoringSSL and ISA-L AES-GCM implementations with our sendfile improvements against a baseline of OpenSSL (with no sendfile changes), under typical Netflix traffic conditions on three different OCA hardware types. Our changes in both the BoringSSL and ISA-L test situations significantly increased both CPU utilization and bandwidth over baseline - increasing performance by up to 30%, depending on the OCA hardware version. We chose the ISA-L cipher implementation, which had slightly better results. With these improvements in place, we can continue the process of adding TLS to our video streams for clients that support it, without suffering prohibitive performance hits. If you would like more detail, check out the papers from AsiaBSDCon 2015 (https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf) and the updated one from 2016 (https://people.freebsd.org/~rrs/asiabsd_tls_improved.pdf) *** OpenBSD on HP Stream 7 (http://www.tedunangst.com/flak/post/OpenBSD-on-HP-Stream-7) Recent events have rocked the mobile computing world to its core. OpenBSD retired the zaurus port, leaving users in desperate need of a new device. And not long before that, Microsoft released the Anniversary Update to Windows 10, but with free space requirements such that it's nigh impossible to install on cheap 32GB eMMC equipped devices such as the HP Stream series, leaving users searching for a new lightweight operating system. With necessity as both mother and father, the scene is set for a truly epic pairing. OpenBSD on the HP Stream 7. The HP Stream line is a series of budget computers in a couple form factors. The Stream 11 is a fairly typical netbook. However, the Stream 7 and 8 are tablets. They look like cheap Android devices, but inside the case, they're real boys, er PCs, with Intel Atom CPUs. To install OpenBSD on such a device, we need a few parts. Obviously, the tablet itself. There's a dearth of ports on these things, but there is a micro USB port. Attaching anything useful requires an OTG “on the go” cable that creates a type A port. Attaching more than one useful thing requires a mini hub. And completing the install requires one each USB stick, keyboard, and network adapter. First, we need to prep the machine to boot from USB. Actually, before doing anything, make sure you have a full charge. It's going to be battery only from here on out. Plug everything in. Flash drive, keyboard, and network into the hub, hub into the OTG cable, cable into the port on top of the Stream. Turn on the machine while holding the volume down button. This launches a mini menu from which we can enter the BIOS. There's a little on screen keyboard in the corner, so this can be done even without a keyboard attached, but the USB keyboard should work. We need to change two settings in the boot section. First, turn off secure boot. Second, switch boot order to prefer USB. Save and exit. The first reboot reveals a confirmation screen checking that we really want to disable secure boot. We must enter a PIN and press enter. Enter the PIN shown on the screen and press enter. And we are go. Then boot up OpenBSD from the USB drive Ted then works there a number of kernel panics and device driver issues, but after disabling ACPI and IntelDRM, the device boots OpenBSD. Of course, there's no X at this point. And definitely no touch screen. And no internal networking. However, by keeping our USB hub attached, we can drive the console and access the network. At least until the battery is depleted, even if we have no way of knowing how long that will be since we disabled all the ACPI devices, which also means no suspend or resume. With some xorg.conf hacking, he did get Xorg working *** DragonflyBSD steps towards base LibreSSL (http://lists.dragonflybsd.org/pipermail/commits/2016-September/624493.html) Project: DragonFlyBSD / Switch base to use private LibreSSL libraries (http://freshbsd.org/commit/dfbsd/304ca408000cd34559ef5319b4b5a6766d6eb35b) DragonFly BSD adopts uses of LibreSSL (http://undeadly.org/cgi?action=article&sid=20160911231651) The number of projects beginning to switch over to LibreSSL is growing and it appears we can now throw DragonFly into that camp. Following something that sounds vaguely familiar (Allan!) DFLY is now creating “private” LibreSSL libraries which are only linked against by base system binaries. For the moment OpenSSL is still built, primarily so that various ports and 3rd party apps can continue to function as before. A NO_OPENSSL option has also been added, but doesn't really do much (yet), since it'll still build and install headers / libraries even if set. *** OpenBSD g2k16 Hackathon g2k16 Hackathon Report: Antoine Jacoutot on Binary Patches (http://undeadly.org/cgi?action=article&sid=20160911012316) g2k16 Hackathon Report: Matthieu Herrb on xenodm (http://undeadly.org/cgi?action=article&sid=20160911231712) g2k16 Hackathon Report: Vincent Gross on iked(8), armv7 and sys/netinet[6] (http://undeadly.org/cgi?action=article&sid=20160911000337) g2k16 Hackathon Report: Florian Obser on httpd, networking, acme-client, and more (http://undeadly.org/cgi?action=article&sid=20160911000052) g2k16 Hackathon Report: Jasper Lievisse Adriaanse on ddb(4) and more (http://undeadly.org/cgi?action=article&sid=20160909012520) g2k16 Hackathon Report: Christian Weisgerber on gettext progress, RTC work, removing kernel cruft (http://undeadly.org/cgi?action=article&sid=20160908002430) g2k16 Hackathon Report: Brent Cook on Chromebooks, crypto, and more (http://undeadly.org/cgi?action=article&sid=20160907131655) g2k16 Hackathon Report: Ted Unangst on doas, signify, code removal (http://undeadly.org/cgi?action=article&sid=20160906230610) g2k16 Hackathon Report: Marc Espie on package signing evolution (http://undeadly.org/cgi?action=article&sid=20160905235911) g2k16 Hackathon Report: Adam Wolk on ports, wireless drivers and more (http://undeadly.org/cgi?action=article&sid=20160906004915) g2k16 Hackathon Report: Mike Larkin on vmm + vmd progress (http://undeadly.org/cgi?action=article&sid=20160905134009&mode=expanded) *** News Roundup OpenBSD (with encrypted softraid) on the Chromebook Pixel (https://jcs.org/notaweblog/2016/08/26/openbsd_chromebook/) Looking for a Laptop to make your OpenBSD road-warrior? If so, we have a great blog tutorial on getting OpenBSD setup on the Chromebook Pixel with encrypted softraid! Author Joshua Stein gives us a very verbose look at how to install and dial-in the laptop perfectly. But first for those wondering about the hardware in the pixel: The Chromebook Pixel LS (2015) has an Intel Core i7 processor (Broadwell) at 2.4Ghz, 16Gb of RAM, a 2560x1700 400-nit IPS screen (239ppi), and Intel 802.11ac wireless. It has a Kingston 64Gib flash chip, of which about 54Gib can be used by OpenBSD when dual-booting with a 1Gb Chrome OS partition. Due to this being a chromebook with seaBIOS, some manual key-press trickery will be required to initially get the OpenBSD Installer up and running. From here you'll want to pay special close attention to the disk partitioning. In particular Joshua will show us how to shrink the existing encrypted /home that ChromeOS uses, keeping the dual-boot intact. This will become important if you ever plan on updating the device. From here, we move back to a more traditional setup, but with the added bonus of doing a soft-raid setup. But the fun isn't over yet! If you want to make OpenBSD the default boot, that'll require cracking the lid on the device and removing a special pink write-protect screw. And of course if you want to remove the default splash-screen image, Joshua has you covered as well, although some flashrom magic will be required. At this point you are nearly done. Final details on enabling specific bits of hardware are discussed. Most things work, apart from Audio and Bluetooth as of right now. *** doas mastery (http://www.tedunangst.com/flak/post/doas-mastery) “doas” mastery - Paging MWL! Our buddy Ted Unangst has written up a great ‘mastery' guide of the doas command, which can come in handy if you are among the un-initiated in doas land. UNIX systems have two classes of user, the super user and regular users. The super user is super, and everybody else is not. This concentration of power keeps things simple, but also means that often too much power is granted. Usually we only need super user powers to perform one task. We would rather not have such power all the time. Think of the responsibility that would entail! Like the sudo command, doas allows for subdivision of super user privileges, granting them only for specific tasks. He starts with the basic doas.conf setup, which starts with an empty config file The doas config is much like a pf ruleset, the default is to block everything > We add the root rule second because doas evaluates rules in a last match manner. root is in the wheel group, so the first rule will match, and then we need to override that with a second rule. Remember to always start with general rules, then make them more specific. *** iXsystems iXsystems to host MeetBSD (https://www.ixsystems.com/blog/ixsystems-host-meetbsd-california-2016-uc-berkeley/) FreeBSD Foundation Welcomes New Board Members New Board Members (https://www.freebsdfoundation.org/blog/freebsd-foundation-welcomes-new-board-members/) The FreeBSD Foundation has added two new board members Interview with Kylie Liang (https://www.freebsdfoundation.org/blog/new-board-member-interview-kylie-liang/) Kylie will focus on representing FreeBSD at conferences and businesses in China I live in China. There, I can act as a bridge between Chinese companies and the FreeBSD community to help drive FreeBSD adoption. Through my leadership role in the FreeBSD Foundation, I will help promote FreeBSD in China and also represent the Foundation at conferences and events in my region. Kylie leads the team the ensures FreeBSD runs well on Hyper-V and Azure, including providing commercial support for customers who run FreeBSD or FreeBSD based appliances on the Azure Cloud I joined Microsoft and started to lead the project called FreeBSD Integration Service to get FreeBSD running well on Hyper-V and Azure. To promote our work and to understand the FreeBSD ecosystem, I started to participate in FreeBSD events where I was inspired by this technical community. Interview with Philip Paeps (https://www.freebsdfoundation.org/blog/new-board-member-interview-philip-paeps/) Philip started with FreeBSD in the early 2000s and got his commit bit in 2004 The patches I submitted to make ACPI and input devices work on that laptop led to a src commit bit in 2004. While I haven't worked on ACPI or input devices since, I have been contributing to different areas of the kernel. Taking up maintainership of some ports I cared about also got me a ports commit bit after some time. Philip will continue to help run EuroBSDCon, but is also spreading the word about FreeBSD in India and Africa Primarily, I think I can be useful! I attend (and organize) a number of conferences around the world every year, particularly in regions that have a mostly “stealthy” FreeBSD community. While I clearly don't need to be on the FreeBSD Foundation board to advocate for FreeBSD, joining as a director will provide an additional asset when working in areas of the world where organizational affiliations are meaningful. Philip has also developed network drivers and various other bits and pieces, and has extensive experience working with and for hardware vendors and appliance vendors Despite intending to eventually contribute their code to the FreeBSD Project as open source, many hardware vendors still find it very difficult to engage directly with the FreeBSD development community. The Foundation helps bridge that gap and helps facilitate collaboration between commercial vendors and the FreeBSD community. I hope to make FreeBSD more visible in regions of the world where it is historically under-represented. I expect I will be attending even more conferences and getting myself invited to even more organizations. more, less, and a story of typical Unix fossilization (https://utcc.utoronto.ca/~cks/space/blog/unix/MoreAndUnixFossilization) Chris Siebenmann from the University of Toronto digs into the history of the difference between ‘less' and ‘more' In the beginning, by which we mean V7, Unix didn't have a pager at all. That was okay; Unix wasn't very visual in those days, partly because it was still sort of the era of the hard copy terminal. Then along came Berkeley and BSD. People at Berkeley were into CRT terminals, and so BSD Unix gave us things like vi and the first pager program, more (which showed up quite early, in 3BSD, although this isn't as early as vi, which appears in 2BSD). Calling a pager more is a little bit odd but it's a Unix type of name and from the beginning more prompted you with '--More--' at the bottom of the screen. All of the Unix vendors that based their work on BSD Unix (like Sun and DEC) naturally shipped versions of more along with the rest of the BSD programs, and so more spread around the BSD side of things. However, more was by no means the best pager ever; as you might expect, it was actually a bit primitive and lacking in features. So fairly early on Mark Nudelman wrote a pager with somewhat more features and it wound up being called less as somewhat of a joke. In a sane world, Unix vendors would have either replaced their version of more with the clearly superior less or at least updated their version of more to the 4.3 BSD version. Maybe less wouldn't have replaced more immediately, but certainly over say the next five years, when it kept on being better and most people kept preferring it when they had a choice.” + “This entire history has led to a series of vaguely absurd outcomes on various modern Unixes. On Solaris derivatives more is of course the traditional version with source code that can probably trace itself all the way back to 3BSD, carefully updated to SUS compliance. Solaris would never dream of changing what more is, not even if the replacement is better. Why, it might disturb someone. Oddly, FreeBSD has done the most sensible thing; they've outright replaced more with less. There is a /usr/bin/more but it's the same binary as less and as you can see the more manpage is just the less manpage. OpenBSD has done the same thing but has a specific manpage for more instead of just giving you the less manpage. So, now you can see why I say that less is more, or more, or both, at several levels. less is certainly more than more, and sometimes less literally is more (or rather more is less, to put it the right way around). Beastie Bits PC-BSD listed in the top 8 'best' alternatives to Windows 10 (http://www.computerworlduk.com/galleries/operating-systems/-free-alternatives-windows-10-3639433/) Creating a quick DNS server with a Rapsberry Pi2 and FreeBSD 11.0-RC1 (http://bsdimp.blogspot.co.uk/2016/08/creating-quick-dns-server-with.html) Dual Boot OpenBSD and Linux + UEFI (https://bsdlaptops.wordpress.com/2016/03/07/vaio-pro-11-part-2/) DesktopBSD 2.0 various versions available (Gnome, Lumina, KDE, LXDE) (http://desktopbsd.boards.net/board/10/announcements) FreeBSD gets new ZFS features including: Compressed ARC (https://svnweb.freebsd.org/base?view=revision&revision=305323) and ZFS Allocation Throttle (https://svnweb.freebsd.org/base?view=revision&revision=305331) One Floppy NetBSD Distribution (https://github.com/user340/fdgw2) A Compendium of BUGs (https://github.com/q5sys/BUGtracker) Feedback/Questions Galahad - OpenBSD X setup (http://pastebin.com/b7W6NHqs) Tang - Subtitles (http://pastebin.com/P4MUs3Pa) Ivan - Zpool Options (http://pastebin.com/LQ8yTp0G) Brad - Replication Issue (http://pastebin.com/XTK5gXMU) MJ - HBA (http://pastebin.com/TdYTMSj9) ***
This week on BSDNow, we have an interview with Matthew Macy, who has some exciting news to share with us regarding the state of graphics This episode was brought to you by Headlines How the number of states affects pf's performance of FreeBSD (http://blog.cochard.me/2016/05/playing-with-freebsd-packet-filter.html) Our friend Olivier of FreeNAS and BSDRP fame has an interesting blog post this week detailing his unique issue with finding a firewall that can handle upwards of 4 million state table entries. He begins in the article with benchmarking the defaults, since without that we don't have a framework to compare the later results. All done on his Netgate RCC-VE 4860 (4 cores ATOM C2558, 8GB RAM) under FreeBSD 10.3. “We notice a little performance impact when we reach the default 10K state table limit: From 413Kpps with 128 states in-used, it lower to 372Kpps.” With the initial benchmarks done and graphed, he then starts the tuning process by adjusting the “net.pf.states_hashsize”sysctl, and then playing with the number of states for the firewall to keep. “For the next bench, the number of flow will be fixed for generating 9800 pf state entries, but I will try different value of pf.states_hashsize until the maximum allowed on my 8GB RAM server (still with the default max states of 10k):” Then he cranks it up to 4 million states “There is only 12% performance penalty between pf 128 pf states and 4 million pf states.” “With 10M state, pf performance lower to 362Kpps: Still only 12% lower performance than with only 128 states” He then looks at what this does of pfsync, the protocol to sync the state table between two redundant pf firewalls Conclusions: There need to be a linear relationship between the pf hard-limit of states and the pf.stateshashsize; RAM needed for pf.stateshashsize = pf.stateshashsize * 80 Byte and pf.stateshashsize should be a power of 2 (from the manual page); Even small hardware can manage large number of sessions (it's a matter of RAM), but under too lot's of pressure pfsync will suffer. Introducing the BCHS Stack = BSD, C, httpd, SQLite (http://www.learnbchs.org/) Pronounced Beaches “It's a hipster-free, open source software stack for web applications” “Don't just write C. Write portable and secure C.” “Get to know your security tools. OpenBSD has systrace(4) and pledge(2). FreeBSD has capsicum(4).” “Statically scan your binary with LLVM” and “Run your application under valgrind” “Don't forget: BSD is a community of professionals. Go to conferences (EuroBSDCon, AsiaBSDCon, BSDCan, etc.)” This seems like a really interesting project, we'll have to get Kristaps Dzonsons back on the show to talk about it *** Installing OpenBSD's httpd server, MariaDB, PHP 5.6 on OpenBSD 5.9 (https://www.rootbsd.net/kb/339/Installing-OpenBSDandsharp039s-httpd-server-MariaDB-PHP-56-on-OpenBSD-59.html) Looking to deploy your next web-stack on OpenBSD 5.9? If so this next article from rootbsd.net is for you. Specifically it will walk you through the process of getting OpenBSD's own httpd server up and running, followed by MariaDB and PHP 5.6. Most of the setup is pretty straight-forward, the httpd syntax may be different to you, if this is your first time trying it out. Once the various packages are installed / configured, the rest of the tutorial will be easy, walking you through the standard hello world PHP script, and enabling the services to run at reboot. A good article for those wanting to start hosting PHP/DB content (wordpress anyone?) on your OpenBSD system. *** The infrastructure behind Varnish (https://www.varnish-cache.org/news/20160425_website.html) Dogfooding. It's a term you hear often in the software community, which essentially means to “Run your own stuff”. Today we have an article by PKH over at varnish-cache, talking about what that means to them. Specifically, they recently went through a website upgrade, which will enable them to run more of their own stuff. He has a great quote on what OS they use:“So, dogfood: Obviously FreeBSD. Apart from the obvious reason that I wrote a lot of FreeBSD and can get world-class support by bugging my buddies about it, there are two equally serious reasons for the Varnish Project to run on FreeBSD: Dogfood and jails.Varnish Cache is not “software for Linux”, it is software for any competent UNIX-like operating system, and FreeBSD is our primary “keep us honest about this” platform.“ He then goes through the process of explaining how they would setup a new Varnish-cache website, or upgrade it. All together a great read, and if you are one of the admin-types, you really should pay attention to how they build from the ground up. Some valuable knowledge here which every admin should try to replicate. I can not reiterate the value of having your config files in a private source control repo strongly enough The biggest take-away is: “And by doing it this way, I know it will work next time also.” *** Interview - Matt Macy - mmacy@nextbsd.org (mailto:mmacy@nextbsd.org)Graphics Stack Update (https://lists.freebsd.org/pipermail/freebsd-x11/2016-May/017560.html) News Roundup Followup on packaging base with pkg(8) (https://lists.freebsd.org/pipermail/freebsd-pkgbase/2016-May/000238.html) In spite of the heroic last minute effort by a team of contributors, pkg'd base will not be ready in time for FreeBSD 11.0 There are just too many issues that were discovered during testing The plan is to continue using freebsd-update in the meantime, and introduce a pkg based upgrade mechanism in FreeBSD 11.1 With the new support model for the FreeBSD 11 branch, 11.1 may come sooner than with previous major releases *** FreeBSD Core Election (https://www.freebsd.org/internal/bylaws.html) It is time once again for the FreeBSD Core Election Application period begins: Wednesday, 18 May 2016 at 18:00:00 UTC Application period ends: Wednesday, 25 May 2016 at 18:00:00 UTC Voting begins: Wednesday, 25 May 2016 at 18:00:00 UTC Voting ends: Wednesday, 22 June 2016 at 18:00:00 UTC Results announced Wednesday, 29 June 2016 New core team takes office: Wednesday, 6 July 2016 As of the time I was writing these notes, 3 hours before the application deadline, the candidates are: Allan Jude: Filling in the potholes Marcelo Araujo: We are not vampires, but we need new blood. Baptiste Daroussin (incumbent): Keep on improving Benedict Reuschling: Learn and Teach Benno Rice: Revitalising The Community Devin Teske: Here to help Ed Maste (incumbent): FreeBSD is people George V. Neville-Neil (incumbent): There is much to do… Hiroki Sato (incumbent): Keep up with our good community and technical strength John Baldwin: Ready to work Juli Mallett: Caring for community. Kris Moore: User-Focused Mathieu Arnold: Someone ask for fresh blood ? Ollivier Robert: Caring for the project and you, its developers The deadline for applications is around the time we finish recording the live show We welcome any of the candidates to schedule an interview in the next few weeks. We will make an attempt to hunt many of them down at BSDCan as well. *** Wayland/Weston with XWayland works on DragonFly (http://lists.dragonflybsd.org/pipermail/users/2016-May/249620.html) We haven't talked a lot about Wayland on BSD recently (or much at all), but today we have a post from Peter to the dragonfly mailing list, detailing his experience with it. Specifically he talks about getting XWayland working, which provides the compat bits for native X applications to run on WayLand displays. So far on the working list of apps: “gtk3: gedit nautilus evince xfce4: - xfce4-terminal - atril firefox spyder scilab” A pretty impressive list, although he said “chrome” failed with a seg-fault This is something I'm personally interested in. Now with the newer DRM bits landing in FreeBSD, perhaps it's time for some further looking into Wayland. Broadcom WiFi driver update (http://adrianchadd.blogspot.ca/2016/05/updating-broadcom-softmac-driver-bwn-or.html) In this blog post Adrian Chadd talks about his recent work on the bwn(4) driver for Broadcom WiFi chips This work has added support for a number of older 802.11g chips, including the one from 2009-era Macbooks Work is ongoing, and the hope is to add 802.11n and 5ghz support as well Adrian is mentoring a number of developers working on embedded or wifi related things, to try to increase the projects bandwidth in those areas If you are interested in driver development, or wifi internals, the blog post has lots of interesting details and covers the story of Adrian's recent adventures in bringing the drivers up *** Beastie Bits The Design of the NetBSD I/O Subsystems (2002) (http://arxiv.org/abs/1605.05810) ZFS, BTRFS, XFS, EXT4 and LVM with KVM – a storage performance comparison (http://www.ilsistemista.net/index.php/virtualization/47-zfs-btrfs-xfs-ext4-and-lvm-with-kvm-a-storage-performance-comparison.html?print=true) Swift added to FreeBSD Ports (http://www.freshports.org/lang/swift/) misc@openbsd: 'NSA addition to ifconfig' (http://marc.info/?l=openbsd-misc&m=146391388912602&w=2) Papers We Love: Memory by the Slab: The Tale of Bonwick's Slab Allocator (http://paperswelove.org/2015/video/ryan-zezeski-memory-by-the-slab/) Feedback/Questions Lars - Poudriere (http://pastebin.com/HRRyfxev) Warren - .NET (http://pastebin.com/fESV1egk) Eddy - Sys Init (http://pastebin.com/kQecpA1X) Tim - ZFS Resources (http://pastebin.com/5096cGXr) Morgan - Ports and Kernel (http://pastebin.com/rYr1CDcV) ***
This week on the show, Allan and I have gotten a bit more sleep since AsiaBSDCon, which is excellent since there is a LOT of news to cover. That plus our interview with Ports SecTeam member Mark Felder. So keep it This episode was brought to you by Headlines FreeNAS 9.10 Released (http://lists.freenas.org/pipermail/freenas-announce/2016-March/000028.html) OS: The base OS version for FreeNAS 9.10 is now FreeBSD 10.3-RC3, bringing in a huge number of OS-related bug fixes, performance improvements and new features. +Directory Services: You can now connect to large AD domains with cache disabled. +Reporting: Add the ability to send collectd data to a remote graphite server. +Hardware Support: Added Support for Intel I219-V & I219-LM Gigabit Ethernet Chipset Added Support for Intel Skylake architecture Improved support for USB devices (like network adapters) USB 3.0 devices now supported. +Filesharing: Samba (SMB filesharing) updated from version 4.1 to 4.3.4 Added GUI feature to allow nfsv3-like ownership when using nfsv4 Various bug fixes related to FreeBSD 10. +Ports: FreeBSD ports updated to follow the FreeBSD 2016Q1 branch. +Jails: FreeBSD Jails now default to a FreeBSD 10.3-RC2 based template. Old jails, or systems on which jails have been installed, will still default to the previous FreeBSD 9.3 based template. Only those machinesusing jails for the first time (or deleting and recreating their jails dataset) will use the new template. +bhyve: ++In the upcoming 10 release, the CLI will offer full support for managing virtual machines and containers. Until then, the iohyve command is bundled as a stop-gap solution to provide basic VM management support - *** Ubuntu BSD's first Beta Release (https://sourceforge.net/projects/ubuntubsd/) Under the category of “Where did this come from?”, we have a first beta release of Ubuntu BSD. Specifically it is Ubuntu, respun to use the FreeBSD kernel and ZFS natively. From looking at the minimal information up on sourceforge, we gather that is has a nice text-based installer, which supports ZFS configuration and iSCSI volume creation setups. Aside from that, it includes the XFCE desktop out of box, but claims to be suitable for both desktops and servers alike right now. We will keep an eye on this, if anybody listening has already tested it out, maybe drop us a line on your thoughts of how this mash-up works out. *** FreeBSD - a lesson in poor defaults (http://vez.mrsk.me/freebsd-defaults.txt) Former BSD producer, and now OpenBSD developer, TJ, writes a post detailing the defaults he changes in a fresh FreeBSD installation Maybe some of these should be the defaults While others are definitely a personal preference, or are not as security related as they seem A few of these, while valid criticisms, but some are done for a reason Specifically, the OpenSSH changes. So, you're a user, you install FreeBSD 10.0, and it comes with OpenSSH version X, which has some specific defaults As guaranteed by the FreeBSD Project, you will have a nice smooth upgrade path to any version in the 10.x branch Just because OpenSSH has released version Y, doesn't mean that the upgrade can suddenly remove support for DSA keys, or re-adding support for AES-CBC (which is not really weak, and which can be hardware accelerated, unlikely most of the replacements) “FreeBSD is the team trying to increase the risk.” Is incorrect, they are trying to reduce the impact on the end user Specifically, a user upgrading from 10.x to 10.3, should not end up locked out of their SSH server, or otherwise confronted by unexpected errors or slowdowns because of upstream changes I will note again, (and again), that the NONE cipher can NOT allow a user to “shoot themselves in the foot”, encryption is still used during the login phase, it is just disabled for the file transfer phase. The NONE cipher will refuse to work for an interactive session. While the post states that the NONE cipher doesn't improve performance that much, it infact does In my own testing, chacha20-poly1305 1.3 gbps, aes128-gcm (fastest) 5.0 gbps, NONE cipher 6.3 gbps That means that the NONE cipher is an hour faster to transfer 10 TB over the LAN. The article suggests just removing sendmail with no replacement. Not sure how they expect users to deliver mail, or the daily/weekly reports Ports can be compiled as a regular user. Only the install phase requires root for ntpd, it is not clear that there is an acceptable replacement yet, but I will not that it is off by default In the sysctl section, I am not sure I see how enabling tcp blackhole actually increases security at all I am not sure that linking to every security advisory in openssl since 2001 is actually useful Encrypted swap is an option in bsdinstall now, but I am not sure it is really that important FreeBSD now uses the Fortuna PRNG, upgraded to replace the older Yarrow, not vanilla RC4. “The resistance from the security team to phase out legacy options makes mewonder if they should be called a compatibility team instead.” I do not think this is the choice of the security team, it is the ABI guarantee that the project makes. The stable/10 branch will always have the same ABI, and a program or driver compiled against it will work with any version on that branch The security team doesn't really have a choice in the matter. Switching the version of OpenSSL used in FreeBSD 9.x would likely break a large number of applications the user has installed Something may need to be done differently, since it doesn't look like any version of OpenSSL, (or OpenSSH), will be supported for 5 years ever again *** ZFS Raidz Performance, Capacity and Integrity (https://calomel.org/zfs_raid_speed_capacity.html) An updated version of an article comparing the performance of various ZFS vdev configurations The settings users in the test may not reflect your workload If you are benchmarking ZFS, consider using multiple files across different datasets, and not making all of the writes synchronous Also, it is advisable to run more than 3 runs of each test Comparing the numbers from the 12 and 24 disk tests, it is surprising to see that the 12 mirror sets did not outperform the other configurations. In the 12 drive tests, the 6 mirror sets had about the same read performance as the other configurations, it is not clear why the performance with more disks is worse, or why it is no longer in line with the other configurations More investigation of this would be required There are obviously so other bottlenecks, as 5x SSDs in RAID-Z1 performed the same as 17x SSDs in RAID-Z1 Interesting results none the less *** iXSystems FreeNAS Mini Review (http://www.nasanda.com/2016/03/ixsystems-freenas-mini-nas-device-reviewed/) Interview - Mark Felder - feld@freebsd.org (mailto:feld@freebsd.org) / @feldpos (https://twitter.com/feldpos) Ports, Ports and more Ports DigitalOcean Digital Ocean's guide to setting up an OpenVPN server (https://www.digitalocean.com/community/tutorials/how-to-configure-and-connect-to-a-private-openvpn-server-on-freebsd-10-1) News Roundup AsiaBSDCon OpenBSD Papers (http://undeadly.org/cgi?action=article&sid=20160316153158&mode=flat&count=0) + Undeadly.org has compiled a handy list of the various OpenBSD talks / papers that were offered a few weeks ago at AsiaBSDCon 2016. Antoine Jacoutot (ajacoutot@) - OpenBSD rc.d(8) (slides | paper) Henning Brauer (henning@) - Running an ISP on OpenBSD (slides) Mike Belopuhov (mikeb@) - Implementation of Xen PVHVM drivers in OpenBSD (slides | paper) Mike Belopuhov (mikeb@) - OpenBSD project status update (slides) Mike Larkin (mlarkin@) - OpenBSD vmm Update (slides) Reyk Floeter (reyk@) - OpenBSD vmd Update (slides) Each talk provides slides, and some the papers as well. Also included is the update to ‘vmm' discussed at bhyveCon, which will be of interest to virtualization enthusiasts. *** Bitcoin Devs could learn a lot from BSD (http://bitcoinist.net/bitcoin-devs-could-learn-a-lot-from-bsd/) An interesting article this week, comparing two projects that at first glance may not be entirely related, namely BitCoin and BSD. The article first details some of the woes currently plaguing the BitCoin development community, such as toxic community feedback to changes and stakeholders with vested financial interests being unable to work towards a common development purpose. This leads into the crux or the article, about what BitCoin devs could learn from BSD: First and foremost, the way code is developed needs change to stop the current negative trend in Bitcoin. The FreeBSD project has a rigid internal hierarchy of people with write access to their codebase, which the various Bitcoin implementations also have, but BSD does this in a way that is very open to fresh eyes on their code, allowing parallel problem solving without the petty infighting we see in Bitcoin. Anyone can propose a commit publicly to the code, make it publicly available, and democratically decide which change ends up in the codebase. FreeBSD has a tiny number of core developers compared to the size of their codebase, but at any point, they have a huge community advancing their project without hard forks popping up at every small disagreement. Brian Armstrong commented recently on this flaw with Bitcoin development, particularly with the Core Devs: “Being high IQ is not enough for a team to succeed. You need to make reasonable tradeoffs, collaborate, be welcoming, communicate, and be easy to work with. Any team that doesn't have this will be unable to attract top talent and will struggle long term. In my opinion, perhaps the biggest risk in Bitcoin right now is, ironically, one of the things which has helped it the most in the past: the Bitcoin Core developers.” A good summary of the culture that could be adopted is summed up as follows: The other thing Bitcoin devs could learn from is the BSD community's adoption of the Unix Design philosophy. Primarily “Worse is Better,” The rule of Diversity, and Do One Thing and Do It Well. “Worse is Better” emphasizes using extant functional solutions rather than making more complex ones, even if they would be more robust. The Rule of Diversity stresses flexibility of the program being developed, allowing for modification and different implementations without breaking. Do one Thing and Do it well is a mantra of the BSD and Unix Communities that stresses modularity and progress over “perfect” solutions. Each of these elements help to make BSD a wildly successful open source project with a healthy development community and lots of inter-cooperation between the different BSD systems. While this is the opposite of what we see with Bitcoin at present, the situation is salvageable provided changes like this are made, especially by Core Developers. All in all, a well written and interesting take on the FreeBSD/BSD project. We hope the BitCoin devs can take something useful from it down the road. *** FreeBSD cross-compiling with gcc and poudriere (http://ben.eficium.net/2016/03/freebsd-cross-compiling-with-gcc.html) Cross-Compiling, always a challenge, has gotten easier using poudriere and qemu in recent years. However this blog post details some of the particular issues still being face when trying to compile some certain ports for ARM (I.E. rPi) that don't play nicely with FreeBSD's default CLANG compiler. The writer (Ben Slack) takes us through some of the work-arounds he uses to build some troublesome ports, namely lsof and libatomic_ops. Note this is not just an issue with cross compile, the above mentioned ports also don't build with clang on the Pi directly. After doing the initial poudriere/qemu cross-compile setup, he then shows us the minor tweaks to adjust which compiler builds specific ports, and how he triggers the builds using poudriere. With the actual Makefile adjustment being so minor, one wonders if this shouldn't just be committed upstream, with some if (ARM) - USE_GCC=yes type conditional. *** Nvidia releases new Beta graphics driver for FreeBSD (https://devtalk.nvidia.com/default/topic/925607/unix-graphics-announcements-and-news/linux-solaris-and-freebsd-driver-364-12-beta-/) Added support for the following GPUs: GeForce 920MX & GeForce 930MX Added support for the Vulkan API version 1.0. Fixed a bug that could cause incorrect frame rate reporting on Quadro Sync configurations with multiple GPUs. Added a new RandR property, CscMatrix, which specifies a 3x4 color-space conversion matrix. Improved handling of the X gamma ramp on GF119 and newer GPUs. On these GPUs, the RandR gamma ramp is always 1024 entries and now applies to the cursor and VDPAU or workstation overlays in addition to the X root window. Fixes for bugs and added several other EGL extensions *** Beastie Bits New TN Bug started (http://knoxbug.org/) DragonFlyBSD Network/TCP Performance's gets a bump (http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/4a43469a10cef8c17553c342aab9d73611ea7bc8?utm_source=anzwix) FreeBSD Foundation introduces a new website and logo (https://www.freebsdfoundation.org/blog/introducing-a-new-look-for-the-foundation/) Our producer made these based on the new logo: http://q5sys.sh/2016/03/a-new-freebsd-foundation-logo-means-its-time-for-some-new-wallpapers/ http://q5sys.sh/2016/03/pc-bsd-and-lumina-desktop-wallpapers/ https://github.com/pcbsd/lumina/commit/60314f46247b7ad6e877af503b3814b0be170da8 IPv6 errata for 5.7/5.8, pledge errata for 5.9 (http://undeadly.org/cgi?action=article&sid=20160316190937&mode=flat) Sponsoring “PAM Mastery” (http://blather.michaelwlucas.com/archives/2577) A visualization of FreeBSD commits on GitHub for 2015 (https://rocketgraph.com/s/v89jBkKN4e-) The VAX platform is no more (http://undeadly.org/cgi?action=article&sid=20160309192510) Feedback/Questions Hunter - Utils for Blind (http://slexy.org/view/s20KPYDOsq) Chris - ZFS Quotas (http://slexy.org/view/s2EHdI3z3L) Anonymous - Tun, Tap and Me! (http://slexy.org/view/s21Nx1VSiU) Andrew - Navigating the BSDs (http://slexy.org/view/s2ZKK2DZTL) Brent - Wifi on BSD (http://slexy.org/view/s20duO29mN) ***
This week on BSDNow, Allan and I are back from AsiaBSDCon and we have an interview with Brad Davis about the new “Packaging Base” call-for-testing. We'll be sharing our thoughts and stories on how the week This episode was brought to you by Headlines AsiaBSDCon 2016 - Wrap-up FreeBSD gets Haswell graphics support in time for 11.0-RELEASE (https://svnweb.freebsd.org/changeset/base/296548) The moment that many have been waiting for has finally arrived, support for Haswell graphics has been committed to FreeBSD -CURRENT The brings the DRM/i915 code up to date with Linux kernel 3.8.13 Work has already started on updating to Linux kernel 3.9 It is hoped that subsequent updates will be much easier, and much faster It does not appear to require setting the i915.preliminaryhwsupport loader tunable *** OpenBSD vmm/vmd Update (http://bhyvecon.org/bhyvecon2016-Mike.pdf) For the third year running, bhyvecon was held last week, during the lead up to AsiaBSDCon Bhyvecon has expanded, and now covers all virtualization on BSDs There were presentations on bhyve, Xen Dom0 on FreeBSD, Xen DomU for OpenBSD, and OpenBSD's vmm OpenBSD vmm started at the Brisbane 2015 hackathon in Australia Work continued through the summer and fall thanks to funding by the OpenBSD Foundation The presentation answered some outstanding questions, such as, why not just port bhyve? Initial focus is OpenBSD on OpenBSD Loader currently supports FreeBSD and NetBSD as well After the initial commits, other developers joined in to help with the work Reyk reworked the vmd and vmctl commands, to provide a better user interface Future plans: Nested VMX i386 support AMD SVM support Filesystem passthru Live migration (with ZFS like command syntax) Other developers are working on related projects: qemu interface: Allow qemu to be accelerated by the vmm backend, while providing emulated hardware, for legacy systems KVM interface: Make vmm look like KVM, so existing tools like openstack “just work” *** Interview - Brad Davis - brd@freebsd.org (mailto:brd@freebsd.org) / @so14k (https://twitter.com/so14k) Packaging Base News Roundup Packaging the base system with pkg(8) (https://lists.freebsd.org/pipermail/freebsd-pkgbase/2016-March/000032.html) The official call for testing for FreeBSD's pkg(8)'d base is out Users are requested to checkout the release-pkg branch, and build it as normal (buildworld, buildkernel) Instead of installworld, run: make packages This will produce a pkg repo in the /usr/obj directory The post to the mailing list includes an example pkg repo config file to point to those packages Run: pkg update -r FreeBSD-base This will read the metadata from the new repository Then run: pkg install -g 'FreeBSD-*' This will find all packages that start with ‘FreeBSD-' and install them In the future, there will be meta packages, so you can just install FreeBSD-base and it will pull in other packages are dependencies Currently, there are a large number of packages (over 700), because each shared library is packaged separately, and almost all optional features are in a separate package The number of packages is also increased because there are separate -debug, -profiling, etc versions of each package New features are being added to pkg(8) to mark important system components, like libc, as ‘vital', so they cannot be deleted accidently However, in the case of using pkg(8)'d base to create a jail, the administrator should be able to delete the entire base system Classic conundrum: “UNIX does not stop you doing something stupid, as that would also stop you doing something clever” Work is still ongoing At AsiaBSDCon, after the interview was recorded, bapt@ and brd@ had a whiteboarding session and have come up with how they expect to handle the kernel package, to ensure there is a /boot/kernel.old for you to fall back to incase the newly installer kernel does not work correctly. *** FreeBSD 10.3-RC2 Now Available (https://lists.freebsd.org/pipermail/freebsd-stable/2016-March/084384.html) The second release candidate for FreeBSD 10.3 is now available for testing Notable changes include: Import an upstream fix for ‘zfs send -i' to avoid data corruption in specific instances Boot loaders and kernel have been taught to handle ELF sections of type SHTAMD64UNWIND. This does not really apply to FreeBSD 10.3, but is required for 11.0, so will make upgrades easier Various mkdb commands (/etc/services, /etc/login.conf, etc) commands now use fsync() instead of opening the files as O_SYNC, greatly increasing the speed of the database generation From the earlier BETA3, the VFS improvements that were causing ZFS hangs, and the new ‘tryforward' routing code, have been reverted Work is ongoing to fix these issues for FreeBSD 11.0 There are two open issues: A fix for OpenSSH CVE-2016-3115 has not be included yet the re-addition of AES-CBC ciphers to the default server proposal list. AES-CBC was removed as part of the update to OpenSSH version 7.1p2, but the plan is to re-add it, specifically for lightweight clients who rely on hardware crypto offload to have acceptable SSH performance Please go out and test *** OPNsense 16.1.6 released (https://forum.opnsense.org/index.php?topic=2378.0) A new point-release of OPNsense has dropped, and apart from the usual security updates, some new features have been included firmware: bootstrap utility can now directly install e.g. the development version dhcp: all GUI pages have been reworked for a polished look and feel proxy: added category-based remote file support if compressed file contains multiple files proxy: added ICAP support (contributed by Fabian Franz) proxy: hook up the transparent FTP proxy proxy: add intercept on IPv6 for FTP and HTTP proxy options logging: syslog facilities, like services, are now fully pluggable vpn: stripped an invalid PPTP server configuration from the standard configuration vpn: converted to pluggable syslog, menu and ACL dyndns: all GUI pages have been reworked for a polished look and feel dyndns: widget now shows IPv6 entries too dns forwarder: all GUI pages have been reworked for a polished look and feel dns resolver: all GUI pages have been reworked for a polished look and feel dns resolver: rewrote the dhcp lease registration hooks dns resolver: allow parallel operation on non-standard port when dns forwarder is running as well firewall: hide outbound nat rule input for "interface address" option and toggle bitmask correctly interfaces: fix problem when VLAN tags weren't generated properly interfaces: improve interface capability reconfigure ipsec: fix service restart behaviour from GUI captive portal: add missing chain in certificate generation configd: improve recovery and reload behaviour load balancer: reordered menu entries for clarity ntp: reordered menu entries for clarity traffic shaper: fix mismatch for direction + dual interfaces setup languages: updated German and French Call for testing - ASLR patch (https://lists.freebsd.org/pipermail/freebsd-arch/2016-March/017719.html) A patch that provides a first pass implementation of basic ASLR (Address Space Layout Randomization) for FreeBSD has been posted to the mailing list “Stack gap, W^X, shared page randomization, KASLR and other techniques are explicitly out of scope of this work.” “ASLR is enabled on per-ABI basis, and currently it is only enabled on native i386 and amd64 (including compat 32bit) ABIs. I expect to test and enable ASLR for armv6 and arm64 as well, later” “Thanks to Oliver Pinter and Shawn Webb of the HardenedBSD project for pursuing ASLR for FreeBSD. Although this work is not based on theirs, it was inspired by their efforts.” *** Feedback/Questions Daniel - OpenZFS (http://slexy.org/view/s20Z81SPq3) Florian - JBODS (http://slexy.org/view/s2be4zDkG6) Hunter - SSL on DO (http://slexy.org/view/s2o0MijCFy) Ben - Backups (http://slexy.org/view/s2fXlOwdU7) Damian - Bug'in Me! (http://slexy.org/view/s2weBPb8sx) ***
This week, Allan and I are away at AsiaBSDCon! (If you aren't there, you are missing out). We will be back with a live episode next week. However, we've been asked for Allan to tell us about ScaleEngine's This episode was brought to you by Interview - Allan Jude - allanjude@freebsd.org (mailto:allanjude@freebsd.org) / @allanjude (https://twitter.com/allanjude) Spotlight on ScaleEngine *** Beastie Bits NetBSD on an RPi Zero (https://github.com/ebijun/NetBSD/blob/master/dmesg/earmv6hf/RPI0) DragonFly tips for printing with CUPS (http://lists.dragonflybsd.org/pipermail/users/2016-February/228608.html) Fighting fraudulent networks using secure connections (SSL) blacklisting with OPNsense. Blocks known-bad certificates as listed at abuse.ch (https://opnsense.org/fighting-fraudulent-networks-using-secure-connections-ssl-with-opnsense/) Fix for running NetBSD/amd64 7.0 on kvm based virtual machines (https://imil.net/blog/2016/01/29/netbsdamd64-7-0-kvm/) Michael W. Lucas's new book, FreeBSD Mastery: Specialty Filesystems is now escaping (http://blather.michaelwlucas.com/archives/2537) The Penguicon Lucas Tech Track (http://blather.michaelwlucas.com/archives/2534) FreeBSD based nginx/ffmpeg camera recording and live streaming (http://www.unixmen.com/freebsd-nginx-ffmpeg-camera-recording-and-live-streaming/) CFT: New Jenkins Builder for FreeNAS / PC-BSD (https://github.com/iXsystems/ixbuild/) Status Update: PC-BSD's SysAdm Server (https://github.com/pcbsd/sysadm/) Status Update: PC-BSD's SysAdm Client UI (https://github.com/pcbsd/sysadm-ui-qt)
This week on BSDNow, Allan is back from the Storage Summit in Silicon Valley! We are going to get his thoughts on how the conference went, plus bring you the latest ZFS info discussed. That plus the usual BSD news is This episode was brought to you by Headlines OpenBSD website operators urged to fix mind-alteringly bad bug (http://www.theregister.co.uk/2016/02/21/openbsd_website_operator_patch_now_for_the_sake_of_your_sanity/?mt=1456206806399) We start off a bit light-hearted this week, with the important, breaking news that finally a long-standing OpenBSD bug has been addressed for the HTTP daemon. Specifically? It changes the default 404 page fonts away from Comic Sans, to a bit more crowd-pleasing alternative: “For some reason the httpd status pages (e.g. 404) use the Comic Sans typeface. This patch removes comic sans and sets the typeface to the default sans-serif typeface of the client. “This lowers the number of people contacting website maintainers with typeface complaints bordering on harassment”. Operators running HTTPD are highly encouraged to update their systems to the latest code, right now……... No seriously, we are waiting for you. Get it done now and then we'll continue with the show. Registration for AsiaBSDCon 2016 is now open + Talk Schedule (https://2016.asiabsdcon.org/registration/?lang=en) After a few delays, the registration for AsiaBSDCon has now opened! The conference starts in less than two weeks! now, so be sure to get signed up ASAP. In addition the schedule has been posted, and here's some of the highlights of this year's conference. In addition to FreeBSD and NetBSD dev summits on the first two days, we have some excellent tutorials being given this year by Kirk, Gnn, Dru and more! (https://2016.asiabsdcon.org/program.html.en) The regular paper talks also have lots of good ones this year, including this crazy encrypted boot loader one given by our very own Allan Jude! *** OPENBSD ON AWS : AN UNEXPECTED JOURNEY (http://blog.d2-si.fr/2016/02/15/openbsd-on-aws/?hn) We have a blog post from Antoine Jacoutot, talking about the process of getting OpenBSD up and running in AWS It starts with his process of creating an AMI from scratch, which ended up not being that bad: create and loopback-mount a raw image containing a UFS filesystem extract the OpenBSD base sets (which are just regular tarballs) and kernel enable console output (so that one could “aws ec2 get-console-output”) install the boot loader on the image then use the ec2 tools to import the RAW image to S3, convert it into a volume (ec2-import-volume) which we can snapshot (ec2-create-snapshot) and create an AMI from (ec2-register) The blog post also has a link to a script which automates this process, so don't be daunted if you didn't quite follow all of that. Thanks to the recently landed DomU support, the final pieces of the puzzle fell into place, allowing OpenBSD to function as a proper guest (with networking!) Next it details the process of injecting a public SSH key into the instances for instant remote access. An ec2-init.sh script was created (also on github) which does the following: setting the hostname installing the provided SSH public key to /root/.ssh/authorized_keys executing user-data (if it starts with a shebang) displaying the host SSH fingerprints on the console (to match cloud-init) With that done, OpenBSD is pretty much AWS ready! He then gives a brief walkthrough of setting up nginx for new users, but if you've already done this before then the instance is ready for you to hacking on. Start thinking of ideas for things with FreeBSD for Google's 2016 Summer of Code (https://wiki.freebsd.org/SummerOfCodeIdeas) Students and Developers, listen up! It's time to start thinking about GSoC again, and FreeBSD is looking to update its project ideas page. There's some good ones on the list, plus ones that should be pruned (such as GELI boot), but now is the time to start adding new ones before we get too deep into the process. This goes for the other BSD's as well, start thinking about your proposals, or if you are developer, which projects would be a good fit for mentoring. (Improving the Linux Compat layer is one I think should be done!) Guide to getting started with kernel hacking (https://wiki.freebsd.org/Graphics/Getting%20started%20with%20kernel%20projects) One of the things that's been asked frequently is how to contribute towards the efforts to bring updated DRM / X drivers to the FreeBSD kernel. Jean-Sébastien Pédron has started a great guide on the Wiki which details how to get started with the porting effort, and that developers need not be afraid of helping. *** Storage Summit Roundup Earlier this week a number of developers from FreeBSD, as well as various vendors that use FreeBSD, or provide products used with FreeBSD met for a Storage Summit (https://wiki.freebsd.org/201602StorageSummit), to discuss the future of these technologies The summit was co-located with the USENIX FAST (Filesystems And Storage Technologies) conference The summit was sponsored by the FreeBSD Foundation and FlightAware After a short introduction, the event opened with a Networking Synergy panel The focus of this panel was to see if there were techniques and lessons learned in improving the networking stack over the last 10 years that could be applied to improving the storage stack A lot of time was spent discussing issues like multi-queue support, CPU scheduling, and ways to modernize the stack CAM Scheduling & Locking Revamp (https://wiki.freebsd.org/201602StorageSummit/CAM) No notes posted User Space Storage Stack (https://wiki.freebsd.org/201602StorageSummit/UserSpace) One of the user space storage stacks discussed was Diskmap Like netmap, but for disks (diskmap) Kernel bypass for accessing disks Ilias Marinos, who is working on diskmap at Cambridge University, described diskmap to the group A design discussion then followed in which the memory management was covered as that's an issue for any sort of "IO" map system Action Items: Discuss with Luigi the idea of code merges Need a reset path API Kernel buffer mapping for reliability Support for other interfaces (SATA/SCSI) GEOM layer adaptation Adapting to New Storage Technologies (https://wiki.freebsd.org/201602StorageSummit/NewStorageTechnologies) This working group was led by Adrian Palmer, from Seagate SMR Persistent Memory Session 1: Device Identification and the structural requirements Agenda: We'll look over the Identification nuances and what needs to change to support the structure. Support for IO order guarantees, forward-write only requirements, new commands and topology. Dig into CAM and GEOM layers. Solutions should be fast and have as few code paths as possible Results: Small audience. We talked about zoned characteristics, and how it can be used in various workloads, projected to be implemented in years Session 2: Information dissemination and consumption Agenda: Where and how will information from the report_zones command be gathered, stored, combined and used. This will include userspace storage and multi-volume management. Will CAM store this data, or will GEOM? How frequently will this need to be queried/updated/verified from the drive? Results: Merged with ZFS working group to discuss SMR. Came up with idea that could be implemented as circular buffer zone type. Began to discuss solutions among developers ZFS (https://wiki.freebsd.org/201602StorageSummit/ZFS) During the first session we discussed how to improve dedup support + A dedup throttle or cap was discussed. When the size of the DDT grows beyond this size, new entries would not be deduped. An alternative to this was also discussed, where when the DDT reached the cap size, it would remove a random entry with only a single reference from the DDT to make room for the new entry. When a block is going to be freed, if it is not found in the DDT, it is assumed to have only 1 reference, and removed. There was also discussion of replacing the DDT with an in-memory hash table and a “log” of increment/decrement operations, that is periodically compacted. The hash table is recreated from the log at pool import time. This would reduce the in-memory footprint of the DDT, as well as speed up all write operations as adding an entry to the dedup log will be less expensive than updating the DDT. There was also discussion of using dedicated device(s) for the DDT, either using the DDT on SSD work by Nexenta, or the Metadata Classes work by Intel The first session also discussed Secure Delete and related things The desire for an implementation of TRIM that uses the “secure erase” functionality provided by some disks was expressed Overwriting sectors with patterns of garbage may be insufficient because SSDs may internally remap where a specific LBA physically resides The possibility of using something like the “eager zero” feature to periodically write zeros over all free blocks in the pool to erase any lingering data fragments Problems with the FreeBSD TRIM implementation were discussed, as well as looking at ways to implement the new ZFS TRIM implementation on FreeBSD ABD (ARC Buf Data) was discussed, a new design that lessens the requirement for contiguous memory. Only a small area of contiguous blocks is reserved at boot, and compressed ARC blocks are constructed of scatter-gather lists of individual pages The second session combined with the SMR group and talked about SMR support in ZFS Later in the second session ZFS Encryption was also discussed, mostly with a focus on what the use cases are The third session combined all of the groups for an overview of upcoming ZFS features including device removal and channel programs There was also a request for code review, for mostly finished projects like Persistent L2ARC, Writeback cache, and Large dnode support Hallway Track ZFS / VFS Interaction Adrian Palmer has been a FreeBSD hobbyist since FreeBSD 7, and I think I managed to convince him to start contributing *** News Roundup One Week with NetBSD 7.0: Back to Unix basics (http://jamesdeagle.blogspot.com/2016/02/one-week-with-netbsd-70-back-to-unix.html) The author of this blog series is sending a week using NetBSD 7.0, following a previous series on Solaris 10 “This is actually familiar territory, as I've been using BSD variants almost exclusively since 2006. My recent SunOS explorations were triggered last summer by OpenBSD having choked on my current laptop's NVIDIA card, and from what I could see at the time, FreeBSD had the same problem, although I now know NVIDIA drivers exist for that system. The thing that keeps me from going all-in with FreeBSD 10.x, however, is the fact that Firefox crashes and leaves "core dump" messages in its wake, and I'm just not a Chrome kinda guy.” “For those with a catholic taste in Unix, NetBSD is a keg party at the Vatican. If you're an absolute Unix beginner, or have been living on Ubuntu-based Linux distros for too long, then you may feel stranded at first by NetBSD's sparseness. You'll find yourself staring into the abyss and seeing only a blinking cursor staring back. If you have the presence of mind to type startx, you'll be greeted by twm, a window manager offering little more than an xterm window with the same blinking cursor until you learn how to configure the .twmrc file to include whatever applications you want or need in the right-click menu.” “As for NetBSD itself, I can't think of any major productivity applications that can't be installed, and most multimedia stuff works fine.” Issues the author hopes to sort out in later posts: Audio playback (youtube videos in Firefox) Wireless Flash Digital Camera SD Card readability, video playback Audacity A “fancy” desktop like Gnome 2, KDE, or xfce In a follow-up post (http://jamesdeagle.blogspot.com/2016/02/one-week-with-netbsd-70-libreoffice.html), the author got LibreOffice installed and sorted out the audio issues they were having In a later follow-up (http://jamesdeagle.blogspot.com/2016/02/one-week-with-netbsd-70-mixed-review-of.html) XFCE is up and running as well *** ZFS is for Containers in Ubuntu 16.04 (http://blog.dustinkirkland.com/2016/02/zfs-is-fs-for-containers-in-ubuntu-1604.html) As you may have heard, Ubuntu 16.04 will include ZFS -- baked directly into Ubuntu -- supported by Canonical “ZFS one of the most beloved features of Solaris, universally coveted by every Linux sysadmin with a Solaris background. To our delight, we're happy to make to OpenZFS available on every Ubuntu system.” What does “supported by Canonical” mean? “You'll find zfs.ko automatically built and installed on your Ubuntu systems. No more DKMS-built modules” “The user space zfsutils-linux package will be included in Ubuntu Main, with security updates provided by Canonical” The article then provides a quick tutorial for setting up Linux Containers (LXC) backed by ZFS In the example, ZFS is backed by a file on the existing disk, not by a real disk, and with no redundancy However, the setup script seems to support using real block devices The Software Freedom Conservancy (https://sfconservancy.org/) is expected to issue a statement detailing their opinion on the legalities and licensing issues of bundling ZFS with Linux. *** Polling is a Hack: Server Sent Events (EventSource) with gevent, Flask, nginx, and FreeBSD (http://hypatia.software/2016/01/29/polling-is-a-hack-server-sent-events-eventsource-with-gevent-flask-nginx-and-freebsd/) A tutorial on setting up ‘Server-Sent Events', also know as EventSource in javascript, to notify website clients of new data, rather than having the javascript constantly poll for new data. The setup uses FreeBSD, nginx, gevent, Python, and the Flask framework The tutorial walks through setting a basic Python application using the Flask framework Then setting up the client side in Javascript Then for the server side setup, it covers installing and configuring nginx, and py-supervisor on FreeBSD The tutorial also includes links to additional resources and examples, including how to rate limit the Flash application *** Why FreeBSD? (http://www.aikchar.me/blog/why-freebsd.html) An excellent article written by Hamza Sheikh, discussing why FreeBSD is now his clear choice for learning UNIX. The article is pretty well written and lengthy, but has some great parts which we wanted to share with you: There were many rough edges in the Linux world and some of them exist even today. Choosing the right distribution (distro) for the task at hand is always the first and most difficult decision to make. While this is a strength of the Linux community it is also its weakness. This is exacerbated with the toxic infighting within the community in the last few years. A herd of voices believes it is their right to bring down a distro community because it is not like their distro of choice. Forking upstream projects has somehow become taboo. Hurling abuse in mailing lists is acceptable. Helping new users is limited to lambasting their distro of choice. Creating conspiracy theories over software decisions is the way to go. Copyleft zealots roam social media declaring non-copyleft free software heretic abominations. It all boils down to an ecosystem soured by the presence of maniacs who have the loudest voices and they seem to be everywhere you turn. Where is the engineering among all this noise? Btrfs - baking for a long time - is still nowhere near ZFS in stability or feature parity. systemd is an insatiable entity that feeds on every idea in sight and just devours indiscriminately. Wayland was promised years ago and its time has yet to arrive. Containers are represented by Docker that neither securely contains applications nor makes them easy to manage in production. Firewalling is dithering between firewalld, nftables, etc. SystemTap cannot match DTrace. In the same time span what do various BSDs offer? pf, CARP, ZFS, Hammer, OpenSSH, jails, pkgsrc, (software) ports, DTrace, hardware portability; just to name a few. Few would deny that BSDs have delivered great engineering with free software licenses to the entire world. To me they appear to be better flag bearers of free software with engineering to back it. He then goes through some of the various BSD's and the specifics on why FreeBSD was the logical choice for his situation. But at the end has a great summary on the community as a whole: Finally - and maybe repeating myself here - I have nothing but praise for the community. Be it BSD Now, mailing lists, Reddit, Twitter, LFNW, or SeaGL, people have encouraged me, answered my questions, and filed bugs for me. I have been welcomed and made a part of the community with open arms. These reasons are (good) enough for me to use FreeBSD and contribute to it. BeastieBits OPNsense 16.1.3 released (https://opnsense.org/opnsense-16-1-3-released/) Copies of "FreeBSD Mastery: Specialty Filesystems" seen in the wild (https://twitter.com/Savagedlight/status/700001944547491842) pfsense training available in Europe (http://www.netgate.com/training/) LiteBSD now has 50 ports in its ports tree (https://github.com/ibara/LiteBSD-Ports) Ports tree locked for OpenBSD 5.9 (http://marc.info/?l=openbsd-ports&m=145615281431064&w=2) “FreeBSD Filesystem Fun” at March semibug (http://blather.michaelwlucas.com/archives/2556) Event #46 — Embedded Platforms (BSD, OpenWRT, Plan 9 & Inferno) (http://oshug.org/event/46) Feedback/Questions Frank - ZFS RAM? (http://slexy.org/view/s21lcCKrSB) David - ARM Porting (http://slexy.org/view/s204lxjvlq) Johnny - Lumina Default? (http://slexy.org/view/s2xMiSNLYn) Adam - PC-BSD Install and Q's (http://slexy.org/view/s214gJbLwD) Jeremy - Video Card Q (http://slexy.org/view/s20UNyzEeh) ***
This week on BSDNow, we interview Nick Wolff about how FreeBSD is used across the State of Ohio and some of the specific technology used. That, plus the latest news is coming your way right now on BSDNow, the place to This episode was brought to you by Headlines Doc like an Egyptian: Managing project documentation with Sphinx (https://opensource.com/business/16/1/scale-14x-interview-dru-lavigne) In case you didn't make it out to SCALE a few weeks back, we have a great interview with Dru Lavigne over at OpenSource.com which goes over her talk on “Doc like an Egyptian”. In particular she discusses the challenges of running a wiki for documentation for PC-BSD and FreeNAS which prompted the shift to using Sphinx instead. “While the main purpose of a wiki is to invite user contributions and to provide a low barrier to entry, very few people come to write documentation (however, every spambot on the planet will quickly find your wiki, which creates its own set of maintenance issues). Wikis are designed for separate, one-ish page infobytes, such as how-tos. They really aren't designed to provide navigation in a Table of Contents or to provide a flow of Chapters, though you can hack your pages to provide navigational elements to match the document's flow. This gets more difficult as the document increases in size—our guides tend to be 300+ pages. It becomes a nightmare as you try to provide versioned copies of each of those pages so that the user is finding and reading the right page for their version of software. While wiki translation extensions are available, how to configure them is not well documented, their use is slow and clunky, and translated pages only increase the number of available pages, getting you back to the problems in the previous bullet. This is a big deal for projects that have a global audience. While output-generation wiki extensions are available (for example, to convert your wiki pages to HTML or PDF), how to configure them is not well documented, and they provide very little control for the layout of the generated format. This is a big deal for projects that need to make their documentation available in multiple formats.“ She then discusses some of the hurdles of migration from the Wiki to Sphinx, and follows up with some of the differences using Sphinx you should be aware of for any documentation project. “While Sphinx is easy to learn, it does have its quirks. For example, it does not support stacked tags. This means, for example, you can not bold italic a phrase using tags—to achieve that requires a CSS workaround. And, while Sphinx does have extensive documentation, a lot of it assumes you already know what you are doing. When you don't, it can be difficult to find an example that does what you are trying to achieve. Sphinx is well suited for projects with an existing repository—say, on github—a build infrastructure, and contributors who are comfortable with using text editors and committing to the repo (or creating, say, git pull requests).“ Initial FreeBSD RISC-V Architecture Port Committed. (http://freebsdfoundation.blogspot.com/2016/02/initial-freebsd-risc-v-architecture.html) Touching on a story we mentioned a few weeks back, we have a blog post from from Annie over at the FreeBSD foundation talking about the details behind the initial support for RISC-V. To start us off, you may be wondering what is RISC-V and what makes it special?RISC-V is an exciting new open-source Instruction-Set Architecture (ISA) developed at the University of California at Berkeley, which is seeing increasing interest in the embedded systems and hardware-software research communities. Currently the improvements allows booting FreeBSD in the Spike simulator, from the university of Berkeley, with enough reliability to do various things, such as SSH, shell, mail, etc. The next steps include getting multi-core support working, and getting it working in simulations of Cambridge's open-source LowRISC System-on-Chip functioning, and ready for early hardware. Both ports and packages are expected to land in the coming days, so if you love hacking on branch new architectures, this may be your time to jump in. *** FreeBSD Bhyve hypervisor supporting Windows UEFI guests (https://svnweb.freebsd.org/base?view=revision&revision=295124) If you have not been following bhyve lately, you're in for a treat when FreeBSD 10.3 ships in the coming weeks bhyve now supports UEFI and CSM booting, in addition to its existing FreeBSD userboot loader, and grub-bhyve port The EFI support allows Windows guests to be run on FreeBSD Due to the lack of graphics, this requires making a custom .iso to do an ‘Unattended Install' of Windows, but this is easily done just editing and including a .xml file The bootrom can now allocate memory Added some SATA command emulations (no-op) Increased the number of virtio-blk indirect descriptors Added a Firmware guest query interface Add -l option to specify userboot path FreeBSD Bhyve Hypervisor Running Windows Server 2012 R2 Standard (https://jameslodge.com/freebsd-bhyve-hypervisor-running-windows-server-2012-r2-standard/) In related news, TidalScale officially released their product today (http://www.prnewswire.com/news-releases/tidalscale-releases-its-system-scaling-hyperkernel-300216105.html) TidalScale is a commercial product based on bhyve that allows multiple physical machines to be combined into a single massive virtual machine, with the combined processor power, memory, disk I/O, and network capacity of all of the machines *** FreeBSD TACACS+ GNS3 and Cisco 3700 Router (http://www.unixmen.com/freebsd-tacacs-gns3-and-cisco-3700-router/) “TACACS+ – (Terminal Access Controller Access Control System plus) — is a session protocol developed by Cisco.” This tutorial covers configuring FreeBSD and the tac_plus4 port to act as an authentication, authorization, and accounting server for Cisco routers The configuration of FreeBSD, the software, and the router are covered It also includes how to set the FreeBSD server up as a VM on windows, and bridge it to the network I am sure there are some network administrators out there that would appreciate this *** Interview - Nick Wolff - darkfiberiru@gmail.com (mailto:darkfiberiru@gmail.com) / @darkfiberiru (https://twitter.com/darkfiberiru) News Roundup Papers We Love Presents : Bryan Cantrill on Jails & Solaris Zones (http://lists.nycbug.org/pipermail/talk/2016-February/016495.html) The folks over at NYCBug point us to “Papers We Love”, a New York based meetup group where past papers are presented. They have a talk scheduled for tomorrow (Feb 11th) with Bryan Cantrill discussing Jails and Solaris Zones The talk starts at 7PM at the Tumblr building, located between 5th and Park Ave South on 21st street “We're crazy excited to have Bryan Cantrill, CTO of Joyent, formerly of Sun Microsystems, presenting on Jails: Confining the omnipotent root (https://us-east.manta.joyent.com/bcantrill/public/ppwl-cantrill-jails.pdf). by Poul-Henning Kamp and Robert Watson and Solaris Zones: Operating System Support for Consolidating Commercial Workloads (https://us-east.manta.joyent.com/bcantrill/public/ppwl-cantrill-zones.pdf) by Dan Price and Andy Tucker!” The abstract posted gives us a sneak peak of what to expect, first covering jails as a method to “partition” the operating system environment, but maintaining the UNIX “root” model. Next it looks like he will compare and contrast with the Solaris Zones functionality, which creates virtualized application execution environments, within the single OS instance. Sounds like a fantastic talk, hopefully somebody remembers to record and post it for us to enjoy later! There will not be a live stream, but a video of the event should appear online after it has been edited *** FreeBSD Storage Summit (https://wiki.freebsd.org/201602StorageSummit) The FreeBSD Foundation will be hosting a Storage Summit, co-located at the USENIX FAST (Filesystems And Storage Technology) conference Developers and Vendors are invited to work on storage related issues This summit will be a hackathon focused event, rather than a discussion focused devsummit After setup and introductions, the summit will start with a “Networking Synergies Panel”, to discuss networking as it relates to storage After a short break, the attendees will break up into a number of working groups focused on solving actual problems The current working groups include: CAM Scheduling & Locking, led by Justin Gibbs: “Updating CAM queuing/scheduling and locking models to minimize cross-cpu contention and support multi-queue controllers” ZFS, led by Matt Ahrens: topics will include enabling the new cryptographic hashes supported by OpenZFS on FreeBSD, Interaction with the kernel memory subsystem, and other upcoming features. User Space Storage Stack, led by George Neville-Neil This event offers a unique opportunity for developers and vendors from the storage industry to meet at an event they will likely already be attending *** Tor Browser 5.5 for OpenBSD/amd64 -current is completed (http://lists.nycbug.org/pipermail/talk/2016-February/016514.html) “The Tor BSD Diversity Project (TDP) is proud to announce the release of Tor Browser (TB) version 5.5 for OpenBSD. Please note that this version of TB remains in development mode, and is not meant to ensure strong privacy, anonymity or security.” “TDP (https://torbsd.github.io) is an effort to extend the use of the BSD Unixes into the Tor ecosystem, from the desktop to the network. TDP is focused on diversifying the Tor network, with TB being the flagship project. Additional efforts are made to increase the number of *BSD relays on the Tor network among other sub-projects” Help test the new browser bundle, or help diversify the Tor network *** “FreeBSD Mastery: Advanced ZFS” Table of Contents (http://blather.michaelwlucas.com/archives/2548) We brought you the news about sponsoring the Advanced ZFS book that MWL is working on, now Michael has given us the tentative chapter layout of the (sure to be a classic) tome coming from him and Allan. 0: Introduction 1: Boot Environments 2: Delegation and Jails 3: Sharing 4: Replication 5: zvols 6: Advanced Hardware 7: Caches 8: Performance 9: Tuning 10: ZFS Potpourri In addition to the tease about the upcoming book, michael has asked the community for assistance in coming up with the cover art for it as well. In particular it should probably be in-line with his previous works, with a parody of some other classic art-work. If you have something, go tweet out to him at @mwlauthor Beastie Bits Online registration for AsiaBSDCon 2016 now open SOON (https://2016.asiabsdcon.org/index.html.en) BhyveCon 2016 (http://bhyvecon.org/) NYC*BUG shell-fu talk slides (http://www.nycbug.org/index.cgi?action=view&id=10640) Possible regression in DragonFly i915 graphics on older Core2Duos (http://lists.dragonflybsd.org/pipermail/users/2016-February/228597.html) Videos from FOSDEM 2016. BSD dev room was k4601 (http://video.fosdem.org/2016/) Feedback/Questions Andrew - SMART Tests (http://slexy.org/view/s2F39XEu9w) JT - Secure File Delete (http://slexy.org/view/s20kk6lzc9) Jordan - Migrate (http://slexy.org/view/s21zjZ0ci8) Lars - Pros and Cons of VM (http://slexy.org/view/s2Hqbt0Uq8) Alex - IPSEC (http://slexy.org/view/s2HnO1hxSO) ***
This week on BSDNow, we are going to be talking to Pawel about how his This episode was brought to you by iX Systems Mission Complete (https://www.ixsystems.com/missioncomplete/) Submit your story of how you accomplished a mission with FreeBSD, FreeNAS, or iXsystems hardware, and you could win monthly prizes, and have your story featured in the FreeBSD Journal! *** Headlines Note the recent passing of 2 members of the BSD community Juergen Lock / Nox (https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributors/contrib-develinmemoriam.html) Benjamin Perrault / creepingfur (https://twitter.com/michaeldexter/status/676290499389485057) Memories from Michael Dexter (http://pastebin.com/4BQ5uVsT) Additional Memories (http://www.filis.org/rip_ben.txt) Benjamin and Allan at Ben's local bar (http://www.allanjude.com/bsd/bp/IMG_20151101_161727-auto.jpg) Benjamin treated Allan and Michael Dexter to their first ever Bermese food (http://www.allanjude.com/bsd/bp/IMG_20151101_191344-auto.jpg) Benjamin enjoying the hallway track at EuroBSDCon 2015 (http://www.allanjude.com/bsd/bp/IMG_20151003_105457-auto.jpg) *** NGINX as Reverse Proxy for Apache on FreeBSD 10.2 (http://linoxide.com/linux-how-to/install-nginx-reverse-proxy-apache-freebsd-10-2/) A tutorial on setting up NGINX as a reverse proxy for Apache Sometimes your users or application require some feature of Apache, that cannot be easily replicated in NGINX, like .htaccess files or a custom apache module In addition, because the default worker model in Apache does not accept new work until it is finished sending the request, a user with a slow connection can tie down that worker for a long time With NGINX as a reverse proxy, it will receive the data from the Apache worker over localhost, freeing that worker to answer the next request, while NGINX takes care of sending the data to the user The tutorial walks through the setup, which is very easy on modern FreeBSD One could also add mod_rpaf2 to the Apache, to securely pass through the users' real IP address for use by Apache's logging and the PHP scripts *** FreeBSD and FreeNAS in Business by Randy Westlund (http://bsdmag.org/freebsd_freenas/) The story of how a Tent & Awning company switched from managing orders with paper, to a computerized system backed by a FreeNAS “At first, I looked at off-the-shelf solutions. I found a number of cloud services that were like Dropbox, but with some generic management stuff layered on top. Not only did these all feel like a poor solution, they were very expensive. If the provider were to go out of business, what would happen to my dad's company?” “Fortunately, sourcing the hardware and setting up the OS was the easiest part; I talked to iXsystems. I ordered a FreeNAS Mini and a nice workstation tower” “I have r2d2 (the tower, which hosts the database) replicating ZFS snapshots to c3po (the FreeNAS mini), and the data is backed up off-site regularly. This data is absolutely mission-critical, so I can't take any risks. I'm glad I have ZFS on my side.” “I replaced Dropbox with Samba on c3po, and the Windows machines in the office now store important data on the NAS, rather than their local drives.” “I also replaced their router with an APU board running pfSense and replaced their PPTP VPN with OpenVPN and certificate authorization.” “FreeBSD (in three different incarnations) helped me focus on improving the company's workflow without spending much time on the OS. And now there's an awning company that is, in a very real sense, powered by FreeBSD.” *** Tutorial, Windows running under bhyve (http://pr1ntf.xyz/windowsunderbhyve.html) With the recent passing of the world's foremost expert on running Windows under bhyve on FreeBSD, this tutorial will help you get up to speed “The secret sauce to getting Windows running under bhyve is the new UEFI support. This is pretty great news, because when you utilize UEFI in bhyve, you don't have to load the operating system in bhyveload or grub-bhyve first.” The author works on iohyve, and wanted to migrate away from VirtualBox, the only thing stopping that was support for Windows Guests iohyve now has support for managing Windows VMs The tutorial uses a script to extract the Windows Server 2008 ISO and set up AutoUnattend.xml to handle the installation of Windows, including setting the default administrator password, this is required because there is no graphical console yet The AutoUnattended setup also includes setting the IP address, laying out the partitions, and configuring the serial console A second script is then used to make a new ISO with the modifications The user is directed to fetch the UEFI firmware and some other bits Then iohyve is used to create the Windows VM The first boot uses the newly created ISO to install Windows Server 2008 Subsequent boots start Windows directly from the virtual disk Remote Desktop is enabled, so the user can manage the Windows Server graphically, using FreeRDP or a Windows client iohyve can then be used to take snapshots of the machine, and clone it *** BSD Router Project has released 1.58 (http://sourceforge.net/projects/bsdrp/files/BSD_Router_Project/1.58/) The BSD Router project has announced the release of version 1.58 with some notable new features Update to FreeBSD 10.2-RELEASE-p8 Disabled some Chelsio Nic features not used by a router Added new easy installation helper option, use with “system install ” Added the debugging symbols for userland Includes the iperf package, and flashrom package, which allows updating system BIOS on supported boxes IMPORTANT: Corrects an important UFS label bug introduced on 1.57. If you are running 1.57, you will need to fetch their fixlabel.sh script before upgrading to 1.58 *** OPNsense 15.7.22 Released (https://opnsense.org/opnsense-15-7-22-released/) An update to OPNsense has landed this week which includes the important updates to OpenSSL 1.0.2e and LibreSSL 2.2.5 A long-standing annoying bug with filter reload timeouts has finally been identified and sorted out as well, allowing the functionality to run quickly and “glitch free” again. Some newer ports for curl (7.46), squid (3.5.12) and lighttpd (1.4.38) have also been thrown in for good measure Some other minor UI fixes have also been included as well With the holidays coming up, if you are still running a consumer router, this may be a good time to convert over to a OPNsense or PFsense box and get yourself ready for the new year. *** iXsystems iXSystems releases vCenter Web Client Plug-in for TrueNAS (https://www.ixsystems.com/whats-new/2015/12/vcenter-web-client-plug-in-for-truenas-now-available/) Interview - Pawel Jakub Dawidek - pjd@FreeBSD.org (mailto:pjd@FreeBSD.org) News Roundup Developer claims the PS4 has been jail-broken (http://www.networkworld.com/article/3014714/security/developer-claims-ps4-officially-jailbroken.html) While not exactly a well-kept secret, the PS4's proprietary “OrbOS” is FreeBSD based. Using this knowledge and a Kernel exploit, developer CTurt (https://twitter.com/CTurtE/) claims he was able jailbreak a WebKit process and gain access to the system. He has posted a small tease to GitHub, detailing some of the information gleaned from the exploit, such as PID list and root FS dump As such with these kinds of jailbreaks, he already requested that users stop sending him requests about game piracy, but the ability to hack on / run homebrew apps on the PS4 seems intriguing *** Sepherosa Ziehau is looking for testers if you have a em(4), emx(4), or igb(4) Intel device (http://lists.dragonflybsd.org/pipermail/users/2015-December/228461.html) DragonFly Testers wanted! Sephe has posted a request for users of the em(4), emx(4) and igb(4) intel drivers to test his latest branch and report back results He mentions that he has tested the models 82571, 82574 and 82573 (em/emx); 82575, 82576, 82580 and i350 specifically, so if you have something different, I'm sure he would be much appreciative of the help. It looks like the em(4) driver has been updated to 7.5.2, and igb(4) 2.4.3, and adds support for the I219-LM and I219-V NICS. *** OpenBSD Xen Support (https://marc.info/?l=openbsd-tech&m=144933933119525&w=2) Filed under the “Ohh, look what's coming soon” section, it appears that patches are starting to surface for OpenBSD Xen DOMU support. For those who aren't up on their Xen terminology, DomU is the unprivileged domain (I.E. Guest mode) Right now the patch exists at the link above, and adds a new (commented out) device to the GENERIC kernel, but this gives Xen users something new to watch for updates to. *** Thinkpad Backlit Keyboard support being worked on (http://freshbsd.org/commit/openbsd/b355449caa22e7bb6c460f7a647874836ef604f0) Another reason why Lenovo / ThinkPads are some of the best laptops currently to use with BSD, the kettenis over at the OpenBSD project has committed a patch to enable support for the “ThinkLight” For those who don't know, this is the little light that helps illuminate the laptop's keyboard under low-light situations. While the initial patch only supports the “real-deal” ThinkLight, he does mention that support will be added soon for the others on ThinkPads No sysctl's to fiddle with, this works directly with the ACPI / keyboard function keys directly, nice! *** Deadline is approaching for Submissions of Tutorial Proposals for AsiaBSDCon 2016 (https://2016.asiabsdcon.org/cfp.html) Call for Papers for BSDCAN 2016 now open (http://www.bsdcan.org/2016/papers.php) + The next two major BSD conferences both have their CFP up right now. First up is AsiaBSDCon in Tokyo from March 10th-13th, followed by BSDCan in Ottawa, June 8th-11th. + If you are working on anything interesting in the BSD community, this is a good way to get the word out about your project, plus the conference pays for Hotel / Travel. + If you can make it to both, DO SO, you won't regret it. Both Allan and Kris will be attending and we would look forward to meeting you. iohyve lands in ports (https://github.com/pr1ntf/iohyve) (http://www.freshports.org/sysutils/iohyve/) + Something we've mentioned in passing has taken its first steps in becoming reality for users! “iohyve” has now landed in the FreeBSD ports tree + While it shares a similar name to “iocage” its not directly related, different developers and such. However it does share a very similar syntax and some principles of ZFS usage + The current version is 0.7, but it already has a rather large feature set + Among the current features are ISO Management, resource management, snapshot support (via ZFS), and support for OpenBSD, NetBSD and Linux (Using grub-bhyve port) BeastieBits hammer mount is forced noatime by default (http://lists.dragonflybsd.org/pipermail/users/2015-November/228445.html) Show your support for FreeBSD (http://freebsdfoundation.blogspot.com/2015/12/show-your-support-for-freebsd.html) OpenBSD running in an Amazon EC2 t2.micro (https://gist.github.com/reyk/e23fde95354d4bc35a40) NetBSD's 2015Q4 Package freeze is coming (http://mail-index.netbsd.org/tech-pkg/2015/12/05/msg016059.html) ‘Screenshots from Developers' that we covered previously from 2002, updated for 2015 (https://anders.unix.se/2015/12/10/screenshots-from-developers--2002-vs.-2015/) Feedback/Questions (slexy was down when I made these, I only did 3, since the last is really long, save rest for next week) Mark - BSD laptops (http://pastebin.com/g0DnFG95) Jamie - zxfer (http://pastebin.com/BNCmDgTe) Anonymous - Long Story (http://pastebin.com/iw0dXZ9P) ***
Interview with Hiroki Sato and George Neville-Neil from AsiaBSDCon.More info at http://www.asiabsdcon.org/.File info: 6MB, 13Min.Ogg Link:https://archive.org/download/bsdtalk074/bsdtalk074.ogg
News:Call for papers and presentations from AsiaBSDCon.BSDCan pushed back a week to May 18 and 19.I had a great weekend at NYCBSDCon. Slides and audio are available at www.nycbsdcon.org.Interview with OpenBSD developer David Gwynne. We talk about the upcoming 4.0 release of OpenBSD and current projects that he is working on.File info: 8MB, 16Min.Ogg Link:https://archive.org/download/bsdtalk079/bsdtalk079.ogg
Interview with AsiaBSDCon 2007 Program Committee Chair George Neville-Neil.File Info: 14Min, 7MB.Ogg Link:https://archive.org/download/bsdtalk099/bsdtalk099.ogg
A quick update on AsiaBSDCon 2008 with Hiroki Sato and George Neville-Neil. More information at http://www.asiabsdcon.org/.File Info: 10 Min, 5MB.Ogg Link:https://archive.org/download/bsdtalk134/bsdtalk134.ogg
It's already our two-year anniversary! This time on the show, we'll be chatting with Scott Courtney, vice president of infrastructure engineering at Verisign, about this year's vBSDCon. What's it have to offer in an already-crowded BSD conference space? We'll find out. This episode was brought to you by Headlines OpenBSD hypervisor coming soon (https://www.marc.info/?l=openbsd-tech&m=144104398132541&w=2) Our buddy Mike Larkin never rests, and he posted some very tight-lipped console output (http://pastebin.com/raw.php?i=F2Qbgdde) on Twitter recently From what little he revealed at the time (https://twitter.com/mlarkin2012/status/638265767864070144), it appeared to be a new hypervisor (https://en.wikipedia.org/wiki/Hypervisor) (that is, X86 hardware virtualization) running on OpenBSD -current, tentatively titled "vmm" Later on, he provided a much longer explanation on the mailing list, detailing a bit about what the overall plan for the code is Originally started around the time of the Australia hackathon, the work has since picked up more steam, and has gotten a funding boost from the OpenBSD foundation One thing to note: this isn't just a port of something like Xen or Bhyve; it's all-new code, and Mike explains why he chose to go that route He also answered some basic questions about the requirements, when it'll be available, what OSes it can run, what's left to do, how to get involved and so on *** Why FreeBSD should not adopt launchd (http://blog.darknedgy.net/technology/2015/08/26/0/) Last week (http://www.bsdnow.tv/episodes/2015_08_26-beverly_hills_25519) we mentioned a talk Jordan Hubbard gave about integrating various parts of Mac OS X into FreeBSD One of the changes, perhaps the most controversial item on the list, was the adoption of launchd to replace the init system (replacing init systems seems to cause backlash, we've learned) In this article, the author talks about why he thinks this is a bad idea He doesn't oppose the integration into FreeBSD-derived projects, like FreeNAS and PC-BSD, only vanilla FreeBSD itself - this is also explained in more detail The post includes both high-level descriptions and low-level technical details, and provides an interesting outlook on the situation and possibilities Reddit had quite a bit (https://www.reddit.com/r/BSD/comments/3ilhpk) to say (https://www.reddit.com/r/freebsd/comments/3ilj4i) about this one, some in agreement and some not *** DragonFly graphics improvements (http://lists.dragonflybsd.org/pipermail/commits/2015-August/458108.html) The DragonFlyBSD guys are at it again, merging newer support and fixes into their i915 (Intel) graphics stack This latest update brings them in sync with Linux 3.17, and includes Haswell fixes, DisplayPort fixes, improvements for Broadwell and even Cherryview GPUs You should also see some power management improvements, longer battery life and various other bug fixes If you're running DragonFly, especially on a laptop, you'll want to get this stuff on your machine quick - big improvements all around *** OpenBSD tames the userland (https://www.marc.info/?l=openbsd-tech&m=144070638327053&w=2) Last week we mentioned OpenBSD's tame framework getting support for file whitelists, and said that the userland integration was next - well, now here we are Theo posted a mega diff of nearly 100 smaller diffs, adding tame support to many areas of the userland tools It's still a work-in-progress version; there's still more to be added (including the file path whitelist stuff) Some classic utilities are even being reworked to make taming them easier - the "w" command (https://www.marc.info/?l=openbsd-cvs&m=144103945031253&w=2), for example The diff provides some good insight on exactly how to restrict different types of utilities, as well as how easy it is to actually do so (and en masse) More discussion can be found on HN (https://news.ycombinator.com/item?id=10135901), as one might expect If you're a software developer, and especially if your software is in ports already, consider adding some more fine-grained tame support in your next release *** Interview - Scott Courtney - vbsdcon@verisign.com (mailto:vbsdcon@verisign.com) / @verisign (https://twitter.com/verisign) vBSDCon (http://vbsdcon.com/) 2015 News Roundup OPNsense, beyond the fork (https://opnsense.org/opnsense-beyond-the-fork) We first heard about (http://www.bsdnow.tv/episodes/2015_01_14-common_sense_approach) OPNsense back in January, and they've since released nearly 40 versions, spanning over 5,000 commits This is their first big status update, covering some of the things that've happened since the project was born There's been a lot of community growth and participation, mass bug fixing, new features added, experimental builds with ASLR and much more - the report touches on a little of everything *** LibreSSL nukes SSLv3 (http://undeadly.org/cgi?action=article&sid=20150827112006) With their latest release, LibreSSL began to turn off SSLv3 (http://disablessl3.com) support, starting with the "openssl" command At the time, SSLv3 wasn't disabled entirely because of some things in the OpenBSD ports tree requiring it (apache being one odd example) They've now flipped the switch, and the process of complete removal has started From the Undeadly summary, "This is an important step for the security of the LibreSSL library and, by extension, the ports tree. It does, however, require lots of testing of the resulting packages, as some of the fallout may be at runtime (so not detected during the build). That is part of why this is committed at this point during the release cycle: it gives the community more time to test packages and report issues so that these can be fixed. When these fixes are then pushed upstream, the entire software ecosystem will benefit. In short: you know what to do!" With this change and a few more to follow shortly, LibreSSL won't actually support SSL anymore - time to rename it "LibreTLS" *** FreeBSD MPTCP updated (http://caia.swin.edu.au/urp/newtcp/mptcp/tools/v05/mptcp-readme-v0.5.txt) For anyone unaware, Multipath TCP (https://en.wikipedia.org/wiki/Multipath_TCP) is "an ongoing effort of the Internet Engineering Task Force's (IETF) Multipath TCP working group, that aims at allowing a Transmission Control Protocol (TCP) connection to use multiple paths to maximize resource usage and increase redundancy." There's been work out of an Australian university to add support for it to the FreeBSD kernel, and the patchset was recently updated Including in this latest version is an overview of the protocol, how to get it compiled in, current features and limitations and some info about the routing requirements Some big performance gains can be had with MPTCP, but only if both the client and server systems support it - getting it into the FreeBSD kernel would be a good start *** UEFI and GPT in OpenBSD (https://www.marc.info/?l=openbsd-cvs&m=144092912907778&w=2) There hasn't been much fanfare about it yet, but some initial UEFI and GPT-related commits have been creeping into OpenBSD recently Some support (https://github.com/yasuoka/openbsd-uefi) for UEFI booting has landed in the kernel, and more bits are being slowly enabled after review This comes along with a number (https://www.marc.info/?l=openbsd-cvs&m=143732984925140&w=2) of (https://www.marc.info/?l=openbsd-cvs&m=144088136200753&w=2) other (https://www.marc.info/?l=openbsd-cvs&m=144046793225230&w=2) commits (https://www.marc.info/?l=openbsd-cvs&m=144045760723039&w=2) related to GPT, much of which is being refactored and slowly reintroduced Currently, you have to do some disklabel wizardry to bypass the MBR limit and access more than 2TB of space on a single drive, but it should "just work" with GPT (once everything's in) The UEFI bootloader support has been committed (https://www.marc.info/?l=openbsd-cvs&m=144115942223734&w=2), so stay tuned for more updates (http://undeadly.org/cgi?action=article&sid=20150902074526&mode=flat) as further (https://twitter.com/kotatsu_mi/status/638909417761562624) progress (https://twitter.com/yojiro/status/638189353601097728) is made *** Feedback/Questions John writes in (http://slexy.org/view/s2sIWfb3Qh) Mason writes in (http://slexy.org/view/s2Ybrx00KI) Earl writes in (http://slexy.org/view/s20FpmR7ZW) ***
Coming up this week, we'll be chatting with Lucas Holt, founder of MidnightBSD. It's a slightly lesser-known fork of FreeBSD, with a focus on easy desktop use. We'll find out what's different about it and why it was created. Answers to your emails and all this week's news, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Zocker, it's like docker on FreeBSD (http://toni.yweb.fi/2015/05/zocker-diy-docker-on-freebsd.html) Containment is always a hot topic, and docker has gotten a lot of hype in Linux land in the last couple years - they're working on native FreeBSD support at the moment This blog post is about a docker-like script, mainly for ease-of-use, that uses only jails and ZFS in the base system In total, it's 1,500 lines of shell script (https://github.com/toddnni/zocker) The post goes through the process of using the tool, showing off all the subcommands and explaining the configuration In contrast to something like ezjail, Zocker utilizes the jail.conf system in the 10.x branch *** Patrol Read in OpenBSD (https://www.marc.info/?l=openbsd-cvs&m=143285964216970&w=4) OpenBSD has recently imported some new code to support the Patrol Read (http://www.intel.com/support/motherboards/server/sb/CS-028742.htm) function of some RAID controllers In a nutshell, Patrol Read is a function that lets you check the health of your drives in the background, similar to a zpool "scrub" operation The goal is to protect file integrity by detecting drive failures before they can damage your data It detects bad blocks and prevents silent data corruption, while marking any bad sectors it finds *** HAMMER 2 improvements (http://lists.dragonflybsd.org/pipermail/commits/2015-May/418653.html) DragonFly BSD has been working on the second generation HAMMER FS It now uses LZ4 compression by default, which we've been big fans of in ZFS They've also switched to a faster CRC (http://lists.dragonflybsd.org/pipermail/commits/2015-May/418652.html) algorithm, further improving HAMMER's performance, especially (http://lists.dragonflybsd.org/pipermail/commits/2015-May/418651.html) when using iSCSI *** FreeBSD foundation May update (https://www.freebsdfoundation.org/press/2015mayupdate.pdf) The FreeBSD foundation has published another update newsletter, detailing some of the things they've been up to lately In it, you'll find some development status updates: notably more ARM64 work and the addition of 64 bit Linux emulation Some improvements were also made to FreeBSD's release building process for non-X86 architectures There's also an AsiaBSDCon recap that covers some of the presentations and the dev events They also have an accompanying blog post (http://freebsdfoundation.blogspot.com/2015/05/another-data-center-site-visit-nyi.html) where Glen Barber talks about more sysadmin and clusteradm work at NYI *** Interview - Lucas Holt - questions@midnightbsd.org (mailto:questions@midnightbsd.org) / @midnightbsd (https://twitter.com/midnightbsd) MidnightBSD News Roundup The launchd on train is never coming (http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/launchd-on-bsd.html) Replacement of init systems has been quite controversial in the last few years Fortunately, the BSDs have avoided most of that conflict thus far, but there have been a few efforts made to port launchd from OS X (https://en.wikipedia.org/wiki/Launchd) This blog post details the author's opinion on why he thinks we're never going to have launchd in any of the BSDs Email us your thoughts on the matter *** Native SSH comes to… Windows (http://blogs.msdn.com/b/looking_forward_microsoft__support_for_secure_shell_ssh1/archive/2015/06/02/managing-looking-forward-microsoft-support-for-secure-shell-ssh.aspx) In what may be the first (and last) mention of Microsoft on BSD Now... They've just recently announced that PowerShell will get native SSH support in the near future It's not based on the commercial SSH either, it's the same one from OpenBSD that we already use everywhere Up until now, interacting between BSD and Windows has required something like PuTTY, WinSCP, FileZilla or Cygwin - most of which are based on really outdated versions The announcement also promises that they'll be working with the OpenSSH community, so we'll see how many Microsoft-submitted patches make it upstream (or how many donations (http://www.openbsdfoundation.org/index.html) they make) *** Moving to FreeBSD (http://www.textplain.net/blog/2015/moving-to-freebsd/) This blog post describes a long-time Linux user's first BSD switching experience The author first talks about his Linux journey, eventually coming to love the more customization-friendly systems, but the journey ended with systemd After doing a bit of research, he gave FreeBSD a try and ended up liking it - the rest of the post mostly covers why that is He also plans to write about his experience with other BSDs, and is writing some tutorials too - we'll check in with him again later on *** Feedback/Questions Adam writes in (http://slexy.org/view/s29hS2cI05) Dan writes in (http://slexy.org/view/s20VRZYBsw) Ivan writes in (http://slexy.org/view/s20bumJ5u9) Josh writes in (http://slexy.org/view/s21BU6Pnka) ***
On this week's mini-episode, we'll be talking with Baptiste Daroussin about packaging the FreeBSD base system with pkgng. Is this the best way going forward, or are we getting dangerously close to being Linux-like? We'll find out, and also get to a couple of your emails while we're at it, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Xen dom0 in FreeBSD 11-CURRENT (https://svnweb.freebsd.org/ports?view=revision&revision=382965) FreeBSD has just gotten dom0 (http://wiki.xen.org/wiki/Dom0) support for the Xen hypervisor, something NetBSD has had (http://wiki.netbsd.org/ports/xen/howto/#netbsd-dom0) for a while now The ports tree will now have a Xen kernel and toolstack, meaning that they can be updated much more rapidly than if they were part of base It's currently limited to Intel boxes with EPT and a working IOMMU, running a recent version of the -CURRENT branch, but we'll likely see it when 11.0 comes out How will this affect interest in Bhyve? *** A tale of two educational moments (http://blog.anthrobsd.net/044.html) Here we have a blog post from an OpenBSD developer about some experiences he had helping people get involved with the project It's split into two stories: one that could've gone better, and one that went really well For the first one, he found that someone was trying to modify a package from their ports tree to have fewer dependencies Experience really showed its worth, and he was able to write a quick patch to do exactly what the other person had been working on for a few hours - but wasn't so encouraging about getting it committed In the second story, he discussed updating a different port with a user of a forum, and ended up improving the new user's workflow considerably with just a few tips The lesson to take away from this is that we can all help out to encourage and assist new users - everyone was a newbie once *** What's coming in NetBSD 7 (http://saveosx.org/NetBSD7/) We first mentioned NetBSD 7.0 on the show in July of 2014, but it still hasn't been released and there hasn't been much public info about it This blog post outlines some of the bigger features that we can expect to see when it actually does come out Their total platform count is now over 70, so you'd be hard-pressed to find something that it doesn't run on There have been a lot of improvements in the graphics area, particularly with DRM/KMS, including Intel Haswell and Nouveau (for nVidia cards) Many ARM boards now have full SMP support Clang has also finally made its way into the base system, something we're glad to see, and it should be able to build the base OS on i386, AMD64 and ARM - other architectures are still a WIP In the crypto department: their PNRG has switched from the broken RC4 to the more modern ChaCha20, OpenSSL has been updated in base and LibreSSL is in pkgsrc NetBSD's in-house firewall, npf, has gotten major improvements since its initial debut in NetBSD 6.0 Looking to the future, NetBSD hopes to integrate a stable ZFS implementation later on *** OpenZFS office hours (https://www.youtube.com/watch?v=mS4bfbEq46I) We mentioned a couple weeks back that the OpenZFS office hours series was starting back up They've just uploaded the recording of their most recent freeform discussion, with Justin Gibbs (http://www.bsdnow.tv/episodes/2015_03_11-the_pcbsd_tour_ii) being the main presenter In it, they cover how Justin got into ZFS, running in virtualized environments, getting patches into the different projects, getting more people involved, reviewing code, spinning disks vs SSDs, defragging, speeding up resilvering, zfsd and much more *** Interview - Baptiste Daroussin - bapt@freebsd.org (mailto:bapt@freebsd.org) Packaging the FreeBSD base system with pkgng Discussion Packaging the FreeBSD base system with pkgng (follow-up) Feedback/Questions Jeff writes in (http://slexy.org/view/s20AWp6Av1) Anonymous writes in (http://slexy.org/view/s20QiFcdh8) Alex writes in (http://slexy.org/view/s2YzZlswaB) Joris writes in (http://slexy.org/view/s21Mx9TopQ) *** Mailing List Gold ok feedback@ (https://www.marc.info/?l=openbsd-ports&m=142679136422432&w=2) ***
Coming up this week on the show, we'll be talking to Kamila Součková, a Google intern. She's been working on the FreeBSD pager daemon, and also tells us about her initial experiences trying out BSD and going to a conference. As always, all the week's news and answers to your emails, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Major changes coming in PCBSD 11 (http://blog.pcbsd.org/2015/04/huge-announcement-for-pc-bsd/) The PCBSD team has announced that version 11.0 will have some more pretty big changes (as they've been known to do lately with NTP daemons and firewalls) Switching from PF to IPFW provided some benefits for VIMAGE, but the syntax was just too complicated for regular everyday users To solve this, they've ported over Linux's iptables, giving users a much more straightforward configuration (http://dpaste.com/2F1KM6T.txt) While ZFS has served them well as the default filesystem for a while, Kris decided that Btrfs would be a better choice going forward Since the FreeBSD kernel doesn't support it natively, all filesystem calls will be through FUSE from now on - performance is Good Enough People often complain about PCBSD's huge ISO download, so, to save space, the default email client will be switched to mutt, and KDE will be replaced with DWM as the default window manager To reconfigure it, or make any appearance changes, users just need to edit a simple C header file and recompile - easy peasy As we've mentioned on the show, PCBSD has been promoting safe backup solutions for a long time with its "life preserver" utility, making it simple to manage multiple snapshots too To test if people have been listening to this advice, Kris recently activated the backdoor he put in life preserver that deletes all the users' files - hope you had that stuff backed up *** NetBSD and FreeBSD join forces (http://www.freebsddiary.org/fretbsd.php) The BSD community has been running into one of the same problems Linux has lately: we just have too many different BSDs to choose from What's more, none of them have any specific areas they focus on or anything like that (they're all basically the same) That situation is about to improve somewhat, as FreeBSD and NetBSD have just merged codebases... say hello to FretBSD Within a week, all mailing lists and webservers for the legacy NetBSD and FreeBSD projects will be terminated - the mailing list for the new combined project will be hosted from the United Nations datacenter on a Microsoft Exchange server As UN monitors will be moderating the mailing lists to prevent disagreements and divisive arguments before they begin, this system is expected to be adequate for the load With FretBSD, your toaster can now run ZFS, so you'll never need to worry about the bread becoming silently corrupted again *** Puffy in the cloud (http://homing-on-code.blogspot.com/2015/03/puffy-in-cloud.html) If you've ever wanted to set up a backup server, especially for family members or someone who's not as technology-savvy, you've probably realized there are a lot of options This post explores the option of setting up your own Dropbox-like service with Owncloud and PostgreSQL, running atop the new OpenBSD http daemon Doing it this way with your own setup, you can control all the security aspects - disk encryption, firewall rules, who can access what and from where, etc He also mentions our pf tutorial (http://www.bsdnow.tv/tutorials/pf) being helpful in blocking script kiddies from hammering the box Be sure to encourage your less-technical friends to always back up their important data *** NetBSD at AsiaBSDCon (https://blog.netbsd.org/tnf/entry/asiabsdcon_2015) Some NetBSD developers have put together a report of what they did at the most recent event in Tokyo It includes a wrap-up of the event, as well as a list of presentations (https://www.netbsd.org/gallery/presentations/#asiabsdcon2015) that NetBSD developers gave Have you ever wanted even more pictures of NetBSD running on lots of devices? There's a never-ending supply, apparently At the BSD research booth of AsiaBSDCon, there were a large number of machines on display, and someone has finally uploaded pictures of all of them (http://www.ki.nu/~makoto/p15/20150315/) There's also a video (https://www.youtube.com/watch?v=K1y9cdmLFjw) of an OMRON LUNA-II running the luna68k port *** Interview - Kamila Součková - kamila@ksp.sk (mailto:kamila@ksp.sk) / @anotherkamila (https://twitter.com/anotherkamila) BSD conferences, Google Summer of Code, various topics News Roundup FreeBSD foundation March update (https://www.freebsdfoundation.org/press/2015marchupdate.pdf) The FreeBSD foundation has published their March update for fundraising and sponsored projects In the document, you'll find information about upcoming ARMv8 enhancements, some event recaps and a Google Summer of Code status update They also mention our interview with the foundation president (http://www.bsdnow.tv/episodes/2015_03_11-the_pcbsd_tour_ii) - be sure to check it out if you haven't *** Inside OpenBSD's new httpd (http://sdtimes.com/inside-openbsds-new-httpd-web-server/) BSD news continues to dominate mainstream tech news sites… well not really, but they talk about it once in a while The SD Times is featuring an article about OpenBSD's in-house HTTP server, after seeing Reyk's AsiaBSDCon presentation (http://www.openbsd.org/papers/httpd-slides-asiabsdcon2015.pdf) about it (which he's giving at BSDCan this year, too) In this article, they talk about the rapid transition of webservers in the base system - apache being replaced with nginx, only to be replaced with httpd shortly thereafter Since the new daemon has had almost a full release cycle to grow, new features and fixes have been pouring in The post also highlights some of the security features: everything runs in a chroot with privsep by default, and it also leverages strong TLS 1.2 defaults (including Perfect Forward Secrecy) *** Using poudriere without OpenSSL (http://bsdxbsdx.blogspot.com/2015/04/build-packages-in-poudriere-without.html) Last week we talked about (http://www.bsdnow.tv/episodes/2015_03_25-ssl_in_the_wild) using LibreSSL in FreeBSD for all your ports One of the problems that was mentioned is that some ports are configured improperly, and end up linking against the OpenSSL in the base system even when you tell them not to This blog post shows how to completely strip OpenSSL out of the poudriere (http://www.bsdnow.tv/tutorials/poudriere) build jails, something that's a lot more difficult than you'd think If you're a port maintainer, pay close attention to this post, and get your ports fixed to adhere to the make.conf options properly *** HAMMER and GPT in OpenBSD (https://www.marc.info/?l=openbsd-tech&m=142755452428573&w=2) Someone, presumably a Google Summer of Code student, wrote in to the lists about his HAMMER FS (http://www.bsdnow.tv/tutorials/hammer) porting proposal He outlined the entire process and estimated timetable, including what would be supported and which aspects were beyond the scope of his work (like the clustering stuff) There's no word yet on if it will be accepted, but it's an interesting idea to explore, especially when you consider that HAMMER really only has one developer In more disk-related news, Ken Westerback (http://www.bsdnow.tv/episodes/2015_02_25-from_the_foundation_2) has been committing quite a lot of GPT-related fixes (https://www.marc.info/?l=openbsd-cvs&w=2&r=1&s=gpt&q=b) recently Full GPT support will most likely be finished before 5.8, but anything involving HAMMER FS is still anyone's guess *** Feedback/Questions Morgan writes in (http://slexy.org/view/s20e30p4qf) Dustin writes in (http://slexy.org/view/s20clKByMP) Stan writes in (http://slexy.org/view/s20aBlmaT5) Mica writes in (http://slexy.org/view/s2ufFrZY9y) *** Mailing List Gold Developers in freefall (https://lists.freebsd.org/pipermail/freebsd-current/2015-April/055281.html) Xorg thieves pt. 1 (https://www.marc.info/?l=openbsd-cvs&m=142786808725483&w=4) Xorg thieves pt. 2 (https://www.marc.info/?l=openbsd-cvs&m=142790740405547&w=4) ***
We're back from AsiaBSDCon! This week on the show, we'll be talking to Lawrence Teo about how Calyptix uses OpenBSD in their line of commercial routers. They're getting BSD in the hands of Windows admins who don't even realize it. We also have all this week's news and answer to your emails, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Using OpenBGPD to distribute pf table updates (http://www.echothrust.com/blogs/using-openbgpd-distribute-pf-table-updates-your-servers) For those not familiar, OpenBGPD (https://en.wikipedia.org/wiki/OpenBGPD) is a daemon for the Border Gateway Protocol (https://en.wikipedia.org/wiki/Border_Gateway_Protocol) - a way for routers on the internet to discover and exchange routes to different addresses This post, inspired by a talk about using BGP to distribute spam lists (https://www.youtube.com/watch?v=Vet0eQB00X0), details how to use the protocol to distribute some other useful lists and information It begins with "One of the challenges faced when managing our OpenBSD firewalls is the distribution of IPs to pf tables without manually modifying /etc/pf.conf on each of the firewalls every time. This task becomes quite tedious, specifically when you want to distribute different types of changes to different systems (eg administrative IPs to a firewall and spammer IPs to a mail server), or if you need to distribute real time blacklists to a large number of systems." If you manage a lot of BSD boxes, this might be an interesting alternative to some of the other ways to distribute configuration files OpenBGPD is part of the OpenBSD base system, but there's also an unofficial port to FreeBSD (https://www.freshports.org/net/openbgpd/) and a "work in progress" pkgsrc version (http://pkgsrc.se/wip/openbgpd) *** Mounting removable media with autofs (http://freebsdfoundation.blogspot.com/2015/03/freebsd-from-trenches-using-autofs5-to_13.html) The FreeBSD foundation has a new article in the "FreeBSD from the trenches" series, this time about the sponsored autofs (https://www.freebsd.org/cgi/man.cgi?query=autofs&sektion=5) tool It's written by one of the autofs developers, and he details his work on creating and using the utility "The purpose of autofs(5) is to mount filesystems on access, in a way that's transparent to the application. In other words, filesystems get mounted when they are first accessed, and then unmounted after some time passes." He talks about all the components that need to work together for smooth operation, how to configure it and how to enable it by default for removable drives It ends with a real-world example of something we're all probably familiar with: plugging in USB drives and watching the magic happen There's also some more advanced bonus material on GEOM classes and all the more technical details *** The Tor Browser on BSD (http://trac.haqistan.net/blog/adventures-ports-tor-browser) The Tor Project has provided a "browser bundle (https://www.torproject.org/projects/torbrowser/design/)" for a long time, which is more or less a repackaged Firefox with many security and privacy-related settings preconfigured and some patches applied to the source Just tunneling your browser through a transparent Tor proxy is not safe enough - many things can lead to passive fingerprinting or, even worse, anonymity being completely lost It has, however, only been released for Windows, OS X and Linux - no BSD version "[...] we are pushing back against an emerging monoculture, and this is always a healthy thing. Monocultures are dangerous for many reasons, most importantly to themselves." Some work has begun to get a working port on BSD going, and this document tells about the process and how it all got started If you've got porting skills, or are interested in online privacy, any help would be appreciated of course (see the post for details on getting involved) *** OpenSSH 6.8 released (https://lists.mindrot.org/pipermail/openssh-unix-dev/2015-March/033686.html) Continuing their "tick tock" pattern of releases alternating between new features and bugfixes, the OpenSSH team has released 6.8 - it's a major upgrade, focused on new features (we like those better of course) Most of the codebase has gone through refactoring, making it easier for regression tests and improving the general readability This release adds support for SHA256-hashed, base64-encoded host key fingerprints, as well as making that the default - a big step up from the previously hex-encoded MD5 fingerprints Experimental host key rotation support also makes it debut, allowing for easy in-place upgrading of old keys to newer (or refreshed) keys You can now require multiple, different public keys to be verified for a user to authenticate (useful if you're extra paranoid or don't have 100% confidence in any single key type) The native version will be in OpenBSD 5.7, and the portable version should hit a ports tree near you soon Speaking of the portable version, it now has a configure option to build without OpenSSL or LibreSSL, but doing so limits you to Ed25519 key types and ChaCha20 and AES-CTR ciphers *** NetBSD at AsiaBSDCon (https://mail-index.netbsd.org/netbsd-advocacy/2015/03/15/msg000682.html) The NetBSD guys already have a wrap-up of the recent event, complete with all the pictures and weird devices you'd expect It covers their BoF session, the six NetBSD-related presentations and finally their "work in progress" session There was a grand total of 34 different NetBSD gadgets (https://docs.google.com/spreadsheets/d/14q6zJK5PjlMoSeBV5HBiEik5LkqlrcrbSxPoxVKKlec/edit#gid=0) on display at the event *** Interview - Lawrence Teo - lteo@openbsd.org (mailto:lteo@openbsd.org) / @lteo (https://twitter.com/lteo) OpenBSD at Calyptix (http://www.nycbsdcon.org/2010/presentations/lteo-nycbsdcon2010.pdf) News Roundup HardenedBSD introduces Integriforce (http://hardenedbsd.org/article/shawn-webb/2015-03-11/call-testing-secadm-integriforce) A little bit of background on this one first: NetBSD has something called veriexec (https://www.netbsd.org/docs/guide/en/chap-veriexec.html), used for checking file integrity (http://wiki.netbsd.org/guide/veriexec/) at the kernel level By doing it at the kernel level, similar to securelevels (https://en.wikipedia.org/wiki/Securelevel), it offers some level of protection even when the root account is compromised HardenedBSD has introduced a similar mechanism into their "secadm" utility You can list binaries in the config file that you want to be protected from changes, then specify whether those can't be run (http://i.imgur.com/wHp2eAN.png) at all, or if they just print a warning They're looking for some more extensive testing of this new feature *** More s2k15 hackathon reports (http://undeadly.org/cgi?action=article&sid=20150305100712&mode=flat) A couple more Australian hackathon reports have poured in since the last time The first comes from Jonathan Gray, who's done a lot of graphics-related work in OpenBSD recently He worked on getting some newer "Southern Islands" and "Graphics Core Next" AMD GPUs working, as well as some OpenGL and DRM-related things Also on his todo list was to continue hitting various parts of the tree with American Fuzzy Lop, which ended up fixing a few crashes in mandoc (http://www.bsdnow.tv/episodes/2014_11_12-a_mans_man) Ted Unangst also sent in a report (http://undeadly.org/cgi?action=article&sid=20150307165135&mode=flat) to detail what he hacked on at the event With a strong focus on improving SMP scalability, he tackled the virtual memory layer His goal was to speed up some syscalls that are used heavily during code compilation, much of which will probably end up in 5.8 All the trip reports are much more detailed than our short summaries, so give them a read if you're interested in all the technicalities *** DragonFly 4.0.4 and IPFW3 (https://www.dragonflydigest.com/2015/03/10/15733.html) DragonFly BSD has put out a small point release to the 4.x branch, 4.0.4 It includes a minor list of fixes (http://lists.dragonflybsd.org/pipermail/commits/2015-March/418098.html), some of which include a HAMMER FS history fix, removing the no-longer-needed "new xorg" and "with kms" variables and a few LAGG fixes There was also a bug in the installer that prevented the rescue image from being installed correctly, which also gets fixed in this version Shortly after it was released, their new IPFW2 firewall was added to the tree (http://lists.dragonflybsd.org/pipermail/commits/2015-March/418133.html) and subsequently renamed to IPFW3 (http://lists.dragonflybsd.org/pipermail/commits/2015-March/418160.html) (since it's technically the third revision) *** NetBSD gets Raspberry Pi 2 support (https://blog.netbsd.org/tnf/entry/raspberry_pi_2_support_added) NetBSD has announced initial support for the second revision (http://www.raspberrypi.org/products/raspberry-pi-2-model-b/) of the ever-popular Raspberry Pi board There are -current snapshots available for download, and multiprocessor support is also on the way The NetBSD wiki page about the Raspberry Pi also has some more information (https://wiki.netbsd.org/ports/evbarm/raspberry_pi/) and an installation guide The usual Hacker News discussion (https://news.ycombinator.com/item?id=9172100) on the subject If anyone has one of these little boards, let us know - maybe write up a blog post about your experience with BSD on it *** OpenIKED as a VPN gateway (http://puffysecurity.com/wiki/openikedoffshore.html) In our first discussion segment, we talked about a few different ways to tunnel your traffic While we've done full tutorials on things like SSH tunnels (http://www.bsdnow.tv/tutorials/stunnel), OpenVPN (http://www.bsdnow.tv/tutorials/openvpn) and Tor (http://www.bsdnow.tv/tutorials/tor), we haven't talked a whole lot about OpenBSD's IPSEC suite This article should help fill that gap - it walks you through the complete IKED setup From creating the public key infrastructure to configuring the firewall to configuring both the VPN server and client, this guide's got it all *** Feedback/Questions Gary writes in (http://slexy.org/view/s21G9TWALE) Robert writes in (http://slexy.org/view/s206aZrxOi) Joris writes in (http://slexy.org/view/s28Um5R7LG) Mike writes in (http://slexy.org/view/s2yAJsl1Es) Anders writes in (http://slexy.org/view/s21dMAE55M) *** Mailing List Gold Can you hear me now (https://www.marc.info/?l=openbsd-misc&m=142577632205484&w=2) He must be GNU here (https://lists.freebsd.org/pipermail/freebsd-hackers/2015-March/047207.html) I've seen some... (https://www.marc.info/?l=openbsd-cvs&m=142593175408756&w=2) ***
We're away at AsiaBSDCon this week, but we've still got a packed episode for you. First up is a sequel to the "PC-BSD tour" segment from a while back, highlighting how ZFS boot environments work. After that, Justin Gibbs joins us to talk about the FreeBSD foundation's 15th anniversary. We'll return next week with a normal episode of BSD Now - which is of course, the place to B.. SD. This episode was brought to you by Special segment Demystifying Boot Environments in PC-BSD Interview - Justin Gibbs - gibbs@freebsd.org (mailto:gibbs@freebsd.org) / @freebsdfndation (https://twitter.com/freebsdfndation) The FreeBSD foundation's 15th anniversary Discussion The story of PC-BSD
Coming up this time on the show, we'll be talking to Sean Bruno. He's been using poudriere and QEMU to cross compile binary packages, and has some interesting stories to tell about it. We've also got answers to viewer-submitted questions and all this week's news, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines AsiaBSDCon 2015 schedule (http://2015.asiabsdcon.org/timetable.html.en) Almost immediately after we finished recording an episode last week, the 2015 AsiaBSDCon schedule went up This year's conference will be between 12-15 March at the Tokyo University of Science in Japan The first and second days are for tutorials, as well as the developer summit and vendor summit Days four and five are the main event with the presentations, which Kris and Allan both made the cut for once again Not counting the ones that have yet to be revealed (as of the day we're recording this), there will be thirty-six different talks in all - four BSD-neutral, four NetBSD, six OpenBSD and twenty-two FreeBSD Summaries of all the presentations are on the timetable page if you scroll down a bit *** FreeBSD foundation updates and more (https://www.freebsdfoundation.org/press/2015febupdate.pdf) The FreeBSD foundation (http://www.bsdnow.tv/episodes/2015_02_04-from_the_foundation_1) has posted a number of things this week, the first of which is their February 2015 status update It provides some updates on the funded projects, including PCI express hotplugging and FreeBSD on the POWER8 platform There's a FOSDEM recap and another update of their fundraising goal for 2015 They also have two new blog posts: a trip report from SCALE13x (http://freebsdfoundation.blogspot.com/2015/02/scale-13x-trip-report-michael-dexter.html) and a featured "FreeBSD in the trenches (http://freebsdfoundation.blogspot.com/2015/02/freebsd-from-trenches-zfs-and-how-to.html)" article about how a small typo caused a lot of ZFS chaos in the cluster "Then panic ensued. The machine didn't panic -- I did." *** OpenBSD improves browser security (https://www.marc.info/?l=openbsd-misc&m=142523501726732&w=2) No matter what OS you run on your desktop, the most likely entry point for an exploit these days is almost certainly the web browser Ted Unangst writes in to the OpenBSD misc list to introduce a new project he's working on, simply titled "improving browser security" He gives some background on the W^X memory protection (https://en.wikipedia.org/wiki/W%5EX) in the base system, but also mentions that some applications in ports don't adhere to it For it to be enforced globally instead of just recommended, at least one browser (or specifically, one JIT (https://en.wikipedia.org/wiki/Just-in-time_compilation) engine) needs to be fixed to use it "A system that is 'all W^X except where it's not' is the same as a system that's not W^X. We've worked hard to provide a secure foundation for programs; we'd like to see them take advantage of it." The work is being supported by the OpenBSD foundation (http://www.bsdnow.tv/episodes/2015_02_25-from_the_foundation_2), and we'll keep you updated on this undertaking as more news about it is released There's also some discussion on Hacker News (https://news.ycombinator.com/item?id=9128360) and Undeadly (http://undeadly.org/cgi?action=article&sid=20150303075848&mode=expanded) about it *** NetBSD at Open Source Conference 2015 Tokyo (https://mail-index.netbsd.org/netbsd-advocacy/2015/02/28/msg000680.html) The Japanese NetBSD users group has once again invaded a conference, this time in Tokyo There's even a spreadsheet (https://docs.google.com/spreadsheets/d/1DTJbESfnOUgOiVkFG8vsrxTq6oCGRpf8PkRcMkhWYWQ/edit#gid=0) of all the different platforms they were showing off at the booth (mostly ARM, MIPS, PowerPC and Landisk this time around) If you just can't get enough strange devices running BSD, check the mailing list post for lots of pictures Their next target is, as you might guess, AsiaBSDCon 2015 - maybe we'll run into them *** Interview - Sean Bruno - sbruno@freebsd.org (mailto:sbruno@freebsd.org) / @franknbeans (https://twitter.com/franknbeans) Cross-compiling packages with poudriere (http://www.bsdnow.tv/tutorials/poudriere) and QEMU News Roundup The Crypto Bone (http://crypto-bone.com/what.html) The Crypto Bone is a new device (http://www.crypto-bone.com/) that's aimed at making encryption and secure communications easier (http://crypto-bone.com/cbb-usersview.html) and more accessible Under the hood, it's actually just a Beaglebone (http://beagleboard.org/bone) board, running stock OpenBSD with a few extra packages It includes a web interface (http://crypto-bone.com/release/root/var/www/apache/html/) for configuring keys and secure tunnels The source code (http://crypto-bone.com/release/root/) is freely available for anyone interested in hacking on it (or auditing the crypto), and there's a technical overview (http://crypto-bone.com/cbb-technicalview.html) of how everything works on their site If you don't want to teach your mom how to use PGP, buy her one of these(?) *** BSD in the 2015 Google Summer of Code (https://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2015/about_page) For those who don't know, GSoC is a way for students to get paid to work on a coding project for an open source organization Good news: both FreeBSD and OpenBSD were accepted (https://www.google-melange.com/gsoc/org/list/public/google/gsoc2015) for the 2015 event FreeBSD has a wiki page (https://wiki.freebsd.org/SummerOfCodeIdeas) of ideas for people to work on OpenBSD also has an ideas page (http://www.openbsdfoundation.org/gsoc2015.html) where you can see some of the initial things that might be interesting If you're a student looking to get involved with BSD development, this might be a great opportunity to even get paid to do it Who knows, you may even end up on the show (http://www.bsdnow.tv/episodes/2015_01_07-system_disaster) if you work on a cool project GSoC will be accepting idea proposals starting March 16th, so you have some time to think about what you'd like to hack on *** pfSense 2.3 roadmap (https://blog.pfsense.org/?p=1588) The pfSense team has posted a new blog entry, detailing some of their plans for future versions PPTP will finally be deprecated, PHP will be updated to 5.6 and other packages will also get updated to newer versions PBIs are scheduled to be replaced with native pkgng packages Version 3.0, something coming much later, will be a major rewrite that gets rid of PHP entirely Their ultimate goal is for pfSense to be a package you can install atop of a regular FreeBSD install, rather than a repackaged distribution *** PCBSD 10.1.2 security features (http://blog.pcbsd.org/2015/03/a-look-at-the-upcoming-features-for-10-1-2/) PCBSD 10.1.2 will include a number of cool security features, some of which are detailed in a new blog post A new "personacrypt" utility is introduced, which allows for easy encryption and management of external drives for your home directory Going along with this, it also has a "stealth mode" that allows for one-time temporary home directories (but it doesn't self-destruct, don't worry) The LibreSSL integration also continues, and now packages will be built with it by default If you're using the Life Preserver utility for backups, it will encrypt the remote copy of your files in the next update They've also been working on introducing some new options to enable tunneling your traffic through Tor There will now be a fully-transparent proxy option that utilizes the switch to IPFW we mentioned last week A small disclaimer: remember that many things can expose your true IP when using Tor, so use this option at your own risk if you require full anonymity Look forward to Kris wearing a Tor shirt (https://www.torproject.org/getinvolved/tshirt.html) in future episodes *** Feedback/Questions Antonio writes in (http://slexy.org/view/s2ofBPRT5n) Chris writes in (http://slexy.org/view/s26LsYcoJF) Van writes in (http://slexy.org/view/s28Rho0jvL) Stu writes in (http://slexy.org/view/s21AkGbniU) *** Mailing List Gold H (https://lists.freebsd.org/pipermail/freebsd-ports/2015-February/098183.html) Pay up, mister Free (https://lists.freebsd.org/pipermail/freebsd-chat/2015-February/007024.html) Heritage protected (https://www.mail-archive.com/tech%40openbsd.org/msg22663.html) Blind leading the blind (https://lists.freebsd.org/pipermail/freebsd-questions/2015-February/264466.html) What are the chances (https://lists.freebsd.org/pipermail/svn-src-head/2015-February/068682.html) ***
This week on the show, we'll be chatting with Alex Reece and Matt Ahrens about what's new in the world of OpenZFS. After that, we're starting a new tutorial series on submitting your first patch. All the latest BSD news and answers to your emails, coming up on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Revisiting FreeBSD after 20 years (http://changelog.complete.org/archives/9317-has-linux-lost-its-way-comments-prompt-a-debian-developer-to-revisit-freebsd-after-20-years) With comments like "has Linux lost its way?" floating around, a Debian developer was prompted to revisit FreeBSD after nearly two decades This blog post goes through his experiences trying out a modern BSD variant, and includes the good, the bad and the ugly - not just praise this time He loves ZFS and the beadm tool, and finds the FreeBSD implementation to be much more stable than ZoL On the topic of jails, he summarizes: "Linux has tried so hard to get this right, and fallen on its face so many times, a person just wants to take pity sometimes. We've had linux-vserver, openvz, lxc, and still none of them match what FreeBSD jails have done for a long time." The post also goes through the "just plain different" aspects of a complete OS vs. a distribution of various things pieced together Finally, he includes some things he wasn't so happy about: subpar laptop support, virtualization being a bit behind, a myriad of complaints about pkgng and a few other things There was some decent discussion (https://news.ycombinator.com/item?id=9063216) on Hacker News about this article too, with counterpoints from both sides *** s2k15 hackathon report: network stack SMP (http://undeadly.org/cgi?action=article&sid=20150218085759) The first trip report from the recent OpenBSD hackathon in Australia has finally been submitted One of the themes of this hackathon was SMP (symmetric multiprocessing) improvement, and Martin Pieuchot did some hacking on the network stack If you're not familiar with him, he gave a presentation (http://www.openbsd.org/papers/tamingdragons.pdf) at EuroBSDCon last year, titled Taming OpenBSD Network Stack Dragons (https://va.ludost.net/files/eurobsdcon/2014/Rodopi/03.Saturday/03.Taming%20OpenBSD%20Network%20Stack%20Dragons%20-%20Martin%20Pieuchot.mp4) Teaming up with David Gwynne, they worked on getting some bits of the networking code out of the big lock (https://en.wikipedia.org/wiki/Giant_lock) Hopefully more trip reports will be sent in during the coming weeks Most of the big code changes should probably appear after the 5.7-release testing period *** From BIND to NSD and Unbound (https://www.tumfatig.net/20150215/bind-nsd-unbound-openbsd-5-6/) If you've been running a DNS server on any of the BSDs, you've probably noticed a semi-recent trend: BIND being replaced with Unbound BIND was ripped out in FreeBSD 10.0 and will be gone in OpenBSD 5.7, but both systems include Unbound now as an alternative OpenBSD goes a step further, also including NSD in the base system, whereas you'll need to install that from ports on FreeBSD Instead of one daemon doing everything like BIND tried to do, this new setup splits the authoritative nameserver and the caching resolver into two separate daemons This post takes you through the transitional phase of going from a single BIND setup to a combination of NSD and Unbound All in all, everyone wins here, as there will be a lot less security advisories in both BSDs because of it... *** m0n0wall calls it quits (http://m0n0.ch/wall/end_announcement.php) The original, classic BSD firewall distribution m0n0wall (https://en.wikipedia.org/wiki/M0n0wall) has finally decided to close up shop For those unfamiliar, m0n0wall was a FreeBSD-based firewall project that put a lot of focus on embedded devices: running from a CF card, CD, USB drive or even a floppy disk It started over twelve years ago, which is pretty amazing when you consider that's around half of FreeBSD itself's lifespan The project was probably a lot of people's first encounter with BSD in any form If you were a m0n0wall user, fear not, you've got plenty of choices for a potential replacement: doing it yourself with something like FreeBSD (http://blog.pcbsd.org/2015/01/using-trueos-as-a-ipfw-based-home-router/) or OpenBSD (http://www.bsdnow.tv/tutorials/openbsd-router), or going the premade route with something like pfSense (http://www.bsdnow.tv/episodes/2014_02_19-a_sixth_pfsense), OPNsense (http://www.bsdnow.tv/episodes/2015_01_14-common_sense_approach) or the BSD Router Project (http://www.bsdnow.tv/episodes/2014_10_22-dont_buy_a_router) The founder's announcement includes these closing words: "m0n0wall has served as the seed for several other well known open source projects, like pfSense, FreeNAS and AskoziaPBX. The newest offspring, OPNsense, aims to continue the open source spirit of m0n0wall while updating the technology to be ready for the future. In my view, it is the perfect way to bring the m0n0wall idea into 2015, and I encourage all current m0n0wall users to check out OPNsense and contribute if they can." While m0n0wall didn't get a lot of on-air mention, surely a lot of our listeners will remember it fondly *** Interview - Alex Reece & Matt Ahrens - alex@delphix.com (mailto:alex@delphix.com) & matt@delphix.com (mailto:matt@delphix.com) / @openzfs (https://twitter.com/openzfs) What's new in OpenZFS Tutorial Making your first patch (OpenBSD) (http://www.bsdnow.tv/tutorials/patching-obsd) News Roundup Overlaying remote LANs with OpenBSD's VXLAN (http://www.echothrust.com/blogs/using-openbsd-and-vxlan-overlay-remote-lans) Have you ever wanted to "merge" multiple remote LANs? OpenBSD's vxlan(4) (http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/vxlan.4) is exactly what you need This article talks about using it to connect two virtualized infrastructures on different ESXi servers It gives a bit of networking background first, in case you're not quite up to speed on all this stuff This tool opens up a lot of very cool possibilities, even possibly doing a "remote" LAN party Be sure to check the AsiaBSDCon talk (https://www.youtube.com/watch?v=ufeEP_hzFN0) about VXLANs if you haven't already *** 2020, year of the PCBSD desktop (http://lukewolf.blogspot.com/2015/02/a-prediction-2020-year-of-pc-bsd-on.html) Here we have a blog post about BSD on the desktop, straight from a KDE developer He predicts that PCBSD is going to take off before the year 2020, possibly even overtaking Linux's desktop market share (small as it may be) With PCBSD making a preconfigured FreeBSD desktop a reality, and the new KMS work, the author is impressed with how far BSD has come as a viable desktop option ZFS and easy-to-use boot environments top the list of things he says differentiate the BSD desktop experience from the Linux one There was also some discussion on Slashdot (http://bsd.slashdot.org/story/15/02/16/2355236/pc-bsd-set-for-serious-growth) that might be worth reading *** OpenSSH host key rotation, redux (http://blog.djm.net.au/2015/02/hostkey-rotation-redux.html) We mentioned the new OpenSSH host key rotation and other goodies in a previous episode (http://www.bsdnow.tv/episodes/2015_02_04-from_the_foundation_1), but things have changed a little bit since then djm (http://www.bsdnow.tv/episodes/2013_12_18-cryptocrystalline) says "almost immediately after smugly declaring 'mission accomplished', the bug reports started rolling in." There were some initial complaints from developers about the new options, and a serious bug shortly thereafter After going back to the drawing board, he refactored some of the new code (and API) and added some more regression tests Most importantly, the bigger big fix was described as: "a malicious server (say, "host-a") could advertise the public key of another server (say, "host-b"). Then, when the client subsequently connects back to host-a, instead of answering the connection as usual itself, host-a could proxy the connection to host-b. This would cause the user to connect to host-b when they think they are connecting to host-a, which is a violation of the authentication the host key is supposed to provide." None of this code has been in a formal OpenSSH release just yet, but hopefully it will soon *** PCBSD tries out LibreSSL (https://github.com/pcbsd/pcbsd/commit/6ede13117dcee1272d7a7060b16818506874286e) PCBSD users may soon be seeing a lot less security problems because of two recent changes After switching over to OpenNTPD last week (http://www.bsdnow.tv/episodes/2015_02_11-time_for_a_change), PCBSD decides to give the portable LibreSSL (http://www.bsdnow.tv/episodes/2014_07_30-liberating_ssl) a try too Note that this is only for the packages built from ports, not the base system unfortunately They're not the first ones to do this - OPNsense has been experimenting with replacing OpenSSL in their ports tree for a little while now, and of course all of OpenBSD's ports are built against it A good number of patches (https://github.com/pcbsd/freebsd-ports/commit/2eee669f4d6ab9a641162ecda29b62ab921438eb) are still not committed in vanilla FreeBSD ports, so they had to borrow some from Bugzilla Look forward to Kris wearing a "keep calm and abandon OpenSSL (https://www.openbsdstore.com/cgi-bin/live/ecommerce.pl?site=shop_openbsdeurope_com&state=item&dept_id=01&sub_dept_id=01&product_id=TSHIRTOSSL)" shirt in the near future *** Feedback/Questions Benjamin writes in (http://slexy.org/view/s28nyJ5omV) Mike writes in (http://slexy.org/view/s2wYUmUmh0) Brad writes in (http://slexy.org/view/s2BAKAQvMt) *** Mailing List Gold Debian (https://lists.freebsd.org/pipermail/svn-src-head/2015-February/068405.html) Dejavu (https://lists.freebsd.org/pipermail/freebsd-current/2015-February/054580.html) Package gone missing (http://lists.dragonflybsd.org/pipermail/users/2015-February/207475.html) ***
It's our last episode of 2014, and we'll be chatting with Dan Langille about the upcoming BSDCan conference. We'll find out what's planned and what sorts of presentations they're looking for. As usual, answers to viewer-submitted questions and all the week's news, coming up on BSD Now - the place to B.. SD. This episode was brought to you by Headlines More conference presentation videos (http://2014.asiabsdcon.org/timetable.html.en) Some more of the presentation videos from AsiaBSDCon are appearing online Masanobu Saitoh, Developing CPE Routers Based on NetBSD (https://www.youtube.com/watch?v=ApruZrU5fVs) Reyk Floeter (http://www.bsdnow.tv/episodes/2014_09_03-its_hammer_time), VXLAN and Cloud-based Networking with OpenBSD (https://www.youtube.com/watch?v=ufeEP_hzFN0) Jos Jansen, Adapting OS X to the enterprise (https://www.youtube.com/watch?v=gOPfRQgTjNo) Pierre Pronchery (http://www.bsdnow.tv/episodes/2014_04_01-edgy_bsd_users) & Guillaume Lasmayous, Carve your NetBSD (https://www.youtube.com/watch?v=vh-TjLUj6os) Colin Percival (http://www.bsdnow.tv/episodes/2014_01_22-tendresse_for_ten), Everything you need to know about cryptography in 1 hour (https://www.youtube.com/watch?v=jzY3m5Kv7Y8) (not from AsiaBSDCon) The "bsdconferences" YouTube channel has quite a lot of interesting older BSD talks (https://www.youtube.com/user/bsdconferences/videos?sort=da&view=0&flow=grid) too - you may want to go back and watch them if you haven't already *** OpenBSD PIE enhancements (https://www.marc.info/?l=openbsd-cvs&m=141922027318727&w=2) ASLR (https://en.wikipedia.org/wiki/Address_space_layout_randomization) and PIE (https://en.wikipedia.org/wiki/Position-independent_executable) are great security features that OpenBSD has had enabled by default for a long time, in both the base system and ports, but they have one inherent problem They only work with dynamic libraries and binaries, so if you have any static binaries, they don't get the same treatment For example, the default shells (and many other things in /bin and /sbin) are statically linked In the case of the static ones, you can always predict the memory layout, which is very bad and sort of defeats the whole purpose (https://en.wikipedia.org/wiki/Return-oriented_programming) With this and a few related commits (https://www.marc.info/?l=openbsd-cvs&m=141927571832106&w=2), OpenBSD fixes this by introducing static self-relocation More and more CPU architectures are being tested and getting support too; this isn't just for amd64 and i386 - VAX users can rest easy It'll be available in 5.7 in May, or you can use a -current snapshot (http://www.openbsd.org/faq/faq5.html#BldBinary) if you want to get a slice of the action now *** FreeBSD foundation semi-annual newsletter (https://www.freebsdfoundation.org/press/2014dec-newsletter.html) The FreeBSD foundation publishes a huge newsletter twice a year, detailing their funded projects and some community activities As always, it starts with a letter from the president of the foundation - this time it's about encouraging students and new developers to get involved The article also has a fundraising update with a list of sponsored projects, and they note that the donations meter has changed from dollars to number of donors (since they exceeded the goal already) You can read summaries of all the BSD conferences of 2014 and see a list of upcoming ones next year too There are also sections about the FreeBSD Journal (http://www.bsdnow.tv/episodes/2014_01_29-journaled_news_updates)'s progress, a new staff member and a testimonial from NetApp It's a very long report, so dedicate some time to read all the way through it This year was pretty great for BSD: both the FreeBSD and OpenBSD foundations exceeded their goals and the NetBSD foundation came really close too As we go into 2015, consider donating to whichever (https://www.freebsdfoundation.org/donate) BSD (http://www.openbsdfoundation.org/donations.html) you (https://www.netbsd.org/donations/) use (http://www.dragonflybsd.org/donations/), it really can make a difference *** Modernizing OpenSSH fingerprints (https://www.marc.info/?l=openbsd-cvs&m=141920089614758&w=4) When you connect to a server for the first time, you'll get what's called a fingerprint of the host's public key - this is used to verify that you're actually talking to the same server you intended to Up until now, the key fingerprints have been an MD5 hash, displayed as hex This can be problematic (https://lists.mindrot.org/pipermail/openssh-unix-dev/2014-November/033117.html), especially for larger key types like RSA that give lots of wiggle room for collisions, as an attacker could generate a fake host key that gives the same MD5 string as the one you wanted to connect to This new change replaces the default MD5 and hex with a base64-encoded SHA256 fingerprint You can add a "FingerprintHash" line in your ssh_config to force using only the new type There's also a new option (https://www.marc.info/?l=openbsd-cvs&m=141923470520906&w=2) to require users to authenticate with more than one public key, so you can really lock down login access to your servers - also useful if you're not 100% confident in any single key type The new options should be in the upcoming 6.8 release *** Interview - Dan Langille - info@bsdcan.org (mailto:info@bsdcan.org) / @bsdcan (https://twitter.com/bsdcan) Plans for the BSDCan 2015 conference News Roundup Introducing ntimed, a new NTP daemon (https://github.com/bsdphk/Ntimed) As we've mentioned before in our tutorials (http://www.bsdnow.tv/tutorials/ntpd), there are two main daemons for the Network Time Protocol - ISC's NTPd and OpenBSD's OpenNTPD With all the recent security problems with ISC's NTPd, Poul-Henning Kamp (http://www.bsdnow.tv/episodes/2013_10_16-go_directly_to_jail) has been working on a third NTP daemon It's called "ntimed" and you can try out a preview version of it right now - it's in FreeBSD ports (https://www.freshports.org/net/ntimed/) or on Github PHK also has a few blog entries (http://phk.freebsd.dk/time/) about the project, including status updates *** OpenBSD-maintained projects list (http://mdocml.bsd.lv/openbsd_projects.html) There was recently a read on the misc mailing list (https://www.marc.info/?t=141961588200003&r=1&w=2) asking about different projects started by OpenBSD developers The initial list had marks for which software had portable versions to other operating systems (OpenSSH being the most popular example) A developer compiled a new list from all of the replies to that thread into a nice organized webpage Most people are only familiar with things like OpenSSH, OpenSMTPD, OpenNTPD and more recently LibreSSL, but there are quite a lot more This page also serves as a good history lesson for BSD in general: FreeBSD and others have ported some things over, while a couple OpenBSD tools were born from forks of FreeBSD tools (mergemaster, pkg tools, portscout) *** Monitoring network traffic with FreeBSD (https://forums.freebsd.org/threads/howto-monitor-network-traffic-with-netflow-nfdump-nfsen-on-freebsd.49724/) If you've ever been curious about monitoring network traffic on your FreeBSD boxes, this forum post may be exactly the thing for you It'll show you how to combine the Netflow, NfDump and NfSen suite of tools to get some pretty detailed network stats (and of course put them into a fancy webpage) This is especially useful for finding out what was going on at a certain point in time, for example if you had a traffic spike *** Trapping spammers with spamd (http://www.protoc.org/blog/2014/12/22/trapping-spammers-with-the-openbsd-spam-deferral-daemon) This is a blog post about OpenBSD's spamd (https://en.wikipedia.org/wiki/Spamd) - a spam email deferral daemon - and how to use it for your mail It gives some background on the greylisting approach to spam, rather than just a typical host blacklist "Greylisting is a method of defending e-mail users against spam. A mail transfer agent (MTA) using greylisting will "temporarily reject" any email from a sender it does not recognize. If the sender re-attempts mail delivery at a later time, the sender may be allowed to continue the mail delivery conversation." The post also shows how to combine it with PF and other tools for a pretty fancy mail setup You can find spamd in the OpenBSD base system (http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man8/spamd.8), or use it with FreeBSD (https://www.freshports.org/mail/spamd) or NetBSD (http://pkgsrc.se/mail/spamd) via ports and pkgsrc You might also want to go back and listen to BSDTalk episode 68 (https://archive.org/details/bsdtalk068), where Will talks to Bob Beck about spamd *** Feedback/Questions Sean writes in (http://slexy.org/view/s20rUK9XVJ) Brandon writes in (http://slexy.org/view/s20nfzIuT2) Anders writes in (http://slexy.org/view/s20wCBhFLO) David writes in (http://slexy.org/view/s20xGrBIyl) Kyle writes in (http://slexy.org/view/s2QHRaiZJW) *** Mailing List Gold NTP code comparison (https://www.marc.info/?l=openbsd-tech&m=141903858708123&w=2) - 192870 vs. 2898 (https://www.marc.info/?l=openbsd-tech&m=141905854411370&w=2) NICs have feelings too (https://lists.freebsd.org/pipermail/freebsd-hackers/2014-December/046741.html) Just think about it (https://www.marc.info/?l=openbsd-ports&m=141998130824977&w=2) ***
Coming up this week, we'll be talking with Michael Lucas about his newest BSD book, "FreeBSD Mastery: Storage Essentials." It's got lots of great information about the disk subsystems, GEOM, filesystems, you name it. We've also got the usual round of news and answers to your emails, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines More BSD conference videos (https://www.youtube.com/channel/UCLy8AikPZfWEmzWxUec69PA/videos) We mentioned it a few times, but the "New Directions in Operating Systems" conference was held in November in the UK The presentations videos are now online, with a few BSD-related talks of interest Antti Kantee (http://www.bsdnow.tv/episodes/2013_10_23-a_brief_intorduction), Rump kernels and why / how we got here (https://www.youtube.com/watch?v=GoB73cVyScI) Franco Fichtner, An introduction to userland networking (https://www.youtube.com/watch?v=WiMNuGTRgbA) Robert Watson (http://www.bsdnow.tv/episodes/2014_08_13-vpn_my_dear_watson), New ideas about old OS security (https://www.youtube.com/watch?v=60elN996rtg) Lots of other interesting, but non-BSD-related, talks were also presented, so check the full list (https://www.youtube.com/playlist?list=PLmRrx948XMnEUlzKOCYn3AzT8OAInP_5M) if you're interested in operating systems in general The 2014 AsiaBSDCon videos are also slowly being uploaded (better late than never) Kirk McKusick (http://www.bsdnow.tv/episodes/2013-10-02_stacks_of_cache), An Overview of Security in the FreeBSD Kernel (https://www.youtube.com/watch?v=E04LxKiu79I) Matthew Ahrens (http://www.bsdnow.tv/episodes/2014_05_14-bsdcanned_goods), OpenZFS ensures the continued excellence of ZFS (https://www.youtube.com/watch?v=8T9Rh-46jhI) Eric Allman, Bambi Meets Godzilla: They Elope - Open Source Meets the Commercial World (https://www.youtube.com/watch?v=o2dmreSy76Q) Scott Long (http://www.bsdnow.tv/episodes/2013_12_25-the_gift_of_giving), Modifying the FreeBSD kernel Netflix streaming servers (https://www.youtube.com/watch?v=4sZZN8Szh14) Dru Lavigne (http://www.bsdnow.tv/episodes/2014_04_09-pxe_dust), ZFS for the Masses (https://www.youtube.com/watch?v=z5apZFFvx4k) Kris Moore, Snapshots, Replication, and Boot Environments (https://www.youtube.com/watch?v=w-0PlAVSg5U) David Chisnall (http://www.bsdnow.tv/episodes/2014_05_07-lets_get_raid), The Future of LLVM in the FreeBSD Toolchain (https://www.youtube.com/watch?v=NLqDAclXMMU) Luba Tang, Bold, fast optimizing linker for BSD (https://www.youtube.com/watch?v=fWgbBUPMsVw) John Hixson (http://www.bsdnow.tv/episodes/2014_04_23-its_gonna_get_nasty), Introduction to FreeNAS development (https://www.youtube.com/watch?v=iwF82aep-l8) Zbigniew Bodek, Transparent Superpages for FreeBSD on ARM (https://www.youtube.com/watch?v=2KLXcyLZ_RE) Michael Dexter, Visualizing Unix: Graphing bhyve, ZFS and PF with Graphite (https://www.youtube.com/watch?v=rjNg1eQ7uAk) Peter Grehan (http://www.bsdnow.tv/episodes/2014_01_15-bhyve_mind), Nested Paging in Bhyve (https://www.youtube.com/watch?v=wptkUxJSNMY) Martin Matuška, Deploying FreeBSD systems with Foreman and mfsBSD (https://www.youtube.com/watch?v=nb8jB5x0OX4) James Brown (http://www.bsdnow.tv/episodes/2014_04_16-certified_package_delivery), Analysys of BSD Associate Exam Results (https://www.youtube.com/watch?v=6eKMLuzsTbY) Mindaugas Rasiukevicius, NPF - progress and perspective (https://www.youtube.com/watch?v=cgBh0iC9WhM) Luigi Rizzo, Netmap as a core networking technology (https://www.youtube.com/watch?v=nW8iHgOL9y4) Michael W. Lucas (http://www.bsdnow.tv/episodes/2013_11_06-year_of_the_bsd_desktop), Sudo: You're Doing it Wrong (https://www.youtube.com/watch?v=o0purspHg-o) (not from a BSD conference, but still good) They should make for some great material to watch during the holidays *** OpenBSD vs FreeBSD security features (http://networkfilter.blogspot.com/2014/12/security-openbsd-vs-freebsd.html) From the author of both the OpenBSD and FreeBSD secure gateway articles we've featured in the past comes a new entry about security The article goes through a list of all the security features enabled (and disabled) by default in both FreeBSD and OpenBSD It covers a wide range of topics, including: memory protection, randomization, encryption, privilege separation, Capsicum, securelevels, MAC, Jails and chroots, network stack hardening, firewall features and much more This is definitely one of the most in-depth and complete articles we've seen in a while - the author seems to have done his homework If you're looking to secure any sort of BSD box, this post has some very detailed explanations of different exploit mitigation techniques - be sure to read the whole thing There are also some good comments (http://daemonforums.org/showthread.php?s=16fd0771d929aff294b252924b414f2c&t=8823) on DaemonForums and lobste.rs (https://lobste.rs/s/e3s9xr/security_openbsd_vs_freebsd) that you may want to read *** The password? You changed it, right? (http://bsdly.blogspot.com/2014/12/the-password-you-changed-it-right.html) Peter Hansteen (http://www.bsdnow.tv/episodes/2014_04_30-puffy_firewall) has a new blog post up, detailing some weird SSH bruteforcing he's seen recently He apparently reads his auth logs when he gets bored at an airport This new bruteforcing attempt seems to be targetting D-Link devices, as evidenced by the three usernames the bots try to use More than 700 IPs have tried to get into Peter's BSD boxes using these names in combination with weak passwords Lots more details, including the lists of passwords and IPs, can be found in the full article If you're using a BSD router (http://www.bsdnow.tv/tutorials/openbsd-router), things like this can be easily prevented with PF or fail2ban (and you probably don't have a "d-link" user anyway) *** Get started with FreeBSD, an intro for Linux users (http://www.infoworld.com/article/2858288/unix/intro-to-freebsd-for-linux-users.html) Another new BSD article on a mainstream technology news site - seems we're getting popular This article is written for Linux users who may be considering switching over to BSD and wondering what it's all about It details installing FreeBSD 9.3 and getting a basic system setup, while touching on ports and packages, and explaining some terminology along the way "Among the legions of Linux users and admins, there seems to be a sort of passive curiosity about FreeBSD and other BSDs. Like commuters on a packed train, they gaze out at a less crowded, vaguely mysterious train heading in a slightly different direction and wonder what traveling on that train might be like" ** Interview - Michael W. Lucas - mwlucas@michaelwlucas.com (mailto:mwlucas@michaelwlucas.com) / @mwlauthor (https://twitter.com/mwlauthor) FreeBSD Mastery: Storage Essentials (https://www.tiltedwindmillpress.com/?product=freebsd-mastery-storage-essentials) News Roundup OpenSMTPD status update (https://poolp.org/0xa86e/Some-OpenSMTPD-overview,-part-3) The OpenSMTPD guys (http://www.bsdnow.tv/episodes/2013-09-18_mx_with_ttx), particularly Gilles, have posted an update on what they've been up to lately As of 5.6, it's become the default MTA in OpenBSD, and sendmail will be totally gone in 5.7 Email is a much more tricky protocol than you might imagine, and the post goes through some of the weirdness and problems they've had to deal with There's also another post (https://poolp.org/0xa871/The-state-of-filters) that goes into detail on their upcoming filtering API - a feature many have requested The API is still being developed, but you can test it out now if you know what you're doing - full details in the article OpenSMTPD also has portable versions in FreeBSD ports and NetBSD pkgsrc, so check it out *** OpenCrypto changes in FreeBSD (https://lists.freebsd.org/pipermail/svn-src-head/2014-December/065806.html) A little while back, we talked to John-Mark Gurney (http://www.bsdnow.tv/episodes/2014_10_29-ipsecond_wind) about updating FreeBSD's OpenCrypto framework, specifically for IPSEC Some of that work has just landed in the -CURRENT branch, and the commit has a bit of details The ICM and GCM modes of AES were added, and both include support for AESNI There's a new port - "nist-kat" - that can be used to test the new modes of operation Some things were fixed in the process as well, including an issue that would leak timing info and result in the ability to forge messages Code was also borrowed from both OpenBSD and NetBSD to make this possible *** First thoughts on OpenBSD's httpd (http://www.protoc.org/blog/2014/11/23/first-thoughts-on-the-new-openbsd-httpd-server/) Here we have a blog post from a user of OpenBSD's new homegrown web server that made its debut in 5.6 The author loves that it has proper privilege separation, a very simple config syntax and that it always runs in a chroot He also mentions dynamic content hosting with FastCGI, and provides an example of how to set it up Be sure to check our interview with Reyk (http://www.bsdnow.tv/episodes/2014_09_03-its_hammer_time) about the new httpd if you're curious on how it got started Also, if you're running the version that came with 5.6, there's a huge patch (http://ftp.openbsd.org/pub/OpenBSD/patches/5.6/common/009_httpd.patch.sig) you can apply to get a lot of the features and fixes from -current without waiting for 5.7 *** Steam on PCBSD (https://www.youtube.com/watch?v=B04EuZ9hpAI) One of the most common questions people who want to use BSD as a desktop ask us is "can I run games?" or "can I use steam?" Steam through the Linux emulation layer (in FreeBSD) may be possible soon, but it's already possible to use it with WINE This video shows how to get Steam set up on PCBSD using the Windows version There are also some instructions in the video description to look over A second video (https://www.youtube.com/watch?v=BJ88B8aWdk0) details getting streaming set up *** Feedback/Questions Charlie writes in (http://slexy.org/view/s2JgqXcw4i) Sean writes in (http://slexy.org/view/s2WormjMCs) Predrag writes in (http://slexy.org/view/s20UmdFrbj) ***
This week on the show, we'll be talking with Paul Schenkeveld, chairman of the EuroBSDCon foundation. He tells us about his experiences running BSD conferences and how regular users can get involved too. We've also got answers to all your emails and the latest news, coming up on BSD Now - the place to B.. SD. This episode was brought to you by Headlines More BSD presentation videos (https://www.meetbsd.com/) The MeetBSD video uploading spree continues with a few more talks, maybe this'll be the last batch Corey Vixie, Web Apps in Embedded BSD (https://www.youtube.com/watch?v=Pbks12Mqpp8) Allan Jude, UCL config (https://www.youtube.com/watch?v=TjP86iWsEzQ) Kip Macy, iflib (https://www.youtube.com/watch?v=P4FRPKj7F80) While we're on the topic of conferences, AsiaBSDCon's CFP was extended (https://twitter.com/asiabsdcon/status/538352055245492226) by one week This year's ruBSD (https://events.yandex.ru/events/yagosti/rubsd14/) will be on December 13th in Moscow Also, the BSDCan call for papers (http://lists.bsdcan.org/pipermail/bsdcan-announce/2014-December/000135.html) is out, and the event will be in June next year Lastly, according to Rick Miller, "A potential vBSDcon 2015 event is being explored though a decision has yet to be made." *** BSD-powered digital library in Africa (http://peercorpsglobal.org/nzegas-digital-library-becomes-a-reality/) You probably haven't heard much about Nzega, Tanzania, but it's an East African country without much internet access With physical schoolbooks being a rarity there, a few companies helped out to bring some BSD-powered reading material to a local school They now have a pair of FreeNAS Minis at the center of their local network, with over 80,000 books and accompanying video content stored on them (~5TB of data currently) The school's workstations also got wiped and reloaded with FreeBSD, and everyone there seems to really enjoy using it *** pfSense 2.2 status update (https://blog.pfsense.org/?p=1486) With lots of people asking when the 2.2 release will be done, some pfSense developers decided to provide a status update 2.2 will have a lot of changes: being based on FreeBSD 10.1, Unbound instead of BIND, updating PHP to something recent, including the new(ish) IPSEC stack updates, etc All these things have taken more time than previously expected The post also has some interesting graphs showing the ratio of opened and close bugs for the upcoming release *** Recommended hardware threads (https://www.reddit.com/r/BSD/comments/2n8wrg/bsd_on_mini_itx/) A few threads on caught our attention this week, all about hardware recommendations for BSD setups In the first one, the OP asks about mini-ITX hardware to run a FreeBSD server and NAS Everyone gave some good recommendations for low power, Atom-based systems The second thread (https://www.marc.info/?t=141694918800006&r=1&w=2) started off asking about which CPU architecture is best for PF on an OpenBSD router, but ended up being another hardware thread For a router, the ALIX, APU and Soekris boards still seem to be the most popular choices, with the third (https://www.reddit.com/r/homelab/comments/24m6tj/) and fourth (https://www.reddit.com/r/PFSENSE/comments/2nblgp/) threads confirming this If you're thinking about building your first BSD box - server, router, NAS, whatever - these might be some good links to read *** Interview - Paul Schenkeveld - freebsd@psconsult.nl (mailto:freebsd@psconsult.nl) Running a BSD conference News Roundup From Linux to FreeBSD - for reals (https://www.reddit.com/r/freebsd/comments/2nqa60/) Another Linux user is ready to switch to BSD, and takes to Reddit for some community encouragement (seems to be a common thing now) After being a Linux guy for 20(!) years, he's ready to switch his systems over, and is looking for some helpful guides to transition In the comments, a lot of new switchers offer some advice and reading material If any of the listeners have some things that were helpful along your switching journey, maybe send 'em this guy's way *** Running FreeBSD as a Xen Dom0 (http://wiki.xenproject.org/wiki/FreeBSD_Dom0) Continuing progress has been made to allow FreeBSD to be a host for the Xen hypervisor This wiki article explains how to run the Xen branch of FreeBSD and host virtual machines on it Xen on FreeBSD currently supports PV guests (modified kernels) and HVM (unmodified kernels, uses hardware virtualization features) The wiki provides instructions for running Debian (PV) and FreeBSD (HVM), and discusses the features that are not finished yet *** HardenedBSD updates and changes (http://hardenedbsd.org/article/shawn-webb/2014-11-18/aout-and-null-mapping-support-removal) a.out is the old executable format for Unix The name stands for assembler output, and was coined by Ken Thompson as the fixed name for output of his PDP-7 assembler in 1968 FreeBSD, on which HardenedBSD is based, switched away from a.out in version 3.0 A restriction against NULL mapping was introduced in FreeBSD 7 (https://www.freebsd.org/security/advisories/FreeBSD-EN-09:05.null.asc) and enabled by default in FreeBSD 8 However, for reasons of compatibility, it could be switched off, allowing buggy applications to continue to run, at the risk of allowing a kernel bug to be exploited HardenedBSD has removed the sysctl, making it impossible to run in ‘insecure mode' Package building update: more consistent repo, no more i386 packages (http://hardenedbsd.org/article/shawn-webb/2014-11-30/package-building-infrastructure-maintenance) *** Feedback/Questions Boris writes in (http://slexy.org/view/s2kVPKICqj) Alex writes in (http://slexy.org/view/s21Fic4dZC) (edit: adding "tinker panic 0" to the ntp.conf will disable the sanity check) Chris writes in (http://slexy.org/view/s2zk1Tvfe9) Robert writes in (http://slexy.org/view/s22alvJ4mu) Jake writes in (http://slexy.org/view/s203YMc2zL) *** Mailing List Gold Real world authpf use (https://www.marc.info/?t=141711266800001&r=1&w=2) The (https://svnweb.freebsd.org/ports/head/UPDATING?r1=373564&r2=373563&pathrev=373564) great (https://lists.freebsd.org/pipermail/freebsd-ports/2014-November/096788.html) perl (https://lists.freebsd.org/pipermail/freebsd-ports/2014-November/096799.html) event (https://lists.freebsd.org/pipermail/freebsd-perl/2014-November/010146.html) of (https://lists.freebsd.org/pipermail/freebsd-perl/2014-November/010149.html) 2014 (https://lists.freebsd.org/pipermail/freebsd-perl/2014-November/010167.html) ***
This week on the show we'll be talking with Hiroki Sato about the status of BSD in Japan. We also get to hear about how he got on the core team, and we just might find out why NetBSD is so popular over there! Answers to all your emails, the latest news, and even a brand new segment, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines BSD talks at XDC 2014 (https://www.youtube.com/channel/UCXlH5v1PkEhjzLFTUTm_U7g/videos) This year's Xorg conference featured a few BSD-related talks Matthieu Herrb, Status of the OpenBSD graphics stack (https://www.youtube.com/watch?v=KopgD4nTtnA) Matthieu's talk details what's been done recently in Xenocara the OpenBSD kernel for graphics (slides here (http://www.openbsd.org/papers/xdc2014-xenocara.pdf)) Jean-Sébastien Pédron, The status of the graphics stack on FreeBSD (https://www.youtube.com/watch?v=POmxFleN3Bc) His presentation gives a history of major changes and outlines the current overall status of graphics in FreeBSD (slides here (http://www.x.org/wiki/Events/XDC2014/XDC2014PedronFreeBSD/XDC-2014_FreeBSD.pdf)) Francois Tigeot, Porting DRM/KMS drivers to DragonFlyBSD (https://www.youtube.com/watch?v=NdM7_yPGFDk) Francois' talk tells the story of how he ported some of the DRM and KMS kernel drivers to DragonFly (slides here (http://www.x.org/wiki/Events/XDC2014/XDC2014TigeotDragonFlyBSD/XDC-2014_Porting_kms_drivers_to_DragonFly.pdf)) *** FreeBSD Quarterly Status Report (https://www.freebsd.org/news/status/report-2014-07-2014-09.html) The FreeBSD project has a report of their activities between July and September of this year Lots of ARM work has been done, and a goal for 11.0 is tier one support for the platform The release includes reports from the cluster admin team, release team, ports team, core team and much more, but we've already covered most of the items on the show If you're interested in seeing what the FreeBSD community has been up to lately, check the full report - it's huge *** Monitoring pfSense logs using ELK (http://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-elk-logstash-kibana-elasticsearch/) If you're one of those people who loves the cool graphs and charts that pfSense can produce, this is the post for you ELK (ElasticSearch, Logstash, Kibana) is a group of tools that let you collect, store, search and (most importantly) visualize logs It works with lots of different things that output logs and can be sent to one central server for displaying This post shows you how to set up pfSense to do remote logging to ELK and get some pretty awesome graphs *** Some updates to IPFW (https://svnweb.freebsd.org/base?view=revision&revision=272840) Even though PF gets a lot of attention, a lot of FreeBSD people still love IPFW While mostly a dormant section of the source tree, some updates were recently committed to -CURRENT The commit lists the user-visible changes, performance changes, ABI changes and internal changes It should be merged back to -STABLE after a month or so of testing, and will probably end up in 10.2-RELEASE Also check this blog post (http://blog.cochard.me/2014/10/ipfw-improvement-on-freebsd-current.html) for some more information and fancy graphs *** Interview - Hiroki Sato (佐藤広生) - hrs@freebsd.org (mailto:hrs@freebsd.org) / @hiroki_sato (https://twitter.com/hiroki_sato) BSD in Japan, technology conferences, various topics News Roundup pfSense on Hyper-V (https://virtual-ops.de/?p=600) In case you didn't know, the latest pfSense snapshots support running on Hyper-V Unfortunately, the current stable release is based on an old, unsupported FreeBSD 8.x base, so you have to use the snapshots for now The author of the post tells about his experience running pfSense and gives lots of links to read if you're interested in doing the same He also praises pfSense above other Linux-based solutions for its IPv6 support and high quality code *** OpenBSD as a daily driver (https://www.reddit.com/r/openbsd/comments/2isz24/openbsd_as_a_daily_driver/) A curious Reddit user posts to ask the community about using OpenBSD as an everyday desktop OS The overall consensus is that it works great for that, stays out of your way and is quite reliable Caveats would include there being no Adobe Flash support (though others consider this a blessing..) and it requiring a more hands-on approach to updating If you're considering running OpenBSD as a "daily driver," check all the comments for more information and tips *** Getting PF log statistics (https://secure.ciscodude.net/2014/10/09/firewall-log-stats/) The author of this post runs an OpenBSD box in front of all his VMs at his colocation, and details his experiences with firewall logs He usually investigates any IPs of interest with whois, nslookup, etc. - but this gets repetitive quickly, so.. He sets out to find the best way to gather firewall log statistics After coming across a perl script (http://www.pantz.org/software/pf/pantzpfblockstats.html) to do this, he edited it a bit and is now a happy, lazy admin once again You can try out his updated PF script here (https://github.com/tbaschak/Pantz-PFlog-Stats) *** FlashRD 1.7 released (http://www.nmedia.net/flashrd/) In case anyone's not familiar, flashrd is a tool to create OpenBSD images for embedded hardware devices, executing from a virtualized environment This new version is based on (the currently unreleased) OpenBSD 5.6, and automatically adapts to the number of CPUs you have for building It also includes fixes for 4k drives and lots of various other improvements If you're interested in learning more, take a look at some of the slides and audio from the main developer on the website *** Feedback/Questions Antonio writes in (http://slexy.org/view/s20XvSa4h0) Don writes in (http://slexy.org/view/s20lGUXW3d) Andriy writes in (http://slexy.org/view/s2al5DFIO7) Richard writes in (http://slexy.org/view/s203QoFuWs) Robert writes in (http://slexy.org/view/s29WIplL6k) *** Mailing List Gold Subtle trolling (https://marc.info/?l=openbsd-cvs&m=141271076115386&w=2) Old bugs with old fixes (https://marc.info/?l=openbsd-cvs&m=141275713329601&w=2) A pig reinstall (https://lists.freebsd.org/pipermail/freebsd-ports/2014-October/095906.html) Strange DOS-like environment (https://lists.freebsd.org/pipermail/freebsd-doc/2014-October/024408.html) ***
Coming up this week on the show, we've got something special for you! We'll be giving you an in-depth look at all of the graphical PC-BSD utilities. That's right, BSD doesn't have to be commandline-only anymore! There's also the usual round of answers to your emails and all the latest headlines, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines FreeBSD foundation semi-annual newsletter (https://www.freebsdfoundation.org/press/2014jul-newsletter) The FreeBSD foundation published their semi-annual newsletter, complete with a letter from the president of the foundation "In fact after reading [the president's] letter, I was motivated to come up with my own elevator pitch instead of the usual FreeBSD is like Linux, only better!" It talks about the FreeBSD journal (http://www.bsdnow.tv/episodes/2014_01_29-journaled_news_updates) as being one of the most exciting things they've launched this year, conferences they funded and various bits of sponsored code that went into -CURRENT The full list of funded projects is included, also with details in the financial reports There are also a number of conference wrap-ups: NYCBSDCon, BSDCan, AsiaBSDCon and details about the upcoming EuroBSDCon
This week on the show we'll be showing you how to set up RAID arrays in both FreeBSD and OpenBSD. There's also an interview with David Chisnall - of the FreeBSD core team - about the switch to Clang and a lot more. As usual, we'll be dropping the latest news and answering your emails, so sit back and enjoy some BSD Now - the place to B.. SD. This episode was brought to you by Headlines OpenBSD 5.5 released (http://www.openbsd.org/55.html) If you ordered (https://https.openbsd.org/cgi-bin/order) a CD set (https://twitter.com/blakkheim/status/461909893813784576) then you've probably had it for a little while already, but OpenBSD has formally announced the public release (http://undeadly.org/cgi?action=article&sid=20140501153339) of 5.5 This is one of the biggest releases to date, with a very long list of changes and improvements Some of the highlights include: time_t being 64 bit on all platforms, release sets and binary packages being signed with the new signify tool, a new autoinstall feature of the installer, SMP support on Alpha, a new AViiON port, lots of new hardware drivers including newer NICs, the new vxlan driver, relayd improvements, a new pf queue system for bandwidth shaping, dhcpd and dhclient fixes, OpenSMTPD 5.4.2 and all its new features, position-independent executables being default for i386, the RNG has been replaced with ChaCha20 as well as some other security improvements, FUSE support, tmpfs, softraid partitions larger than 2TB and a RAID 5 implementation, OpenSSH 6.6 with all its new features and fixes... and a lot more The full list of changes (http://www.openbsd.org/plus55.html) is HUGE, be sure to read through it all if you're interested in the details If you're doing an upgrade from 5.4 instead of a fresh install, pay careful attention to the upgrade guide (http://www.openbsd.org/faq/upgrade55.html) as there are some very specific steps for this version Also be sure to apply the errata patches (http://www.openbsd.org/errata55.html) on your new installations... especially those OpenSSL ones (some of which still aren't fixed (http://marc.info/?l=oss-security&m=139906348230995&w=2) in the other BSDs yet) On the topic of errata patches, the project is now going to also send them out (signed (http://undeadly.org/cgi?action=article&sid=20140502103355)) via the announce mailing list (http://lists.openbsd.org/cgi-bin/mj_wwwusr?user=&passw=&func=lists-long-full&extra=announce), a very welcome change Congrats to the whole team on this great release - 5.6 is going to be even more awesome with "Libre"SSL and lots of other stuff that's currently in development *** FreeBSD foundation funding highlights (http://freebsdfoundation.blogspot.com/2014/04/freebsd-foundation-spring-fundraising_28.html) The FreeBSD foundation posts a new update on how they're spending the money that everyone donates "As we embark on our 15th year of serving the FreeBSD Project and community, we are proud of what we've done to help FreeBSD become the most innovative, reliable, and high-performance operation system" During this spring, they want to highlight the new UEFI boot support and newcons (http://freebsdfoundation.blogspot.com/2014/05/freebsd-foundation-newcons-project.html) There's a lot of details about what exactly UEFI is and why we need it going forward FreeBSD has also needed some updates to its console to support UTF8 and wide characters Hopefully this series will continue and we'll get to see what other work is being sponsored *** OpenSSH without OpenSSL (http://marc.info/?l=openbsd-cvs&m=139879453001957&w=2) The OpenSSH team has been hard at work, making it even better, and now OpenSSL is completely optional Since it won't have access to the primitives OpenSSL uses, there will be a trade-off of features vs. security This version will drop support for legacy SSH v1, and the only two cryptographic algorithms supported are an in-house implementation of AES in counter mode and the new combination (http://www.openbsd.org/cgi-bin/cvsweb/src/usr.bin/ssh/PROTOCOL.chacha20poly1305?rev=HEAD;content-type=text%2Fplain) of the Chacha20 stream cipher with Poly1305 for packet integrity Key exchange is limited to elliptic curve Diffie-Hellman and the newer Curve25519 KEXs No support for RSA, DSA or ECDSA public keys - only Ed25519 It also includes a new buffer API (http://marc.info/?l=openbsd-cvs&m=139883582313750&w=2) and a set of wrappers to make it compatible with the existing API Believe it or not, this was planned before all the heartbleed craziness Maybe someday soon we'll have a mini-openssh-portable in FreeBSD ports and NetBSD pkgsrc, would be really neat *** BSDMag's April 2014 issue is out (http://bsdmag.org/magazine/1861-free-pascal-on-bsd-april-bsd-issue) The free monthly BSD magazine has got a new issue available for download This time the articles include: pascal on BSD, an introduction to revision control systems and configuration management, deploying NetBSD on AWS EC2, more GIMP tutorials, an AsiaBSDCon 2014 report and a piece about how easily credit cards are stolen online Anyone can contribute to the magazine, just send the editors an email about what you want to write No Linux articles this time around, good *** Interview - David Chisnall - theraven@freebsd.org (mailto:theraven@freebsd.org) The LLVM/Clang switch, FreeBSD's core team, various topics Tutorial RAID in FreeBSD and OpenBSD (http://www.bsdnow.tv/tutorials/raid) News Roundup BSDTalk episode 240 (http://bsdtalk.blogspot.com/2014/04/bsdtalk240-about-time-with-george.html) Our buddy Will Backman has uploaded a new episode of BSDTalk, this time with our other buddy GNN as the guest - mainly to talk about NTP and keeping reliable time Topics include the specific details of crystals used in watches and computers to keep time, how temperature affects the quality, different sources of inaccuracy, some general NTP information, why you might want extremely precise time, different time sources (GPS, satellite, etc), differences in stratum levels, the problem of packet delay and estimating the round trip time, some of the recent NTP amplification attacks, the downsides to using UDP instead of TCP and... much more GNN also talks a little about the Precision Time Protocol (https://en.wikipedia.org/wiki/Precision_Time_Protocol) and how it's different than NTP Two people (http://www.bsdnow.tv/episodes/2014_01_29-journaled_news_updates) we've interviewed (http://www.bsdnow.tv/episodes/2014_03_05-bsd_now_vs_bsdtalk) talking to each other, awesome If you're interested in NTP, be sure to see our tutorial (http://www.bsdnow.tv/tutorials/ntpd) too *** m2k14 trip reports (http://undeadly.org/cgi?action=article&sid=20140502092427) We've got a few more reports from the recent OpenBSD hackathon in Morocco The first one is from Antoine Jacoutot (who is a key GNOME porter and gave us the screenshots for the OpenBSD desktop tutorial (http://www.bsdnow.tv/tutorials/the-desktop-obsd)) "Since I always fail at actually doing whatever I have planned for a hackathon, this time I decided to come to m2k14 unprepared about what I was going to do" He got lots of work done with ports and pushing GNOME-related patches back up to the main project, then worked on fixing ports' compatibility with LibreSSL Speaking of LibreSSL, there's an article (http://undeadly.org/cgi?action=article&sid=20140505062023) all would-be portable version writers should probably read and take into consideration Jasper Adriaanse also writes (http://undeadly.org/cgi?action=article&sid=20140501185019) about what he got done over there He cleaned up and fixed the puppet port to work better with OpenBSD *** Why you should use FreeBSD on your cloud VPS (https://www.atlantic.net/blog/2014/04/08/freebsd-ssd-cloud-vps-hosting-10-reasons/) Here we have a blog post from Atlantic, a VPS and hosting provider, about 10 reasons for using FreeBSD Starts off with a little bit of BSD history for those who are unfamiliar with it and only know Linux and Windows The 10 reasons are: community, stability, collaboration, ease of use, ports, security, ZFS, GEOM, sound and having lots of options The post goes into detail about each of them and why FreeBSD makes a great choice for a VPS OS *** PCBSD weekly digest (http://blog.pcbsd.org/2014/05/weekly-feature-digest-27-software-system-redesign/) Big changes coming in the way PCBSD manages software The PBI system, AppCafe and related tools are all going to use pkgng now The AppCafe will no longer be limited to PBIs, so much more software will be easily available from the ports tree New rating system coming soon and much more *** Feedback/Questions Martin writes in (http://slexy.org/view/s21bk2oPuQ) John writes in (http://slexy.org/view/s2n9fx1Rpw) Alex writes in (http://slexy.org/view/s2rBBKLA4u) Goetz writes in (http://slexy.org/view/s20JY6ZI71) Jarrad writes in (http://slexy.org/view/s20YV5Ohpa) ***
Finally hit 30 episodes! Today we'll be chatting with Warren Block to discuss BSD documentation efforts and future plans. If you've ever wondered about the scary world of mailing lists, today's tutorial will show you the basics of how to get help and contribute back. There's lots to get to today, so sit back and enjoy some BSD Now - the place to B.. SD. This episode was brought to you by Headlines OpenBSD on a Sun T5120 (http://www.tedunangst.com/flak/post/OpenBSD-on-a-Sun-T5120) Our buddy Ted Unangst (http://www.bsdnow.tv/episodes/2014_02_05-time_signatures) got himself a cool Sun box Of course he had to write a post about installing and running OpenBSD on it The post goes through some of the quirks and steps to go through in case you're interested in one of these fine SPARC machines He's also got another post about OpenBSD on a Dell CS24-SC server (http://www.tedunangst.com/flak/post/Dell-CS24-SC-server) *** Bhyvecon 2014 videos are up (https://www.youtube.com/results?search_query=bhyvecon%20tokyo&sm=3) Like we mentioned last week, Bhyvecon (http://bhyvecon.org/) was an almost-impromptu conference before AsiaBSDCon The talks have apparently already been uploaded! Subjects include Bhyve's past, present and future, OSv on Bhyve, a general introduction to the tool, migrating those last few pesky Linux boxes to virtualization Lots more detail in the videos, so check 'em all out *** Building a FreeBSD wireless access point (http://blog.khubla.com/freebsd/building-my-own-wireless-point) We've got a new blog post about creating a wireless access point with FreeBSD After all the recent news of consumer routers being pwned like candy, it's time for people to start building BSD routers (http://www.bsdnow.tv/tutorials/openbsd-router) The author goes through a lot of the process of getting one set up using good ol' FreeBSD Using hostapd, he's able to share his wireless card in hostap mode and offer DHCP to all the clients Plenty of config files and more messy details in the post *** Switching from Synology to FreeNAS (http://www.notquitemainstream.com/2014/03/15/why-im-switching-from-synology-to-freenas/) The author has been considering getting a NAS for quite a while and documents his research He was faced with the compromise of convenience vs. flexibility - prebuilt or DIY After seeing the potential security issues with proprietary NAS devices, and dealing with frustration with trying to get bugs fixed, he makes the right choice The post also goes into some detail about his setup, all the things he needed a NAS to do as well as all the advantages an open source solution would give *** Interview - Warren Block - wblock@freebsd.org (mailto:wblock@freebsd.org) FreeBSD's documentation project, igor, doceng Tutorial The world of BSD mailing lists (http://www.bsdnow.tv/tutorials/mailing-lists) News Roundup HAMMER2 work and notes (http://www.shiningsilence.com/dbsdlog/2014/03/18/13651.html) Matthew Dillon has posted some updated notes about the development of the new HAMMER version The start of a cluster API was committed to the tree There are also links to design document, a freemap design document, a changes list and a todo list *** BSD Breaking Barriers (https://www.youtube.com/watch?v=buo5JlMnGPI) Our friend MWL (http://www.bsdnow.tv/episodes/2013_11_06-year_of_the_bsd_desktop) gave a talk at NYCBSDCon about BSD "breaking barriers" "What makes the BSD operating systems special? Why should you deploy your applications on BSD? Why does the BSD community keep growing, and why do Linux sites like DistroWatch say that BSD is where the interesting development work is happening? We'll cover the not-so-obvious reasons why BSD still stands tall after almost 40 years." He also has another upcoming talk, (or "webcast") called "Beyond Security: Getting to Know OpenBSD's Real Purpose (http://oreillynet.com/pub/e/3059)" "OpenBSD is frequently billed as a high-security operating system. That's true, but security isn't the OpenBSD Project's main goal. This webcast will introduce systems administrators to OpenBSD, explain the project's mission, and discuss the features and benefits." It's on May 27th and will hopefully be recorded *** FreeBSD in a chroot (http://dreamcat4.github.io/finch/) Finch, "FreeBSD running IN a CHroot," is a new project It's a way to extend the functionality of restricted USB-based FreeBSD systems (FreeNAS, etc.) All the details and some interesting use cases are on the github page He really needs to change the project name (https://www.freshports.org/net-im/finch) though *** PCBSD weekly digest (http://blog.pcbsd.org/2014/03/pc-bsd-weekly-feature-digest-22/) Lots of bugfixes for PCBSD coming down the tubes LZ4 compression is now enabled by default on the whole pool The latest 10-STABLE has been imported and builds are going Also the latest GNOME and Cinnamon builds have been imported and much more *** Feedback/Questions Bostjan writes in (http://slexy.org/view/s20SlvTcwd) (IRC suggests md5deep) Don writes in (http://slexy.org/view/s2PeMqXFid) kaltheat writes in (http://slexy.org/view/s21yii6KZe) (We use R0DE Podcast microphones and Logitech C920 HD webcams) Harri writes in (http://slexy.org/view/s21SkX19Cp) ***
We're back from AsiaBSDCon! This week we'll be chatting with Gleb Kurtsou about some a filesystem-level encryption utility called PEFS. After that, we'll give you a step by step guide on how to actually use it. There's also the usual round of your questions and we've got a lot of news to catch up on, so stay tuned to BSD Now - the place to B.. SD. This episode was brought to you by Headlines Using OpenSSH Certificate Authentication (http://neocri.me/documentation/using-ssh-certificate-authentication/) SSH has a not-so-often-talked-about authentication option in addition to passwords and keys: certificates - you can add certificates to any current authentication method you're using They're not really that complex, there just isn't a lot of documentation on how to use them - this post tries to solve that There's the benefit of not needing a knownhosts file or authorizedusers file anymore The post goes into a fair amount of detail about the differences, advantages and implications of using certificates for authentication *** Back to FreeBSD, a new series (http://www.duckland.org/2014/03/back-to-freebsd-aka-day-1#more) Similar to the "FreeBSD Challenge" blog series, one of our listeners will be writing about his switching BACK to FreeBSD journey "So, a long time ago, I had a box which was running FreeBSD 4, running on a Pentium. 14 years later, I have decided to get back into FreeBSD, now at FreeBSD 10" He's starting off with PCBSD since it's easy to get working with dual graphics Should be a fun series to follow! *** OpenBSD's recent experiments in package building (http://undeadly.org/cgi?action=article&sid=20140307130554) If you'll remember back to our poudriere tutorial (http://www.bsdnow.tv/tutorials/poudriere), it lets you build FreeBSD binary packages in bulk - OpenBSD's version is called dpb (http://www.bsdnow.tv/tutorials/dpb) Marc Espie recently got some monster machines in russia to play with to help improve scaling of dpb on high end hardware This article goes through some of his findings and plans for future versions that increase performance We'll be showing a tutorial of dpb on the show in a few weeks *** Securing FreeBSD with 2FA (http://jafdip.com/securing-freebsd-2fa-two-factor-authentication/) So maybe you've set up two-factor authentication with gmail or twitter, but have you done it with your BSD box? This post walks us through the process of locking down an ssh server (http://www.bsdnow.tv/tutorials/ssh-tmux) with 2FA With just a mobile phone and a few extra tools, you can enable two-factor auth on your BSD box and have just that little extra bit of protections *** Interview - Gleb Kurtsou - gleb.kurtsou@gmail.com (mailto:gleb.kurtsou@gmail.com) PEFS (security audit results here (https://defuse.ca/audits/pefs.htm)) Tutorial Filesystem-based encryption with PEFS (http://www.bsdnow.tv/tutorials/pefs) News Roundup BSDCan 2014 registration (https://www.bsdcan.org/2014/registration.php) Registration is finally open! The prices are available along with a full list of presentations Tutorial sessions for various topics as well You have to go *** Big changes for OpenBSD 5.6 (http://undeadly.org/cgi?action=article&sid=20140314080734) Although 5.5 was just frozen and the release process has started, 5.6 is already looking promising OpenBSD has, for a long time, included a heavily-patched version of Apache based on 1.3 They've also imported nginx into base a few years ago, but now have finally removed Apache Sendmail is also no longer the default MTA, OpenSMTPD is the new default (http://undeadly.org/cgi?action=article&sid=20140313052817) Will BIND be removed next? Maybe so (http://marc.info/?l=openbsd-cvs&m=139492163427518&w=2) They've also discontinued the hp300, mvme68k and mvme88k ports *** Getting to know your portmgr lurkers (http://blogs.freebsdish.org/portmgr/2014/03/11/getting-to-know-your-portmgr-lurker-alexy-dokuchaev/) The "getting to know your portmgr" series makes its return This time we get to talk with danfe@ (probably most known for being the nVidia driver maintainer, but he does a lot with ports) How he got into FreeBSD? He "wanted a unix system that I could understand and that would not get bloated as time goes by" Mentions why he's still heavily involved with the project and lots more *** PCBSD weekly digest (http://blog.pcbsd.org/2014/03/pc-bsd-weekly-feature-digest-20/) Work has started to port Pulseaudio to PCBSD 10.0.1 There's a new "pc-mixer" utility being worked on for sound management as well New PBIs, GNOME/Mate updates, Life Preserver fixes and a lot more PCBSD 10.0.1 was released (http://blog.pcbsd.org/2014/03/pc-bsd-weekly-feature-digest-21-pcbsd-10-0-1-released/) too *** Feedback/Questions Alex writes in (http://slexy.org/view/s2QwjHkL2n) Ben writes in (http://slexy.org/view/s2wLGlHF15) Nick writes in (http://slexy.org/view/s21JsgRjMU) Sami writes in (http://slexy.org/view/s2UX4sYdHy) Christopher writes in (http://slexy.org/view/s26z60Qd6z) ***
This week we're at AsiaBSDCon, so it'll be a shorter episode. We've got an interview with Eric Turgeon, founder of the desktop-focused GhostBSD project. Haven't heard of GhostBSD? Well stay tuned then. There's also a really interesting tutorial on how to serially concatenate disks in NetBSD. We'll be back next week with a normal episode. This episode was brought to you by Interview - Eric Turgeon - ericturgeon@ghostbsd.org (mailto:ericturgeon@ghostbsd.org) / @GhostBSD1 (https://twitter.com/GhostBSD1) GhostBSD Tutorial Serially concatenating disks in NetBSD (http://www.bsdnow.tv/tutorials/nbsd-disks) Feedback/Questions Dave writes in (http://slexy.org/view/s2ff5BOdU0) Shane writes in (http://slexy.org/view/s2F6j5fVYH) Rob writes in (http://slexy.org/view/s2GHmy7tuS) Predrag writes in (http://slexy.org/view/s2uM28feQe) ***
We have a packed show for you this week! We'll sit down for an interview with Chris Buechler, from the pfSense project, to learn just how easy it can be to deploy a BSD firewall. We'll also be showing you a walkthrough of the pfSense interface so you can get an idea of just how convenient and powerful it is. Answers to your questions and the latest headlines, here on BSD Now - the place to B.. SD. This episode was brought to you by Headlines EuroBSDCon and AsiaBSDCon (http://2014.eurobsdcon.org/calendar/call-for-papers/) This year, EuroBSDCon will be in September in Sofia, Bulgaria They've got a call for papers up now, so everyone can submit the talks they want to present There will also be a tutorial section of the conference AsiaBSDCon (http://2014.asiabsdcon.org/timetable.html.en) will be next month, in March! All the info about the registration, tutorials, hotels, timetable and location have been posted Check the link for all the details on the talks - if you plan on going to Tokyo next month, hang out with Allan and Kris and lots of BSD developers! *** FreeBSD 10 on Ubiquiti EdgeRouter Lite (http://rtfm.net/FreeBSD/ERL/) The Ubiquiti EdgeRouter Lite is a router that costs less than $100 and has a MIPS CPU This article goes through the process of installing and configuring FreeBSD on it to use as a home router Lots of good pictures of the hardware and specific details needed to get you set up It also includes the scripts to create your own images if you don't want to use the ones rolled by someone else For such a cheap price, might be a really fun weekend project to replace your shitty consumer router Of course if you're more of an OpenBSD guy, you can always see our tutorial (http://www.bsdnow.tv/tutorials/openbsd-router) for that too *** Signed pkgsrc package guide (http://blog.saveosx.org/signed-packages/) We got a request on IRC for more pkgsrc stuff on the show, and a listener provided a nice write-up It shows you how to set up signed packages with pkgsrc, which works on quite a few OSes (not just NetBSD) He goes through the process of signing packages with a public key and how to verify the packages when you install them The author also happens to be an EdgeBSD developer *** Big batch of OpenBSD hackathon reports (http://undeadly.org/cgi?action=article&sid=20140212083627) Five trip reports from the OpenBSD hackathon in New Zealand! In the first one, jmatthew details his work on fiber channel controller drivers, some octeon USB work and ARM fixes for AHCI In the second (http://undeadly.org/cgi?action=article&sid=20140213065843), ketennis gets into his work with running interrupt handlers without holding the kernel lock, some SPARC64 improvements and a few other things In the third (http://undeadly.org/cgi?action=article&sid=20140213173808), jsg updated libdrm and mesa and did various work on xenocara In the fourth (http://undeadly.org/cgi?action=article&sid=20140214070023), dlg came with the intention to improve SMP support, but got distracted and did SCSI stuff instead - but he talks a little bit about the struggle OpenBSD has with SMP and some of the work he's done In the fifth (http://undeadly.org/cgi?action=article&sid=20140214130039), claudio talks about some stuff he did for routing tables and misc. other things *** Interview - Chris Buechler - cmb@pfsense.com (mailto:cmb@pfsense.com) / @cbuechler (https://twitter.com/cbuechler) pfSense Tutorial pfSense walkthrough News Roundup FreeBSD challenge continues (http://www.thelinuxcauldron.com/2014/02/13/freebsd-challenge-day-13-30/) Our buddy from the Linux foundation continues his switching to BSD journey In day 13, he covers some tips for new users, mentions trying things out in a VM first In day 14 (http://www.thelinuxcauldron.com/2014/02/14/freebsd-challenge-day-14-30/), he starts setting up XFCE and X11, feels like he's starting over as a new Linux user learning the ropes again - concludes that ports are the way to go In day 15 (http://www.thelinuxcauldron.com/2014/02/14/freebsd-challenge-day-15-30/), he finishes up his XFCE configuration and details different versions of ports with different names, as well as learns how to apply his first patch In day 16 (http://www.thelinuxcauldron.com/2014/02/17/freebsd-challenge-day-16-30/), he dives into the world of FreeBSD jails (http://www.bsdnow.tv/tutorials/jails)! *** BSD books in 2014 (http://blather.michaelwlucas.com/archives/1962) BSD books are some of the highest quality technical writings available, and MWL has written a good number of them In this post, he details some of his plans for 2014 In includes at least one OpenBSD book, at least one FreeBSD book and... Very strong possibility of Absolute FreeBSD 3rd edition (watch our interview with him (http://www.bsdnow.tv/episodes/2013_11_06-year_of_the_bsd_desktop)) Check the link for all the details *** How to build FreeBSD/EC2 images (http://www.daemonology.net/blog/2014-02-16-FreeBSD-EC2-build.html) Our friend Colin Percival (http://www.bsdnow.tv/episodes/2014_01_22-tendresse_for_ten) details how to build EC2 images in a new blog post Most people just use the images he makes on their instances, but some people will want to make their own from scratch (https://svnweb.freebsd.org/base/user/cperciva/EC2-build/) You build a regular disk image and then turn it into an AMI It requires a couple ports be installed on your system, but the whole process is pretty straightforward *** PCBSD weekly digest (http://blog.pcbsd.org/2014/02/pc-bsd-weekly-feature-digest-17/) This time around we discuss how you can become a developer Kris also details the length of supported releases Expect lots of new features in 10.1 *** Feedback/Questions Sean writes in (http://slexy.org/view/s216xJoCVG) Jake writes in (http://slexy.org/view/s2gLrR3VVf) Niclas writes in (http://slexy.org/view/s21gfG3Iho) Steffan writes in (http://slexy.org/view/s2JNyw5BCn) Antonio writes in (http://slexy.org/view/s2kg3zoRfm) Chris writes in (http://slexy.org/view/s2ZwSIfRjm) ***
This week we'll be talking to Amitai Schlair of the NetBSD foundation about pkgsrc, NetBSD's future plans and much more. After that, if you've ever wondered what all this SSH stuff is about, today's tutorial has got you covered. We'll be showing you the basics of SSH, as well as how to combine it with tmux for persistent sessions. News, feedback and everything else, right here on BSD Now - the place to B.. SD. Headlines Faces of FreeBSD (http://freebsdfoundation.blogspot.com/2013/11/faces-of-freebsd-colin-percival.html) The FreeBSD foundation is publishing articles on different FreeBSD developers This one is about Colin Percival (cperciva@), the ex-security officer Tells the story of how he first found BSD, what he contributed back, how he eventually became the security officer Running series with more to come *** Lots of BSD presentation videos uploaded (http://www.freebsdnews.net/2013/11/14/eurobsdcon-2013-devsummit-video-recordings/) EuroBSDCon 2013 dev summit videos, AsiaBSDCon 2013 videos, MWL's presentation video Most of us never get to see the dev summit talks since they're only for developers AsiaBSDCon 2013 videos also up (https://www.youtube.com/user/bsdconferences) finally List of AsiaBSDCon presentation topics here (http://2013.asiabsdcon.org/papers/index.html) Our buddy Michael W Lucas gave an "OpenBSD for Linux users" talk (http://blather.michaelwlucas.com/archives/1879) at a Michigan Unix Users Group. He says "Among other things, I compare OpenBSD to Richard Stallman and physically assault an audience member. We also talk long long time, memory randomization, PF, BSD license versus GPL, Microsoft and other OpenBSD stuff" Really informative presentation, pretty long, answers some common questions at the end *** Call for Presentations: FOSDEM 2014 and NYCBSDCon 2014 (https://blog.netbsd.org/tnf/entry/call_for_presentations_bsd_devroom) FOSDEM 2014 will take place on 1–2 February, 2014, in Brussels, Belgium Just like in the last years, there will be both a BSD booth and a developer's room The topics of the devroom include all BSD operating systems. Every talk is welcome, from internal hacker discussion to real-world examples and presentations about new and shiny features. If you are in the area or want to go, check the show notes for details NYCBSDCon is also accepting papers (http://undeadly.org/cgi?action=article&sid=20131119053455). It'll be in New York City at the beginning of February 2014 If anyone wants to give a talk at one of these conferences, go ahead and send in your stuff! *** FreeBSD foundation's year-end fundraising campaign (https://lists.freebsd.org/pipermail/freebsd-announce/2013-November/001511.html) The FreeBSD foundation has been supporting the FreeBSD project and community for over 13 years As of today they have raised about half a million dollars, but still have a while to go Donations go towards new features, paying for the server infrastructure, conferences, supporting the community, hiring full-time staff members and promoting FreeBSD at events They are preparing the debut of a new online magazine, the FreeBSD Journal Typically big companies make their huge donations in December, like a couple of anonymous donors that gave around $250,000 each last year Make your donation today (http://www.freebsdfoundation.org/donate/) over at freebsdfoundation.org, every little bit helps Everyone involved with BSD Now made a donation last year and will do so again this year *** Interview - Amitai Schlair - schmonz@netbsd.org (mailto:schmonz@netbsd.org) / @schmonz (https://twitter.com/schmonz) The NetBSD Foundation, pkgsrc, future plans Tutorial Combining SSH and tmux (http://www.bsdnow.tv/tutorials/ssh-tmux) Note: there was a mistake in the video version of the tutorial, please consult the written version for the proper instructions. *** News Roundup PS4 released (http://www.theregister.co.uk/2013/11/16/sony_playstation_4_kernel) Sony's Playstation 4 is finally released As previously thought, its OS is heavily based on FreeBSD and uses the kernel among other things Link in the show notes contains the full list of BSD software they're using (http://www.scei.co.jp/ps4-license/) Always good to see BSD being so widespread *** BSD Mag November issue (http://bsdmag.org/magazine/1853-hast-on-freebsd-how-to-make-storage-highly-availble-by-using-hast) Free monthly BSD magazine publishes another issue This time their topics include: Configuring a Highly Available Service on FreeBSD, IT Inventory & Asset Management Automation, more FreeBSD Programming Primer, PfSense and Snort and a few others PDF linked in the show notes *** pbulk builds made easy (http://mail-index.netbsd.org/pkgsrc-users/2013/11/09/msg018881.html) NetBSD's pbulk tool (https://www.netbsd.org/docs/pkgsrc/bulk.html) is similar to poudriere (http://www.bsdnow.tv/tutorials/poudriere), but for pkgsrc While working on updating the documentation, a developer cleaned up quite a lot of code He wrote a script that automates pbulk deployment and setup The whole setup of a dedicated machine has been reduced to just three commands *** PCBSD weekly digest (http://blog.pcbsd.org/2013/11/pc-bsd-weekly-feature-digest-111513/) Over 200 PBIs have been populated in to the PC-BSD 10 Stable Appcafe Many PC-BSD programs received some necessary bug fixes and updates Some include network detection in the package and update managers, nvidia graphic detection, security updates for PCDM *** Feedback/Questions Peter writes in (http://slexy.org/view/s21oh3vP7t) Kjell-Aleksander writes in (http://slexy.org/view/s21zfqcWMP) Jordan writes in (http://slexy.org/view/s2ZmW77Odb) Christian writes in (http://slexy.org/view/s2BZq7xiyo) entransic writes in (http://slexy.org/view/s21xrk0M4k) ***
On this week's show, you'll be getting the full jail treatment. We'll show you how to create and deploy BSD jails, as well as chatting with Poul-Henning Kamp - the guy who actually invented them! There's lots of interesting news items to cover as well, so stay tuned to BSD Now - the place to B.. SD. Headlines FreeBSD turns it up to 11 (https://lists.freebsd.org/pipermail/svn-src-head/2013-October/052141.html) The -CURRENT branch is now known as 11 10 has been branched to -STABLE 10-BETA1 ISOs are available now Will be the next -RELEASE, probably next year *** Stopping the SSH bruteforce with BSD and pf (http://bsdly.blogspot.com/2013/10/the-hail-mary-cloud-and-lessons-learned.html) The Hail Mary Cloud is an SSH bruteforce botnet that takes a different approach While most botnets pound port 22 rapidly, THMB does it very slowly and passively This makes prevention based on rate limiting more involved and complex Nice long blog post about some potential solutions (http://home.nuug.no/~peter/pf/en/bruteforce.html) and what we've learned *** ZFS and GELI in bsdinstall coming soon (http://freshbsd.org/commit/freebsd/r256343/) The man with the beard (http://www.bsdnow.tv/episodes/2013-09-25_teskeing_the_possibilities) strikes again, new patch allows for ZFS-on-root installs Supports GELI for disk encryption Might be the push we need to make Michael W Lucas update (https://twitter.com/mwlauthor/status/389524644983611392) his FreeBSD book *** AsiaBSDCon 2014 announced (http://2014.asiabsdcon.org/) Will be held in Tokyo, 13-16 March, 2014 The conference is for anyone developing, deploying and using systems based on FreeBSD, NetBSD, OpenBSD, DragonFlyBSD, Darwin and Mac OS X Call for papers can be found here (http://2014.asiabsdcon.org/cfp.html) *** Interview - Poul-Henning Kamp - phk@freebsd.org (mailto:phk@freebsd.org) / @bsdphk (https://twitter.com/bsdphk) FreeBSD beginnings, md5crypt, jails, varnish and his... telescope project? Tutorial Everything you need to know about Jails (http://www.bsdnow.tv/tutorials/jails) News Roundup New pf queue system (http://freshbsd.org/commit/openbsd/c4661054d1882026efca919b6cd9cf958a8698b4) Henning Brauer committed the new kernel-side bandwidth shaping subsystem Uses the HFSC algorithm behind the scenes ALTQ to be retired "in a release or two" - everyone should migrate soon *** Dragonfly imports FreeBSD KMS driver (http://lists.dragonflybsd.org/pipermail/commits/2013-October/198282.html) Hot on the trails of OpenBSD and later FreeBSD, Dragonfly gets AMD KMS Ported over from the FreeBSD port *** Get paid to hack OpenSSH (https://lists.mindrot.org/pipermail/openssh-unix-dev/2013-October/031695.html) Google has announced they will pay up to $3113.70 for security patches to OpenSSH Patches can fix security or improve security If you come up with something, send it to the OpenSSH guys *** Feedback/Questions Darren writes in (http://slexy.org/view/s24RmwvEvE) Kjell-Aleksander writes in (http://slexy.org/view/s2wFcFk9Yz) Ryan writes in (http://slexy.org/view/s23e920gNG) Alexander writes in (http://slexy.org/view/s2usxPqO9k) ***
We kick off the first episode with the latest BSD news, show you how to avoid intrusion detection systems and talk to Peter Hessler about BGP spam blacklists! Headlines Radeon KMS commited (https://lists.freebsd.org/pipermail/svn-src-head/2013-August/050931.html) Committed by Jean-Sebastien Pedron Brings kernel mode setting to -CURRENT, will be in 10.0-RELEASE (ETA 12/2013) 10-STABLE is expected to be branched in October, to begin the process of stabilizing development Initial testing shows it works well May be merged to 9.X, but due to changes to the VM subsystem this will require a lot of work, and is currently not a priority for the Radeon KMS developer Still suffers from the syscons / KMS switcher issues, same as Intel video More info: https://wiki.freebsd.org/AMD_GPU *** VeriSign Embraces FreeBSD (http://www.eweek.com/enterprise-apps/verisign-embraces-open-source-freebsd-for-diversity/) "BSD is quite literally at the very core foundation of what makes the Internet work" Using BSD and Linux together provides reliability and diversity Verisign gives back to the community, runs vBSDCon "You get comfortable with something because it works well for your particular purposes and can find a good community that you can interact with. That all rang true for us with FreeBSD." *** fetch/libfetch get a makeover (http://freshbsd.org/commit/freebsd/r253680) Adds support for SSL certificate verification Requires root ca bundle (security/rootcanss) Still missing TLS SNI support (Server Name Indication, allows name based virtual hosts over SSL) *** FreeBSD Foundation Semi-Annual Newsletter (http://www.freebsdfoundation.org/press/2013Jul-newsletter) The FreeBSD Foundation took the 20th anniversary of FreeBSD as an opportunity to look at where the project is, and where it might want to go The foundation sets out some basic goals that the project should strive towards: Unify User Experience “ensure that knowledge gained mastering one task translates to the next” “if we do pay attention to consistency, not only will FreeBSD be easier to use, it will be easier to learn” Design for Human and Programmatic Use 200 machines used to be considered a large deployment, with high density servers, blades, virtualization and the cloud, that is not so anymore “the tools we provide for status reporting, configuration, and control of FreeBSD just do not scale or fail to provide the desired user experience” “The FreeBSD of tomorrow needs to give programmability and human interaction equal weighting as requirements” Embrace New Ways to Document FreeBSD More ‘Getting Started' sections in documentation Link to external How-Tos and other documentation “upgrade the cross-referencing and search tools built into FreeBSD, so FreeBSD, not an Internet search engine, is the best place to learn about FreeBSD” Spring Fundraising Campaign, April 17 - May 31, raised a total of $219,806 from 12 organizations and 365 individual donors. In the same period last year we raised a total of $23,422 from 2 organizations and 53 individuals Funds donated to the FreeBSD Foundation have been used on these projects recently: Capsicum security-component framework Transparent superpages support of the FreeBSD/ARM architecture Expanded and faster IPv6 Native in-kernel iSCSI stack Five New TCP Congestion Control Algorithms Direct mapped I/O to avoid extra memory copies Unified Extensible Firmware Interface (UEFI) boot environment Porting FreeBSD to the Genesi Efika MX SmartBook laptop (ARM-based) NAND Flash filesystem and storage stack Funds were also used to sponsor a number of BSD focused conferences: BSDCan, EuroBSDCon, AsiaBSDCon, BSDDay, NYCBSDCon, vBSDCon, plus Vendor summits and Developer summits It is important that the foundation receive donations from individuals, to maintain their tax exempt status in the USA. Even a donation of $5 helps make it clear that the FreeBSD Foundation is backed by a large community, not only a few vendors Donate Today (http://www.freebsdfoundation.org/donate) *** The place to B...SD Ohio Linuxfest, Sept. 13-15, 2013 (http://ohiolinux.org/schedule) Very BSD friendly Kirk McKusick giving the keynote BSD Certification on the 15th, all other stuff on the 14th Multiple BSD talks *** LinuxCon, Sept. 16-18, 2013 (http://events.linuxfoundation.org/events/linuxcon-north-america) Dru Lavigne and Kris Moore will be manning a FreeBSD booth Number of talks of interest to BSD users, including ZFS coop (http://linuxconcloudopenna2013.sched.org/event/b50b23f3ed3bd728fa0052b54021a2cc?iframe=yes&w=900&sidebar=yes&bg=no) EuroBSDCon, Sept. 26-29, 2013 (http://2013.eurobsdcon.org/eurobsdcon-2013/talks/) Tutorials on the 26 & 27th (plus private FreeBSD DevSummit) 43 talks spread over 3 tracks on the 28 & 29th Keynote by Theo de Raadt Hosted in the picturesque St. Julians Area, Malta (Hilton Conference Centre) *** Interview - Peter Hessler - phessler@openbsd.org (mailto:phessler@openbsd.org) / @phessler (https://twitter.com/phessler) Using BGP to distribute spam blacklists and whitelists Tutorial Using stunnel to hide your traffic from Deep Packet Inspection (http://www.bsdnow.tv/tutorials/stunnel) News Roundup NetBSD 6.1.1 released (https://blog.netbsd.org/tnf/entry/netbsd_6_1_1_released) First security/bug fix update of the NetBSD 6.1 release branch Fixes 4 security vulnerabilities Adds 4 new sysctls to avoid IPv6 DoS attacks Misc. other updates *** Sudo Mastery (http://blather.michaelwlucas.com/archives/1792) MWL is a well-known author of many BSD books Also does SSH, networking, DNSSEC, etc. Next book is about sudo, which comes from OpenBSD (did you know that?) Available for preorder now at a discounted price *** Documentation Infrastructure Enhancements (http://freebsdfoundation.blogspot.com/2013/08/new-funded-project-documentation.html) Gábor Kövesdán has completed a funded project to improve the infrastructure behind the documentation project Will upgrade documentation from DocBook 4.2 to DocBook 4.5 and at the same time migrate to proper XML tools. DSSSL is an old and dead standard, which will not evolve any more. DocBook 5.0 tree added *** FreeBSD FIBs get new features (https://svnweb.freebsd.org/base?view=revision&revision=254943) FIBs (as discussed earlier in the interview) are Forward Information Bases (technical term for a routing table) The FreeBSD kernel can be compiled to allow you to maintain multiple FIBs, creating separate routing tables for different processes or jails In r254943 ps(1) is extended to support a new column ‘fib', to display which routing table a process is using *** FreeNAS 9.1.0 and 9.1.1 released (http://www.ixsystems.com/resources/ix/news/ixsystems-announces-revolutionary-freenas-910-release.html) Many improvements in nearly all areas, big upgrade Based on FreeBSD 9-STABLE, lots of new ZFS features Cherry picked some features from 10-CURRENT New volume manager and easy to use plugin management system 9.1.1 released shortly thereafter to fix a few UI and plugin bugs *** BSD licensed "patch" becomes default (http://freshbsd.org/commit/freebsd/r253689) bsdpatch has become mature, does what GNU patch can do, but has a much better license Approved by portmgr@ for use in ports Added WITHGNUPATCH build option for people who still need it ***
© 2020-2025 Ivy Podcast Discovery LLC
