Podcasts about bsd now

2^18 dollars to open source, EuroBSDCon 2023 Trip Report, FreeBSD vs Linux (Debian), Introduction to sysclean8, Run your own Syncthing discovery server on OpenBSD, FreeBSD years: 2000-2005, Using OpenBSD relayd(8) as an Application Layer Gateway, and more NOTES This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow) Headlines 2^18 dollars to open source (https://www.daemonology.net/blog/2023-10-25-2%5E18-dollars-to-open-source.html) Special Thanks to Colin for supporting BSD Now for over 10 years! *** ### EuroBSDCon 2023 Trip Report – Bojan Novković (https://freebsdfoundation.org/blog/eurobsdcon-2023-trip-report-bojan-novkovic/) *** ### FreeBSD vs Linux (Debian) (https://markmcb.com/freebsd/vs_linux/) *** News Roundup Introduction to sysclean8 (https://www.osnews.com/story/137266/introduction-to-sysclean8-on-openbsd/) Run your own Syncthing discovery server on OpenBSD (https://dataswamp.org/~solene/2023-10-18-syncthing-discovery-server.html) My FreeBSD years: 2000-2005 (https://www.cambus.net/my-freebsd-years-2000-2005/) Using OpenBSD relayd(8) as an Application Layer Gateway (https://www.tumfatig.net/2023/using-openbsd-relayd8-as-an-application-layer-gateway/) Beastie Bits How to send syslog messages using command-line utilities (https://sleeplessbeastie.eu/2023/09/11/how-to-send-syslog-messages-using-command-line-utilities/) A Practical Guide of GNU grep With Examples (https://thevaluable.dev/grep-cli-guide-examples/) FreeBSD Container VM for Podman (https://github.com/davidchisnall/container-vm-scripts) User Certbot to create SSL certificates on FreeBSD (https://www.sharpwriting.net/project/use-certbot-to-create-ssl-certificates-on-freebsd/) OpenBSD's built-in memory leak detection (https://www.undeadly.org/cgi?action=article;sid=20231024064619) OpenBSD Webzine Issue #15 (https://webzine.puffy.cafe/issue-15.html) FreeBSD OpenSSL 3.0 ported (https://github.com/FreeBSD/freebsd-ports/commit/d5ec2e12f399b7813994564b77a0915821a0ac42) Harden FreeBSD Script (https://github.com/wravoc/harden-freebsd) Something odd happened... (https://mastodon.bsd.cafe/@stefano/111257154132788711) Tarsnap This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)

table td.shrink { white-space:nowrap } New hosts There were no new hosts this month. Last Month's Shows Id Day Date Title Host 3674 Thu 2022-09-01 Emergency Show posted in 2012. MUD Klaatu 3675 Fri 2022-09-02 Plan 9: An exercise in futility binrc 3676 Mon 2022-09-05 HPR Community News for August 2022 HPR Volunteers 3677 Tue 2022-09-06 Hacker Public Radio 2021 - 2022 New Years Show Part 4 Honkeymagoo 3678 Wed 2022-09-07 "Stupid Users" ... no, not those users, the other "stupid users" Lurking Prion 3679 Thu 2022-09-08 Linux Inlaws S01E64: Non-profits in the US: A closer look at 501(c)s monochromec 3680 Fri 2022-09-09 EDIT Ahuka 3681 Mon 2022-09-12 Rust 101 Episode 3: Functionally Insane BlacKernel 3682 Tue 2022-09-13 Hacker Public Radio 2021 - 2022 New Years Show Part 5 Honkeymagoo 3683 Wed 2022-09-14 Add a favourite to OSMAnd Ken Fallon 3684 Thu 2022-09-15 Wake on Lan JWP 3685 Fri 2022-09-16 Budget and an Android app Archer72 3686 Mon 2022-09-19 Followup for HPR3675: Clarifications on the path traversal bug binrc 3687 Tue 2022-09-20 Hacker Public Radio 2021 - 2022 New Years Show Part 6 Honkeymagoo 3688 Wed 2022-09-21 Education, Certifications, and sipping on the Socials Lurking Prion 3689 Thu 2022-09-22 Linux Inlaws S01E65: TerminusDB monochromec 3690 Fri 2022-09-23 Planning the Trip Ahuka 3691 Mon 2022-09-26 Starship.rs the best prompt I don't use Klaatu 3692 Tue 2022-09-27 What is a real hacker? Lurking Prion 3693 Wed 2022-09-28 Fixing the automatic cutoff mechanism to an electric mower Rho`n 3694 Thu 2022-09-29 Robo Tripping Ravelords of the Apocalypse Mechatroniac 3695 Fri 2022-09-30 How I watch youtube with newsboat binrc Comments this month These are comments which have been made during the past month, either to shows released during the month or to past shows. There are 39 comments in total. Past shows There are 8 comments on 8 previous shows: hpr2449 (2017-12-21) "Org-mode mobile solution" by Brian in Ohio. Comment 2: Sinza on 2022-09-06: "Pinephone" hpr2756 (2019-02-25) "Bash Tips - 20" by Dave Morriss. Comment 1: Unnamed on 2022-09-30: "Untitled" hpr3655 (2022-08-05) "BSD for Linux users" by binrc. Comment 5: Ken Fallon on 2022-09-08: "Featured on BSD Now 471" hpr3658 (2022-08-10) "Linux Inlaws S01E62: HPR's inner workings" by monochromec. Comment 3: operat0r on 2022-09-05: "awesome" hpr3662 (2022-08-16) "Hacker Public Radio 2021 - 2022 New Years Show Part 1" by Honkeymagoo. Comment 1: Ken Fallon on 2022-09-02: "Thanks To:" hpr3667 (2022-08-23) "Hacker Public Radio 2021 - 2022 New Years Show Part 2" by Honkeymagoo. Comment 1: Ken Fallon on 2022-09-02: "Thanks To:" hpr3672 (2022-08-30) "Hacker Public Radio 2021 - 2022 New Years Show Part 3" by Honkeymagoo. Comment 1: Ken Fallon on 2022-09-02: "Thanks To:" hpr3673 (2022-08-31) "Recording for Hacker Public Radio" by dnt. Comment 1: Carl on 2022-09-05: "Great Tips!" This month's shows There are 31 comments on 9 of this month's shows: hpr3675 (2022-09-02) "Plan 9: An exercise in futility" by binrc. Comment 1: one_of_spoons on 2022-09-02: "Directive gem."Comment 2: ClaudioM on 2022-09-02: "plan9 / 9p"Comment 3: norrist on 2022-09-02: "In-Depth Series: Learning Awk"Comment 4: ken on 2022-09-03: "Editors note not Ken Fallon"Comment 5: passerby on 2022-09-03: "fake news"Comment 6: will on 2022-09-03: "come back to plan9" hpr3676 (2022-09-05) "HPR Community News for August 2022" by HPR Volunteers. Comment 1: Archer72 on 2022-09-04: "Good show - I made it to the end"Comment 2: ClaudioM on 2022-09-05: "Another One Made It to the End!"Comment 3: Mike Ray on 2022-09-05: "A11y and abbreviations"Comment 4: operat0r on 2022-09-06: "I made it !"Comment 5: folky on 2022-09-06: "Did make it too"Comment 6: brian-in-ohio on 2022-09-06: "the show"Comment 7: norrist on 2022-09-06: "Public access to HPR site Generator"Comment 8: Miguel on 2022-09-06: "I made it!!"Comment 9: Stache_AF on 2022-09-07: "Made It" hpr3677 (2022-09-06) "Hacker Public Radio 2021 - 2022 New Years Show Part 4" by Honkeymagoo. Comment 1: Ken Fallon on 2022-09-02: "Thanks To:"Comment 2: Filly Buster on 2022-09-06: "Filibuster" hpr3678 (2022-09-07) ""Stupid Users" ... no, not those users, the other "stupid users"" by Lurking Prion. Comment 1: Some Guy On The Internet on 2022-09-08: "Stupid="NO BACKUPS!""Comment 2: Lurking Prion on 2022-09-09: "No Backups"Comment 3: Lurking Prion on 2022-09-09: "Not a podcast"Comment 4: Some Guy On The Internet on 2022-09-10: "You're Fine (...preemptive strike)." hpr3682 (2022-09-13) "Hacker Public Radio 2021 - 2022 New Years Show Part 5" by Honkeymagoo. Comment 1: Ken Fallon on 2022-09-02: "Thanks To:" hpr3687 (2022-09-20) "Hacker Public Radio 2021 - 2022 New Years Show Part 6" by Honkeymagoo. Comment 1: Ken Fallon on 2022-09-02: "Thanks To:" hpr3692 (2022-09-27) "What is a real hacker?" by Lurking Prion. Comment 1: brian-in-ohio on 2022-09-27: "cussing"Comment 2: Lurking Prion on 2022-09-28: "Yes, I probably need a hug" hpr3694 (2022-09-29) "Robo Tripping Ravelords of the Apocalypse" by Mechatroniac. Comment 1: one_of_spoons on 2022-09-29: "{inspirational artifice}"Comment 2: ClaudioM on 2022-09-29: "Great Story"Comment 3: Ken Fallon on 2022-09-29: "I loved this"Comment 4: Mechatroniac on 2022-09-29: "Thanks"Comment 5: brian-in-ohio on 2022-09-30: "the show" hpr3695 (2022-09-30) "How I watch youtube with newsboat" by binrc. Comment 1: folky on 2022-09-30: "Great for gpodder too" Mailing List discussions Policy decisions surrounding HPR are taken by the community as a whole. This discussion takes place on the Mail List which is open to all HPR listeners and contributors. The discussions are open and available on the HPR server under Mailman. The threaded discussions this month can be found here: https://hackerpublicradio.org/pipermail/hpr_hackerpublicradio.org/2022-September/thread.html Events Calendar With the kind permission of LWN.net we are linking to The LWN.net Community Calendar. Quoting the site: This is the LWN.net community event calendar, where we track events of interest to people using and developing Linux and free software. Clicking on individual events will take you to the appropriate web page. Any other business FOSDEM 2023 FOSDEM 2023 will take place in Brussels, Belgium at the Université Libre de Bruxelles (or ULB) next February on the weekend of the 4th and 5th. HPR had a stand at the last in-person event, representing Free Culture Podcasts. There is a call for proposals out at the moment for those wishing to run a stand for the forthcoming event. Older HPR shows on archive.org, phase 2 Now that all shows from number 1 to the latest have been uploaded to the Internet Archive there are other tasks to perform. We are reprocessing and re-uploading shows in the range 871 to 2429 as explained in the Community News show notes released in May 2022. We are keeping a running total here to show progress: Month Month count Running total Remainder 2022-04 130 130 1428 2022-05 140 270 1288 2022-06 150 420 1138 2022-07 155 575 983 2022-08 155 730 828 2022-09 150 880 678 Updated: 2022-10-02 11:47:18

Dr. Brian Callahan Brian is a Lecturer at Rensselaer Polytechnic Institute and an OpenBSD Developer LinkedIn: https://www.linkedin.com/in/brian-robert-callahan-ph-d-707738137/ Notes: https://briancallahan.net/ https://mastodon.com/bcallah@bsdnetwork Brian's interview on BSD Now : https://www.bsdnow.tv/422 Credits: Music by ikson: https://www.iksonmusic.com Special Guest: Brian Callahan.

We explain the physics behind ZFS, DTrace switching to the GPL, Emacs debugging, syncookies coming to PF & FreeBSD's history on EC2. This episode was brought to you by Headlines 128 bit storage: Are you high? (https://blogs.oracle.com/bonwick/128-bit-storage:-are-you-high) For people who have heard about ZFS boiling oceans and wonder where that is coming from, we dug out this old piece from 2004 on the blog of ZFS co-creator Jeff Bonwick, originally from the Sun website. 64 bits would have been plenty ... but then you can't talk out of your ass about boiling oceans then, can you? Well, it's a fair question. Why did we make ZFS a 128-bit storage system? What on earth made us think it's necessary? And how do we know it's sufficient? Let's start with the easy one: how do we know it's necessary? Some customers already have datasets on the order of a petabyte, or 2^50 bytes. Thus the 64-bit capacity limit of 2^64 bytes is only 14 doublings away. Moore's Law for storage predicts that capacity will continue to double every 9-12 months, which means we'll start to hit the 64-bit limit in about a decade. Storage systems tend to live for several decades, so it would be foolish to create a new one without anticipating the needs that will surely arise within its projected lifetime. If 64 bits isn't enough, the next logical step is 128 bits. That's enough to survive Moore's Law until I'm dead, and after that, it's not my problem. But it does raise the question: what are the theoretical limits to storage capacity? Although we'd all like Moore's Law to continue forever, quantum mechanics imposes some fundamental limits on the computation rate and information capacity of any physical device. In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 10^51 operations per second on at most 10^31 bits of information [see Seth Lloyd, "Ultimate physical limits to computation." Nature 406, 1047-1054 (2000)]. A fully-populated 128-bit storage pool would contain 2^128 blocks = 2^137 bytes = 2^140 bits; therefore the minimum mass required to hold the bits would be (2^140 bits) / (10^31 bits/kg) = 136 billion kg. That's a lot of gear. To operate at the 1031 bits/kg limit, however, the entire mass of the computer must be in the form of pure energy. By E=mc^2, the rest energy of 136 billion kg is 1.2x1028 J. The mass of the oceans is about 1.4x1021 kg. It takes about 4,000 J to raise the temperature of 1 kg of water by 1 degree Celcius, and thus about 400,000 J to heat 1 kg of water from freezing to boiling. The latent heat of vaporization adds another 2 million J/kg. Thus the energy required to boil the oceans is about 2.4x106 J/kg * 1.4x1021 kg = 3.4x1027 J. Thus, fully populating a 128-bit storage pool would, literally, require more energy than boiling the oceans. Best part of all: you don't have to understand any of this to use ZFS. Rest assured that you won't hit any limits with that filesystem for a long time. You still have to buy bigger disks over time, though... *** dtrace for Linux, Oracle relicenses dtrace (https://gnu.wildebeest.org/blog/mjw/2018/02/14/dtrace-for-linux-oracle-does-the-right-thing/) At Fosdem we had a talk on dtrace for linux in the Debugging Tools devroom. Not explicitly mentioned in that talk, but certainly the most exciting thing, is that Oracle is doing a proper linux kernel port: ``` commit e1744f50ee9bc1978d41db7cc93bcf30687853e6 Author: Tomas Jedlicka tomas.jedlicka@oracle.com Date: Tue Aug 1 09:15:44 2017 -0400 dtrace: Integrate DTrace Modules into kernel proper This changeset integrates DTrace module sources into the main kernel source tree under the GPLv2 license. Sources have been moved to appropriate locations in the kernel tree. ``` That is right, dtrace dropped the CDDL and switched to the GPL! The user space code dtrace-utils and libdtrace-ctf (a combination of GPLv2 and UPL) can be found on the DTrace Project Source Control page. The NEWS file mentions the license switch (and that it is build upon elfutils, which I personally was pleased to find out). The kernel sources (GPLv2+ for the core kernel and UPL for the uapi) are slightly harder to find because they are inside the uek kernel source tree, but following the above commit you can easily get at the whole linux kernel dtrace directory. The UPL is the Universal Permissive License, which according to the FSF is a lax, non-copyleft license that is compatible with the GNU GPL. Thank you Oracle for making everyone's life easier by waving your magic relicensing wand! Now there is lots of hard work to do to actually properly integrate this. And I am sure there are a lot of technical hurdles when trying to get this upstreamed into the mainline kernel. But that is just hard work. Which we can now start collaborating on in earnest. Like systemtap and the Dynamic Probes (dprobes) before it, dtrace is a whole system observability tool combining tracing, profiling and probing/debugging techniques. Something the upstream linux kernel hackers don't always appreciate when presented as one large system. They prefer having separate small tweaks for tracing, profiling and probing which are mostly separate from each other. It took years for the various hooks, kprobes, uprobes, markers, etc. from systemtap (and other systems) to get upstream. But these days they are. And there is now even a byte code interpreter (eBPF) in the mainline kernel as originally envisioned by dprobes, which systemtap can now target through stapbpf. So with all those techniques now available in the linux kernel it will be exciting to see if dtrace for linux can unite them all. Debugging Emacs or: How I Learned to Stop Worrying and Love DTrace (http://nullprogram.com/blog/2018/01/17/) For some time Elfeed was experiencing a strange, spurious failure. Every so often users were seeing an error (spoiler warning) when updating feeds: “error in process sentinel: Search failed.” If you use Elfeed, you might have even seen this yourself. From the surface it appeared that curl, tasked with the responsibility for downloading feed data, was producing incomplete output despite reporting a successful run. Since the run was successful, Elfeed assumed certain data was in curl's output buffer, but, since it wasn't, it failed hard. Unfortunately this issue was not reproducible. Manually running curl outside of Emacs never revealed any issues. Asking Elfeed to retry fetching the feeds would work fine. The issue would only randomly rear its head when Elfeed was fetching many feeds in parallel, under stress. By the time the error was discovered, the curl process had exited and vital debugging information was lost. Considering that this was likely to be a bug in Emacs itself, there really wasn't a reliable way to capture the necessary debugging information from within Emacs Lisp. And, indeed, this later proved to be the case. A quick-and-dirty work around is to use condition-case to catch and swallow the error. When the bizarre issue shows up, rather than fail badly in front of the user, Elfeed could attempt to swallow the error — assuming it can be reliably detected — and treat the fetch as simply a failure. That didn't sit comfortably with me. Elfeed had done its due diligence checking for errors already. Someone was lying to Elfeed, and I intended to catch them with their pants on fire. Someday. I'd just need to witness the bug on one of my own machines. Elfeed is part of my daily routine, so surely I'd have to experience this issue myself someday. My plan was, should that day come, to run a modified Elfeed, instrumented to capture extra data. I would have also routinely run Emacs under GDB so that I could inspect the failure more deeply. For now I just had to wait to hunt that zebra. Bryan Cantrill, DTrace, and FreeBSD Over the holidays I re-discovered Bryan Cantrill, a systems software engineer who worked for Sun between 1996 and 2010, and is most well known for DTrace. My first exposure to him was in a BSD Now interview in 2015. I had re-watched that interview and decided there was a lot more I had to learn from him. He's become a personal hero to me. So I scoured the internet for more of his writing and talks. Some interesting operating system technology came out of Sun during its final 15 or so years — most notably DTrace and ZFS — and Bryan speaks about it passionately. Almost as a matter of luck, most of it survived the Oracle acquisition thanks to Sun releasing it as open source in just the nick of time. Otherwise it would have been lost forever. The scattered ex-Sun employees, still passionate about their prior work at Sun, along with some of their old customers have since picked up the pieces and kept going as a community under the name illumos. It's like an open source flotilla. Naturally I wanted to get my hands on this stuff to try it out for myself. Is it really as good as they say? Normally I stick to Linux, but it (generally) doesn't have these Sun technologies available. The main reason is license incompatibility. Sun released its code under the CDDL, which is incompatible with the GPL. Ubuntu does infamously include ZFS, but other distributions are unwilling to take that risk. Porting DTrace is a serious undertaking since it's got its fingers throughout the kernel, which also makes the licensing issues even more complicated. Linux has a reputation for Not Invented Here (NIH) syndrome, and these licensing issues certainly contribute to that. Rather than adopt ZFS and DTrace, they've been reinvented from scratch: btrfs instead of ZFS, and a slew of partial options instead of DTrace. Normally I'm most interested in system call tracing, and my go to is strace, though it certainly has its limitations — including this situation of debugging curl under Emacs. Another famous example of NIH is Linux's epoll(2), which is a broken version of BSD kqueue(2). So, if I want to try these for myself, I'll need to install a different operating system. I've dabbled with OmniOS, an OS built on illumos, in virtual machines, using it as an alien environment to test some of my software (e.g. enchive). OmniOS has a philosophy called Keep Your Software To Yourself (KYSTY), which is really just code for “we don't do packaging.” Honestly, you can't blame them since they're a tiny community. The best solution to this is probably pkgsrc, which is essentially a universal packaging system. Otherwise you're on your own. There's also openindiana, which is a more friendly desktop-oriented illumos distribution. Still, the short of it is that you're very much on your own when things don't work. The situation is like running Linux a couple decades ago, when it was still difficult to do. If you're interested in trying DTrace, the easiest option these days is probably FreeBSD. It's got a big, active community, thorough documentation, and a huge selection of packages. Its license (the BSD license, duh) is compatible with the CDDL, so both ZFS and DTrace have been ported to FreeBSD. What is DTrace? I've done all this talking but haven't yet described what DTrace really is. I won't pretend to write my own tutorial, but I'll provide enough information to follow along. DTrace is a tracing framework for debugging production systems in real time, both for the kernel and for applications. The “production systems” part means it's stable and safe — using DTrace won't put your system at risk of crashing or damaging data. The “real time” part means it has little impact on performance. You can use DTrace on live, active systems with little impact. Both of these core design principles are vital for troubleshooting those really tricky bugs that only show up in production. There are DTrace probes scattered all throughout the system: on system calls, scheduler events, networking events, process events, signals, virtual memory events, etc. Using a specialized language called D (unrelated to the general purpose programming language D), you can dynamically add behavior at these instrumentation points. Generally the behavior is to capture information, but it can also manipulate the event being traced. Each probe is fully identified by a 4-tuple delimited by colons: provider, module, function, and probe name. An empty element denotes a sort of wildcard. For example, syscall::open:entry is a probe at the beginning (i.e. “entry”) of open(2). syscall:::entry matches all system call entry probes. Unlike strace on Linux which monitors a specific process, DTrace applies to the entire system when active. To run curl under strace from Emacs, I'd have to modify Emacs' behavior to do so. With DTrace I can instrument every curl process without making a single change to Emacs, and with negligible impact to Emacs. That's a big deal. So, when it comes to this Elfeed issue, FreeBSD is much better poised for debugging the problem. All I have to do is catch it in the act. However, it's been months since that bug report and I'm not really making this connection yet. I'm just hoping I eventually find an interesting problem where I can apply DTrace. Bryan Cantrill: Talks I have given (http://dtrace.org/blogs/bmc/2018/02/03/talks/) *** News Roundup a2k18 Hackathon preview: Syncookies coming to PF (https://undeadly.org/cgi?action=article;sid=20180207090000) As you may have heard, the a2k18 hackathon is in progress. As can be seen from the commit messages, several items of goodness are being worked on. One eagerly anticipated item is the arrival of TCP syncookies (read: another important tool in your anti-DDoS toolset) in PF. Henning Brauer (henning@) added the code in a series of commits on February 6th, 2018, with this one containing the explanation: ``` syncookies for pf. when syncookies are on, pf will blindly answer each and every SYN with a syncookie-SYNACK. Upon reception of the ACK completing the 3WHS, pf will reconstruct the original SYN, shove it through pf_test, where state will be created if the ruleset permits it. Then massage the freshly created state (we won't see the SYNACK), set up the sequence number modulator, and call into the existing synproxy code to start the 3WHS with the backend host. Add an - somewhat basic for now - adaptive mode where syncookies get enabled if a certain percentage of the state table is filled up with half-open tcp connections. This makes pf firewalls resilient against large synflood attacks. syncookies are off by default until we gained more experience, considered experimental for now. see http://bulabula.org/papers/2017/bsdcan/ for more details. joint work with sashan@, widely discussed and with lots of input by many ``` The first release to have this feature available will probably be the upcoming OpenBSD 6.3 if a sufficient number of people test this in their setups (hint, hint). More info is likely to emerge soon in post-hackathon writeups, so watch this space! [Pale Moon] A Perfect example of how not to approach OS developers/packagers Removed from OpenBSD Ports due to Licensing Issues (https://github.com/jasperla/openbsd-wip/issues/86) FreeBSD Palemoon branding violation (https://lists.freebsd.org/pipermail/freebsd-ports/2018-February/112455.html) Mightnight BSD's response (https://twitter.com/midnightbsd/status/961232422091280386) *** FreeBSD EC2 History (http://www.daemonology.net/blog/2018-02-12-FreeBSD-EC2-history.html) A couple years ago Jeff Barr published a blog post with a timeline of EC2 instances. I thought at the time that I should write up a timeline of the FreeBSD/EC2 platform, but I didn't get around to it; but last week, as I prepared to ask for sponsorship for my work I decided that it was time to sit down and collect together the long history of how the platform has evolved and improved over the years. Normally I don't edit blog posts after publishing them (with the exception of occasional typographical corrections), but I do plan on keeping this post up to date with future developments. August 25, 2006: Amazon EC2 launches. It supports a single version of Ubuntu Linux; FreeBSD is not available. December 13, 2010: I manage to get FreeBSD running on EC2 t1.micro instances. March 22, 2011: I manage to get FreeBSD running on EC2 "cluster compute" instances. July 8, 2011: I get FreeBSD 8.2 running on all 64-bit EC2 instance types, by marking it as "Windows" in order to get access to Xen/HVM virtualization. (Unfortunately this meant that users had to pay the higher "Windows" hourly pricing.) January 16, 2012: I get FreeBSD 9.0 running on 32-bit EC2 instances via the same "defenestration" trick. (Again, paying the "Windows" prices.) August 16, 2012: I move the FreeBSD rc.d scripts which handle "EC2" functionality (e.g., logging SSH host keys to the console) into the FreeBSD ports tree. October 7, 2012: I rework the build process for FreeBSD 9.1-RC1 and later to use "world" bits extracted from the release ISOs; only the kernel is custom-built. Also, the default SSH user changes from "root" to "ec2-user". October 31, 2012: Amazon launches the "M3" family of instances, which support Xen/HVM without FreeBSD needing to pay the "Windows" tax. November 21, 2012: I get FreeBSD added to the AWS Marketplace. October 2, 2013: I finish merging kernel patches into the FreeBSD base system, and rework the AMI build (again) so that FreeBSD 10.0-ALPHA4 and later use bits extracted from the release ISOs for the entire system (world + kernel). FreeBSD Update can now be used for updating everything (because now FreeBSD/EC2 uses a GENERIC kernel). October 27, 2013: I add code to EC2 images so that FreeBSD 10.0-BETA2 and later AMIs will run FreeBSD Update when they first boot in order to download and install any critical updates. December 1, 2013: I add code to EC2 images so that FreeBSD 10.0-BETA4 and later AMIs bootstrap the pkg tool and install packages at boot time (by default, the "awscli" package). December 9, 2013: I add configinit to FreeBSD 10.0-RC1 and later to allow systems to be easily configured via EC2 user-data. July 1, 2014: Amazon launches the "T2" family of instances; now the most modern family for every type of EC2 instance (regular, high-memory, high-CPU, high-I/O, burstable) supports HVM and there should no longer be any need for FreeBSD users to pay the "Windows tax". November 24, 2014: I add code to FreeBSD 10.2 and later to automatically resize their root filesystems when they first boot; this means that a larger root disk can be specified at instance launch time and everything will work as expected. April 1, 2015: I integrate the FreeBSD/EC2 build process into the FreeBSD release building process; FreeBSD 10.2-BETA1 and later AMIs are built by the FreeBSD release engineering team. January 12, 2016: I enable Intel 82599-based "first generation EC2 Enhanced Networking" in FreeBSD 11.0 and later. June 9, 2016: I enable the new EC2 VGA console functionality in FreeBSD 11.0 and later. (The old serial console also continues to work.) June 24, 2016: Intel 82599-based Enhanced Networking works reliably in FreeBSD 11.0 and later thanks to discovering and working around a Xen bug. June 29, 2016: I improve throughput on Xen blkfront devices (/dev/xbd*) by enabling indirect segment I/Os in FreeBSD 10.4 and later. (I wrote this functionality in July 2015, but left it disabled by default a first because a bug in EC2 caused it to hurt performance on some instances.) July 7, 2016: I fix a bug in FreeBSD's virtual memory initialization in order to allow it to support boot with 128 CPUs; aka. FreeBSD 11.0 and later support the EC2 x1.32xlarge instance type. January 26, 2017: I change the default configuration in FreeBSD 11.1 and later to support EC2's IPv6 networking setup out of the box (once you flip all of the necessary switches to enable IPv6 in EC2 itself). May 20, 2017: In collaboration with Rick Macklem, I make FreeBSD 11.1 and later compatible with the Amazon "Elastic File System" (aka. NFSv4-as-a-service) via the newly added "oneopenown" mount option (and lots of bug fixes). May 25, 2017: I enable support for the Amazon "Elastic Network Adapter" in FreeBSD 11.1 and later. (The vast majority of the work — porting the driver code — was done by Semihalf with sponsorship from Amazon.) December 5, 2017: I change the default configuration in FreeBSD 11.2 and later to make use of the Amazon Time Sync Service (aka. NTP-as-a-service). The current status The upcoming FreeBSD release (11.2) supports: IPv6, Enhanced Networking (both generations), Amazon Elastic File System, Amazon Time Sync Service, both consoles (Serial VGA), and every EC2 instance type (although I'm not sure if FreeBSD has drivers to make use of the FPGA or GPU hardware on those instances). Colin's Patreon' page if you'd like to support him (https://www.patreon.com/cperciva) X network transparency X's network transparency has wound up mostly being a failure (https://utcc.utoronto.ca/~cks/space/blog/unix/XNetworkTransparencyFailure) I was recently reading Mark Dominus's entry about some X keyboard problems, in which he said in passing (quoting himself): I have been wondering for years if X's vaunted network transparency was as big a failure as it seemed: an interesting idea, worth trying out, but one that eventually turned out to be more trouble than it was worth. [...] My first reaction was to bristle, because I use X's network transparency all of the time at work. I have several programs to make it work very smoothly, and some core portions of my environment would be basically impossible without it. But there's a big qualification on my use of X's network transparency, namely that it's essentially all for text. When I occasionally go outside of this all-text environment of xterms and emacs and so on, it doesn't go as well. X's network transparency was not designed as 'it will run xterm well'; originally it was to be something that should let you run almost everything remotely, providing a full environment. Even apart from the practical issues covered in Daniel Stone's slide presentation, it's clear that it's been years since X could deliver a real first class environment over the network. You cannot operate with X over the network in the same way that you do locally. Trying to do so is painful and involves many things that either don't work at all or perform so badly that you don't want to use them. In my view, there are two things that did in general X network transparency. The first is that networks turned out to not be fast enough even for ordinary things that people wanted to do, at least not the way that X used them. The obvious case is web browsers; once the web moved to lots of images and worse, video, that was pretty much it, especially with 24-bit colour. (It's obviously not impossible to deliver video across the network with good performance, since YouTube and everyone else does it. But their video is highly encoded in specialized formats, not handled by any sort of general 'send successive images to the display' system.) The second is that the communication facilities that X provided were too narrow and limited. This forced people to go outside of them in order to do all sorts of things, starting with audio and moving on to things like DBus and other ways of coordinating environments, handling sophisticated configuration systems, modern fonts, and so on. When people designed these additional communication protocols, the result generally wasn't something that could be used over the network (especially not without a bunch of setup work that you had to do in addition to remote X). Basic X clients that use X properties for everything may be genuinely network transparent, but there are very few of those left these days. (Not even xterm is any more, at least if you use XFT fonts. XFT fonts are rendered in the client, and so different hosts may have different renderings of the same thing, cf.) < What remains of X's network transparency is still useful to some of us, but it's only a shadow of what the original design aimed for. I don't think it was a mistake for X to specifically design it in (to the extent that they did, which is less than you might think), and it did help X out pragmatically in the days of X terminals, but that's mostly it. (I continue to think that remote display protocols are useful in general, but I'm in an usual situation. Most people only ever interact with remote machines with either text mode SSH or a browser talking to a web server on the remote machine.) PS: The X protocol issues with synchronous requests that Daniel Stone talks about don't help the situation, but I think that even with those edges sanded off X's network transparency wouldn't be a success. Arguably X's protocol model committed a lesser version of part of the NeWS mistake. X's network transparency was basically free at the time (https://utcc.utoronto.ca/~cks/space/blog/unix/XFreeNetworkTransparency) I recently wrote an entry about how X's network transparency has wound up mostly being a failure for various reasons. However, there is an important flipside to the story of X's network transparency, and that is that X's network transparency was almost free at the time and in the context it was created. Unlike the situation today, in the beginning X did not have to give up lots of performance or other things in order to get network transparency. X originated in the mid 1980s and it was explicitly created to be portable across various Unixes, especially BSD-derived ones (because those were what universities were mostly using at that time). In the mid to late 1980s, Unix had very few IPC methods, especially portable ones. In particular, BSD systems did not have shared memory (it was called 'System V IPC' for the obvious reasons). BSD had TCP and Unix sockets, some System V machines had TCP (and you could likely assume that more would get it), and in general your safest bet was to assume some sort of abstract stream protocol and then allow for switchable concrete backends. Unsurprisingly, this is exactly what X did; the core protocol is defined as a bidirectional stream of bytes over an abstracted channel. (And the concrete implementation of $DISPLAY has always let you specify the transport mechanism, as well as allowing your local system to pick the best mechanism it has.) Once you've decided that your protocol has to run over abstracted streams, it's not that much more work to make it network transparent (TCP provides streams, after all). X could have refused to make the byte order of the stream clear or required the server and the client to have access to some shared files (eg for fonts), but I don't think either would have been a particularly big win. I'm sure that it took some extra effort and care to make X work across TCP from a different machine, but I don't think it took very much. (At the same time, my explanation here is probably a bit ahistorical. X's initial development seems relatively strongly tied to sometimes having clients on different machines than the display, which is not unreasonable for the era. But it doesn't hurt to get a feature that you want anyway for a low cost.) I believe it's important here that X was intended to be portable across different Unixes. If you don't care about portability and can get changes made to your Unix, you can do better (for example, you can add some sort of shared memory or process to process virtual memory transfer). I'm not sure how the 1980s versions of SunView worked, but I believe they were very SunOS dependent. Wikipedia says SunView was partly implemented in the kernel, which is certainly one way to both share memory and speed things up. PS: Sharing memory through mmap() and friends was years in the future at this point and required significant changes when it arrived. Beastie Bits Grace Hopper Celebration 2018 Call for Participation (https://www.freebsdfoundation.org/news-and-events/call-for-papers/grace-hopper-celebration-2018-call-for-participation/) Google Summer of Code: Call for Project Ideas (https://www.freebsdfoundation.org/blog/google-summer-of-code-call-for-project-ideas/) The OpenBSD Foundation 2018 Fundraising Campaign (https://undeadly.org/cgi?action=article;sid=20180129190641) SSH Mastery 2/e out (https://blather.michaelwlucas.com/archives/3115) AsiaBSDcon 2018 Registration is open (https://2018.asiabsdcon.org/) Tarsnap support for Bitcoin ending April 1st; and a Chrome bug (http://mail.tarsnap.com/tarsnap-announce/msg00042.html) Feedback/Questions Todd - Couple Questions (http://dpaste.com/195HGHY#wrap) Seth - Tar Snap (http://dpaste.com/1N7NQVQ#wrap) Alex - sudo question (http://dpaste.com/3D9P1DW#wrap) Thomas - FreeBSD on ARM? (http://dpaste.com/24NMG47#wrap) Albert - Austria BSD User Group (http://dpaste.com/373CRX7#wrap)

We provide you with updates to Spectre and Meltdown from various BSD projects, a review of TrueOS from Linux, how to set up FreeBSD on ThinkPad x240, and a whole bunch of beastie bits. This episode was brought to you by Headlines KPTI patch lands in FreeBSD -current (https://svnweb.freebsd.org/base?view=revision&revision=328083) After a heroic effort by Konstantin Belousov kib@FreeBSD.org, the first meltdown patch has landed in FreeBSD This creates separate page tables for the Kernel and userland, and switches between them when executions enters the kernel, and when it returns to userland It is currently off by default, but you are encouraged to test it, so it can be merged back to the release branches. Set vm.pmap.pti=1 in /boot/loader.conf The existing implementation of PCID (process-context identifiers), is not compatible with the new PTI code, and is disabled when PTI is enabled, decreasing performance. A future patch will use PCID in a way that is compatible with PTI. PCID allows the OS to annotate memory mappings to specific processes, so that they can be flushed selectively, and so that they are only used when in the context of that application. Once the developers are relatively confident in the correctness of the code that has landed in -current, it will be ported back to FreeBSD 10 and 11, and released as a security advisory. Apparently porting back to FreeBSD 11 only has some relatively simple merge conflicts, but 10 will be more work. Former FreeBSD Security Officer Dag-Erling Smørgrav has created a meltdown testing and PoC tool (https://github.com/dag-erling/meltdown) that you can use to check your system. It is not finished yet, and doesn't seem to work with newer processors (haswell and newer). The first partial mitigation for Spectre variant 2 (https://svnweb.freebsd.org/changeset/base/328011) for bhyve on AMD64 has also been committed The latest information is always available on the FreeBSD Wiki (https://wiki.freebsd.org/action/edit/SpeculativeExecutionVulnerabilities) *** Some thoughts on Spectre and Meltdown (http://www.daemonology.net/blog/2018-01-17-some-thoughts-on-spectre-and-meltdown.html) Colin Percival breaks down how these vulnerabilities work, with same nice analogies What is a side channel: I want to know when my girlfriend's passport expires, but she won't show me her passport (she complains that it has a horrible photo) and refuses to tell me the expiry date. I tell her that I'm going to take her to Europe on vacation in August and watch what happens: If she runs out to renew her passport, I know that it will expire before August; while if she doesn't get her passport renewed, I know that it will remain valid beyond that date. Her desire to ensure that her passport would be valid inadvertently revealed to me some information: Whether its expiry date was before or after August. Spectre Variant 1: I tell my girlfriend that I'm going to take her on vacation in June, but I don't tell her where yet; however, she knows that it will either be somewhere within Canada (for which she doesn't need a passport, since we live in Vancouver) or somewhere in Europe. She knows that it takes time to get a passport renewed, so she checks her passport and (if it was about to expire) gets it renewed just in case I later reveal that I'm going to take her to Europe. If I tell her later that I'm only taking her to Ottawa — well, she didn't need to renew her passport after all, but in the meantime her behaviour has already revealed to me whether her passport was about to expire. This is what Google refers to "variant 1" of the Spectre vulnerability: Even though she didn't need her passport, she made sure it was still valid just in case she was going to need it. Spectre Variant 2: I spend a week talking about how Oxford is a wonderful place to visit and I really enjoyed the years I spent there, and then I tell her that I want to take her on vacation. She very reasonably assumes that — since I've been talking about Oxford so much — I must be planning on taking her to England, and runs off to check her passport and potentially renew it... but in fact I tricked her and I'm only planning on taking her to Ottawa. Meltdown: I tell my girlfriend that I want to take her to the Korean peninsula. She knows that her passport is valid for long enough; but she immediately runs off to check that her North Korean visa hasn't expired. Why does she have a North Korean visa, you ask? Good question. She doesn't — but she runs off to check its expiry date anyway! Because she doesn't have a North Korean visa, she (somehow) checks the expiry date on someone else's North Korean visa, and then (if it is about to expire) runs out to renew it — and so by telling her that I want to take her to Korea for a vacation I find out something she couldn't have told me even if she wanted to. Final thoughts on vulnerability disclosure The way these issues were handled was a mess; frankly, I expected better of Google, I expected better of Intel, and I expected better of the Linux community. When I found that Hyper-Threading was easily exploitable, I spent five months notifying the security community and preparing everyone for my announcement of the vulnerability; but when the embargo ended at midnight UTC and FreeBSD published its advisory a few minutes later, the broader world was taken entirely by surprise. Nobody knew what was coming aside from the people who needed to know; and the people who needed to know had months of warning. Contrast that with what happened this time around. Google discovered a problem and reported it to Intel, AMD, and ARM on June 1st. Did they then go around contacting all of the operating systems which would need to work on fixes for this? Not even close. FreeBSD was notified the week before Christmas, over six months after the vulnerabilities were discovered. Now, FreeBSD can occasionally respond very quickly to security vulnerabilities, even when they arise at inconvenient times — on November 30th 2009 a vulnerability was reported at 22:12 UTC, and on December 1st I provided a patch at 01:20 UTC, barely over 3 hours later — but that was an extremely simple bug which needed only a few lines of code to fix; the Spectre and Meltdown issues are orders of magnitude more complex. To make things worse, the Linux community was notified and couldn't keep their mouths shut. Standard practice for multi-vendor advisories like this is that an embargo date is set, and nobody does anything publicly prior to that date. People don't publish advisories; they don't commit patches into their public source code repositories; and they definitely don't engage in arguments on public mailing lists about whether the patches are needed for different CPUs. As a result, despite an embargo date being set for January 9th, by January 4th anyone who cared knew about the issues and there was code being passed around on Twitter for exploiting them. This is not the first time I've seen people get sloppy with embargoes recently, but it's by far the worst case. As an industry we pride ourselves on the concept of responsible disclosure — ensuring that people are notified in time to prepare fixes before an issue is disclosed publicly — but in this case there was far too much disclosure and nowhere near enough responsibility. We can do better, and I sincerely hope that next time we do. CPU microcode update code for amd64 (https://undeadly.org/cgi?action=article;sid=20180115073406) (https://marc.info/?l=openbsd-tech&m=151588857304763&w=2) Patrick Wildt (patrick@) recently committed some code that will update the Intel microcode on many Intel CPUs, a diff initially written by Stefan Fritsch (sf@). The microcode of your CPU is basically the firmware that runs on your (Intel) processor, defining its instruction set in terms of so called "microinstructions". The new code depends, of course, on the corresponding firmware package, ported by Patrick which can be installed using a very recent fw_update(1). Of course, this all plays into the recently revealed problems in Intel (and other) CPUs, Meltdown and Spectre. Now Theo has explained the workings of the code on openbsd-tech, detailing some of the challenges in updating microcode on CPUs where your OS is already starting to run. Theo hints at future updates to the intel-firmware package in his mail: (https://marc.info/?l=openbsd-tech&m=151588857304763&w=2) Patrick and others committed amd64 Intel cpu microcode update code over the last few days. The approach isn't perfect, but it is good enough for a start. I want to explain the situation. When you fw_update, you'll get the firmware files. Upon a reboot, it will attempt to update the microcode on your cpus. Maybe there isn't a new microcode. Maybe your BIOS has a copy of the microcode and installs it before booting OpenBSD. This firmware installation is done a little late. Doing it better will require some work in the bootblocks to find the firmware files, but time is a bit short to do that right now. The branch-target-cache flushing features added in new microcode are not being used yet. There is more code which has to be written, but again other work is happening first. Also, Intel is saying their new microcodes sucks and people should wait a little. "Hi, my name is Intel and I'm an cheating speculator". Several developers are working on mitigations for these issues, attacking the problem from several angles. Expect to see more updates to a CVS tree near you soon. Intel: as a *BSD user, I am fucking pissed! (https://malcont.net/2018/01/dont-like-meltdown-spectre-releated-bugs-handled/) I wasn't going to write anything on the recently found x64 architecture – related bugs. I'm not a kernel developer nor even a programmer and I can't say that I have a solid understanding of what Meltdown and Spectre attacks are. Also there already is a ton of articles and posts written by people who have no grasp of the subject. I'm however a malcontent and I find this a good way to express my feelings: Intel: as a *BSD user, I am fucking pissed! Meltdown, Spectre and BSD – the “pissed” part Part of my work is UNIX-like systems administration – including BSDs and Linuces. As much as I am happy with Linux changes already made, I am beyond pissed about how the BSDs were handled by Intel – because they were not. FreeBSD Security Team received some heads-up just before Xmas, while OpenBSD, NetBSD and DragonflyBSD teams received no prior warnings. Meltdown and Spectre attacks are hard to perform. It is a hard work to mitigate them in the software, as the bugs lay in the CPUs and are not fixable by microcode updates. Developers are trying to mitigate these bugs in a way that will deliver smallest performance losses. A lot of time consuming work is needed to fix CPU vendors' mistakes. Linux developers had this time. BSD developers did not. BSD user base too small? BSD user base is small in comparison to Linux. Seems that it's too small for Intel. PlayStation4 consoles are FreeBSD-based (and use AMD CPUs) but I think it's safe to say that gaming devices are not the most important systems to be fixed. Netflix serves their content off FreeBSD but the bugs are not remotely exploitable (possibly not including JavaScript, but it's running someone's code locally) so there's probably not much harm to be done here either. However gamers and Netflix aren't the only ones who use *BSD systems. I'd say that there is more than a few FreeBSD, NetBSD, OpenBSD and DragonFlyBSD servers on the internet. In March 2017, Intel promised “more timely support to FreeBSD”. They knew about flaws in their CPUs in June and decided that a timely manner is the end of December – short before the embargo was to be lifted. Intel and Google (probably Intel more): it was your job to pick the correct people to whom the bugs can be disclosed. In my humble opinion you chose poorly by disclosing these issues with ONLY Apple, Microsoft, and the Linux Foundation, of OS vendors. You did much harm to the BSD community. Intel: It's your bugs. And you offered “more support” to the FreeBSD Foundation less than 3 months prior to being informed (my guess is that you knew much earlier) on the flaws in YOUR products. I don't want to write more here as the wording would be too strong. Interview - Viewer Questions These days, do you consider yourself more of an programmer or a sysadmin? Which one do you enjoy more? Does FreeBSD/BSD enable your business or would another OS suit your needs just as well? You've hinted that you use FreeBSD as part of your business. Can you elaborate on that and give some technical detail on how it's used in that environment? If you were allowed three wishes for anything at all to be implemented or changed in ZFS, what would they be, and why? Per Dataset throughput and IOPS limiting Per-File Cloning and/or zfsmv (move a file from one dataset to another, without copying) Cluster support Allan, you have previously mentioned that you have worked on FreeBSD on MIPS, what made you choose the Onion Omega over something like the Raspberry Pi? What is BSD Now's association with Jupiter broadcasting, and how did the relationship come to be? Jupiter seems to be associated with several Linux-themed podcasts, and I'm wondering how and why BSD Now joined Jupiter. The two communities (the Linuxes and BSDs) don't always seem to mix freely -- or do they? What kind of keyboard is that? Have you ever tried an ErgoDox? The ErgoDox EZ is made by a Canadian. You mentioned when doing one of your talks on UCL for FreeBSD that you had only recently learned C. I am also aware of your history also on contributing to the FreeBSD handbook and to documentation in general. Given you started with C relatively recently, what made you want to learn it, how quickly did you pick it up, and is it your favourite language? It is most inspiring to me, as you are clearly so talented, and of all the languages I have learned (including C++), I still prefer C in my heart of hearts. I'd be really interested to hear your answer, many thanks. *** News Roundup LinuxAndUbuntu Review Of TrueOS A Unix Based OS (http://www.linuxandubuntu.com/home/linuxandubuntu-review-of-trueos-a-unix-based-os) Trust me, the name TrueOS takes me back to 1990s when Tru64 UNIX operating system made its presence. TrueOS is PC-BSD's new unified brand built upon FreeBSD-CURRENT code base. Note that TrueOS is not a Linux distro but is BSD Unix. FreeBSD is known for its cutting-edge features, security, scalability, and ability to work both as a server and desktop operating system. TrueOS aims at having user-friendliness with the power of FreeBSD OS. Let us start with going into details of different aspects of the TrueOS. TrueOS History ? TrueOS was founded by Kris Moore in 2005 with name PC-BSD. Initial version focused to make FreeBSD easy to use starting with providing GUI based installer (to relatively complicated FreeBSD installer). In the year 2006, PC-BSD was acquired by iXsystems. Before rebranding as TrueOS in Sept 2016, PC-BSD reached a stage starting considering better than vanilla FreeBSD. Older PC-BSD version used to support both x86 and x86-64 architecture. Kris Moore, the developer founder, says about rebranding: “We've already been using TrueOS for the server side of PC-BSD, and it made sense to unify the names. PC-BSD doesn't reflect server or embedded well. TrueOS Desktop/Server/Embedded can be real products, avoids some of the alphabet soup, and gives us a more catchy name.” TrueOS First Impression ? The startup is little longer; may be due to starting up of many services. The heavy KDE well suited to PC-BSD. The C++/Qt5 based Lumina desktop environment is light and fast. The Lumina offers an easy way to configure menu and panels. I did not face any problems for continuous use of two weeks on a virtual machine having the minimal configuration: 1 GB RAM, 20 GB hard disk and Intel 3.06 GHz i3 processor. The Lumina desktop is light and fast. The developers of Lumina know what they are doing and have a good idea of what makes a good IDE. As it happens with any new desktop environment, it needs some time to settle. Let us hope that they keep to the path they are on with it. Conclusion ? The TrueOS is impressive when consider it as relatively young. It is a daring step that TrueOS developers took FreeBSD Current rather than FreeBSD Stable code base. Overall it has created its own place from the legacy shadow of PC-BSD. Starting with easy installation TrueOS is a good combination of software and utilities that make the system ready to use. Go and get a TrueOS ISO to unleash the “bleeding edge” tag of FreeBSD Thinkpad x240 - FreeBSD Setup (http://stygix.org/nix/x240-freebsd.php) What follows is a record of how I set up FreeBSD to be my daily driver OS on the Lenovo Thinkpad X240. Everything seems to work great. Although, the touchpad needs some tweaking. I've tried several configurations, even recompiling Xorg with EVDEV support and all that, to no avail. Eventually I will figure it out. Do not sleep the laptop from the command line. Do it from within Xorg, or it will not wake up. I don't know why. You can do it from a terminal within Xorg, just not from the naked command line without Xorg started. It also will not sleep by closing the lid. I included a sudo config that allows you to run /usr/sbin/zzz without a password, so what I do is I have a key combo assigned within i3wm to run "sudo /usr/sbin/zzz". It works fine this way. I go into detail when it comes to setting up Xorg with i3wm. You can skip this if you want, but if you've never used a tiling window manager, it will handle screen real estate very efficiently on a laptop with a 12.5-inch screen and a touchpad. First, download the amd64 image for 11.1-RELEASE and flash it to a USB pen drive. For the Unices, use this: # dd if=FreeBSD-11.1-RELEASE-amd64-memstick.img of=/dev/da0 bs=1M conv=sync Obviously, you'll change /dev/da0 to whatever the USB pen drive is assigned. Plug it in, check dmesg. Leave it plugged in, restart the laptop. When prompted, tap Enter to halt the boot process, then F12 to select a bootable device. Choose the USB drive. I won't go through the actual install process, but it is pretty damn easy so just look at a guide or two and you'll be fine. If you can install Debian, you can install FreeBSD. I will, however, recommend ZFS if you have over 4GB of RAM (my particular variant of the X240 has 8GB of RAM, so yours should have at least 4GB), along with an encrypted disk, and an encrypted SWAP partition. When prompted to add an additional user, and you get to the question where it asks for additional groups, please make sure you add the user to "wheel". The rest should be self-explanatory during the install. Now for the good shit. You just booted into a fresh FreeBSD install. Now what? Well, time to fire up vi and open some config files... CNN Article about CDROM.com and FreeBSD, from 1999 (https://www.cnn.com/TECH/computing/9904/08/cdrom.idg/index.html) Walnut Creek CDROM sells a lot of CD-ROMs, but it gives away even more data. Specifically, anyone who has Internet access is free to log into wcarchive (ftp.cdrom.com) and start downloading bits. Even with a good Internet connection, however, you should expect to be at it for a while. At the present time, wcarchive resides on half a terabyte (500 GB) of RAID 5-disk storage. Even if your 56-Kbps modem can deliver seven kilobytes per second, downloading the complete archive would take you 70 million seconds. Even then, some of the files would be more than two years out of date, so a bit of "back and fill" would be needed. Of course, nobody uses wcarchive that way. Instead, they just drop in when they need the odd file or two. The FTP server is very accommodating; 3,600 simultaneous download sessions is the current limit and an upgrade to 10,000 sessions is in the works. This translates to about 800 GB per day of downloads. Bob Bruce (Walnut Creek's founder) says he's thinking about issuing a press release when they reach a terabyte a day. But 800 GB isn't all that shabby.... The hardware Because FTP archives don't do a lot of thinking, wcarchive doesn't need a massive cluster of CPUs. In fact, it gets by with a single 200-MHz P6 Pentium Pro and a measly(!) 1 GB of RAM. The I/O support, however, is fairly impressive. A six-channel Mylex RAID controller (DAC960SXI; Ultra-Wide SCSI-SCSI) is the centerpiece of the I/O subsystem. Two channels link it to the PC ("Personal Computer"!?!), via a dual-channel Adaptec card (AHA-3940AUW; PCI to Ultra-Wide SCSI). An 256-MB internal cache helps it to eliminate recurring disk accesses. Four nine-drive disk arrays provide the actual storage. The two larger arrays use 18-GB IBM drives; the two smaller arrays use 9-GB Micropolis and Quantum drives. A separate 4-GB Quantum drive is used as the "system disk." The output side is handled by a single Intel 100Base-T controller (Pro/100B PCI), which feeds into the Internet through a number of shared DS3 (45 Mbps) and OC3 (155 Mbps) circuits. A detailed description of the system is available as ftp.cdrom.com/archive-info/configuration; The software The system software is rather prosaic: a copy of FreeBSD, supplemented by home-grown FTP mirroring and server code. Because of the massive hardware support, the software "only" needs to keep the I/O going in an efficient and reliable manner. FreeBSD, the "prosaic" operating system mentioned above, merits a bit more discussion. Like Linux, FreeBSD is open source. Anyone can examine, modify, and/or redistribute the source code. And, like Linux, an active user community helps the authors to find bugs, improve documentation, and generally support the OS. Unlike Linux, FreeBSD is derived from the Berkeley Unix code that forms the foundation for most commercial Unix variants. When you use the "fast file system" (cylinder groups, long file names, symbolic links, etc.), TCP/IP networking, termcap, or even vi, you are using Berkeley Unix additions. The version of BSD underlying FreeBSD, however, is "pure" BSD; don't look for the System V modifications you see in Solaris. Instead, think of it as SunOS, brought up to date with Kerberos, modern sendmail, an updated filesystem, and more. Solid, fast, and free! One of FreeBSD's finest innovations, the Ports Collection, makes FreeBSD a delight for open source application users. The Ports Collection automates the downloading, building, and installation (including de-installation) of 2,300+ open source packages. The company Walnut Creek CDROM has been around for several years now, so you are likely to be familiar with its offerings. You may not realize, however, that it provides the major financial support for FreeBSD. The FreeBSD support has two purposes. First, it provides the company with a solid base to run wcarchive and other massive projects. Second, it ties in with the company's mission of making software (and data) economically accessible. Bob Bruce, the firm's founder, is an interesting guy: laid back and somewhat conservative in manner, but productive and innovative in practice. Here is a possibly illustrative story. When Bob started selling CD-ROMs, disc caddies were selling for $15 each. Bob thought that was rather high, so he started investigating the marketplace. A long-distance call to Japan got him Sony's fax number; a series of faxes got him in touch with the salespeople. It turned out that caddies were available, in bulk, for only a few dollars each. Bulk, in this case, meant pallet-loads of 10,000 caddies. In an act of great faith, Bob purchased a pallet of caddies, then proceeded to sell them for five dollars each. The results were everything he might have wished. Folks who bought his CD-ROMs added caddies to their orders; folks who bought piles of caddies added in a disc or two. Either way, Walnut Creek CDROM was making a name for itself. Many pallet-loads later, the company is still selling caddies, making and distributing CD-ROMs, and giving away bits. Walnut Creek CDROM is a real open-source success story; its breadth and depth of offerings is well worth a look. Beastie Bits OpenBSD adds kqueue event support to DRM, to detect device changes like HDMI cables being plugged in, and trigger randr events (https://github.com/openbsd/src/commit/b8584f4233dc11a328cd245a5843ec3d67462200) Thesis describing QUAD3, a unix-like, multi-tasking operating system for the 6502 processor (https://archive.org/details/AMultiTaskingOperatingSystemForMicrocomputers) Windows is getting chmod and chown... (https://blogs.msdn.microsoft.com/commandline/2018/01/12/chmod-chown-wsl-improvements/) Timeline: How they kept Meltdown and Spectre secret for so long (https://www.theverge.com/platform/amp/2018/1/11/16878670/meltdown-spectre-disclosure-embargo-google-microsoft-linux) bsd.network is a *BSD-themed Mastodon Instance (https://bsd.network/): Peter Hessler is administering a new Mastodon instance, running in an OpenBSD VM on top of an OpenBSD vmm hypervisor Computer-Aided Instruction on UNIX (https://virtuallyfun.com/wordpress/wp-content/uploads/2017/12/whfUb.pdf) AsiaBSDCon 2018 Travel Grant Application Now Open (https://www.freebsdfoundation.org/blog/asiabsdcon-2018-travel-grant-application-now-open/) AsiaBSDCon 2018 FreeBSD Developers Summit Call for Proposals (https://www.freebsdfoundation.org/news-and-events/call-for-papers/asiabsdcon-2018-freebsd-developers-summit-call-for-proposals/) LinuxFest Northwest 2018 Call for Proposals (https://www.freebsdfoundation.org/news-and-events/call-for-papers/linuxfest-northwest-2018-call-for-proposals/) Feedback/Questions Jason - Dont break my ports (http://dpaste.com/05PRNG2) Wilyarti - show content (http://dpaste.com/1BG8GZW) https://clinetworking.wordpress.com/2017/12/08/data-de-duplication-file-diff-ing-and-s3-style-object-storage-using-digital-ocean-spaces Scott - Your show is Perfect! (http://dpaste.com/0KER8YE#wrap) Ken - Community Culture (http://dpaste.com/0WT8285#wrap)

EuroBSDcon trip report, how to secure OpenBSD's LDAP server, ZFS channel programs in FreeBSD HEAD and why software is storytelling. This episode was brought to you by Headlines EuroBSDcon Trip Report This is from Frank Moore, who has been supplying us with collections of links for the show and who we met at EuroBSDcon in Paris for the first time. Here is his trip report. My attendance at the EuroBSDCon 2017 conference in Paris was sprinkled with several 'firsts'. My first visit to Paris, my first time travelling on a EuroTunnel Shuttle train and my first time at any BSD conference. Hopefully, none of these will turn out to be 'lasts'. I arrived on the Wednesday afternoon before the conference started on Thursday morning. My hotel was conveniently located close to the conference centre in Paris' 3rd arrondissement. This area is well-known as a buzzy enclave of hip cafes, eateries, independent shops, markets, modern galleries and museums. It certainly lived up to its reputation. Even better, the weather held over the course of the conference, only raining once, with the rest of the time being both warm and sunny. The first two days were taken up with attending Dr Kirk McKusick's excellent tutorial 'An Introduction to the FreeBSD Open-Source Operating System'. This is training "straight from the horse's mouth". Kirk has worked extensively on The FreeBSD operating system since the 1980's, helping to design the original BSD filesystem (FFS) and later working on UFS as well. Not only is Kirk an engaging speaker, making what could be a dry topic very interesting, he also sprinkles liberal doses of history and war stories throughout his lectures. Want to know why a protocol was designed the way that it was? Or why a system flag has a particular value or position in a record? Kirk was there and has the first-hand answer. He reminisces about his meetings and work with other Unix and BSD luminaries and debunks and confirms common myths in equal measure. Kirk's teaching style and knowledge are impressive. Every section starts with an overview and a big picture diagram before drilling down into the nitty-gritty detail. Nothing feels superfluous, and everything fits together logically. It's easy to tell that the material and its delivery have been honed over many years, but without feeling stale. Topics covered included the kernel, processes, virtual memory, threads, I/O, devices, FFS, ZFS, and networking. The slides were just as impressive, with additional notes written by a previous student and every slide containing a reference back to the relevant page(s) in the 2nd edition of Kirk's operating system book. As well as a hard copy for those that requested it, Kirk also helpfully supplied soft copies of all the training materials. The breaks in between lectures were useful for meeting the students from the other tutorials and for recovering from the inevitable information overload. It's not often that you can get to hear someone as renowned as Dr McKusick give a lecture on something as important as the FreeBSD operating system. If you have any interest in FreeBSD, Unix history, or operating systems in general, I would urge you to grab the opportunity to attend one of his lectures. You won't be disappointed. The last two days of the conference consisted of various hour-long talks by members of each of the main BSD systems. All of them were fairly evenly represented except Dragonfly BSD which unfortunately only had one talk. With three talks going on at any one time, it was often difficult to pick which one to go to. At other times there might be nothing to pique the interest. Attendance at a talk is not mandatory, so for those times when no talks looked inviting, just hanging out in one of the lobby areas with other attendees was often just as interesting and informative. The conference centre itself was certainly memorable with the interior design of an Egyptian temple or pyramid. All the classrooms were more than adequate while the main auditorium was first-class and easily held the 300+ attendees comfortably. All in all, the facilities, catering and organisation were excellent. Kudos to the EuroBSDCon team, especially Bapt and Antoine for all their hard work and hospitality. As a long-time watcher and occasional contributor to the BSD Now podcast it was good to meet both Allan and Benedict in the flesh. And having done some proofreading for Michael Lucas previously, it was nice to finally meet him as well. My one suggestion to the organisers of the next conference would be to provide more hand-holding for newbies. As a first-time attendee at a BSD conference it would have been nice to have been formally introduced to various people within the projects as the goto people for their areas. I could do this myself, but it's not always easy finding the right person and wrangling an introduction. I also think it was a missed opportunity for each project to recruit new developers to their cause. Apparently, this is already in place at BSDCan, but should probably be rolled out across all BSD conferences. Having said all that, my aims for the conference were to take Dr McKusick's course, meet a few BSD people and make contacts within one of the BSD projects to start contributing. I was successful on all these fronts, so for me this was mission accomplished. Another first! autoconf/clang (No) Fun and Games (https://undeadly.org/cgi?action=article;sid=20170930133438) Robert Nagy (robert@) wrote in with a fascinating story of hunting down a recent problem with ports: You might have been noticing the amount of commits to ports regarding autoconf and nested functions and asking yourself… what the hell is this all about? I was hanging out at my friend Antoine (ajacoutot@)'s place just before EuroBSDCon 2017 started and we were having drinks and he told me that there is this weird bug where Gnome hangs completely after just a couple of seconds of usage and the gnome-shell process just sits in the fsleep state. This started to happen at the time when inteldrm(4) was updated, the default compiler was switched to clang(1) and futexes were turned on by default. The next day we started to have a look at the issue and since the process was hanging in fsleep, it seemed clear that the cause must be futexes, so we had to start bisecting the base system, which resulted in random success and failure. In the end we figured out that it is neither futex nor inteldrm(4) related, so the only thing that was left is the switch to clang. Now the problem is that we have to figure out what part of the system needs to be build with clang to trigger this issue, so we kept on going and systematically recompiled the base system with gcc until everything was ruled out … and it kept on hanging. We were drunk and angry that now we have to go and check hundreds of ports because gnome is not a small standalone port, so between two bottles of wine a build VM was fired up to do a package build with gcc, because manually building all the dependencies would just take too long and we had spent almost two days on this already. Next day ~200 packages were available to bisect and figure out what's going on. After a couple of tries it turned out that the hang is being caused by the gtk+3 package, which is bad since almost everything is using gtk+3. Now it was time to figure out what file the gtk+3 source being built by clang is causing the issue. (Compiler optimizations were ruled out already at this point.) So another set of bisecting happened, building each subdirectory of gtk+3 with clang and waiting for the hang to manifest … and it did not. What the $f? Okay so something else is going on and maybe the configure script of gtk+3 is doing something weird with different compilers, so I quickly did two configure runs with gcc and clang and simply diff'd the two directories. Snippets from the diff: -GDKHIDDENVISIBILITYCFLAGS = -fvisibility=hidden GDKHIDDENVISIBILITYCFLAGS = -ltcvprogcompilerrttiexceptions=no ltcvprogcompilerrttiexceptions=yes -#define GDKEXTERN attribute((visibility("default"))) extern -ltprogcompilernobuiltinflag=' -fno-builtin' +ltprogcompilernobuiltinflag=' -fno-builtin -fno-rtti -fno-exceptions' Okay, okay that's something, but wait … clang has symbol visibility support so what is going on again? Let's take a peek at config.log: configure:29137: checking for -fvisibility=hidden compiler flag configure:29150: cc -c -fvisibility=hidden -I/usr/local/include -I/usr/X11R6/include conftest.c >&5 conftest.c:82:17: error: function definition is not allowed here int main (void) { return 0; } ^ 1 error generated. Okay that's clearly an error but why exactly? autoconf basically generates a huge shell script that will check for whatever you throw at it by creating a file called conftest.c and putting chunks of code into it and then trying to compile it. In this case the relevant part of the code was: | int | main () | { | int main (void) { return 0; } | ; | return 0; | } That is a nested function declaration which is a GNU extension and it is not supported by clang, but that's okay, the question is why the hell would you use nested functions to check for simple compiler flags. The next step was to go and check what is going on in configure.ac to see how the configure script is generated. In the gtk+3 case the following snippet is used: AC_MSG_CHECKING([for -fvisibility=hidden compiler flag]) ACTRYCOMPILE([], [int main (void) { return 0; }], ACMSGRESULT(yes) enablefvisibilityhidden=yes, ACMSGRESULT(no) enablefvisibilityhidden=no) According to the autoconf manual the ACTRYCOMPILE macro accepts the following parameters: That clearly states that a function body has to be specified because the function definition is already provided automatically, so doing ACTRYCOMPILE([], [int main (void) { return 0;}], instead of ACTRYCOMPILE([],[] will result in a nested function declaration, which will work just fine with gcc, even though the autoconf usage is wrong. After fixing the autoconf macro in gtk+3 and rebuilding the complete port from scratch with clang, the hang completely went away as the proper CFLAGS and LDFLAGS were picked up by autoconf for the build. At this point we realized that most of the ports tree uses autoconf so this issue might be a lot bigger than we thought, so I asked sthen@ to do a grep on the ports object directory and just search for "function definition is not allowed here", which resulted in about ~60 additional ports affected. Out of the list of ports there were only two false positive matches. These were actually trying to test whether the compiler supports nested functions. The rest were a combination of several autoconf macros used in a wrong way, e.g: ACTRYCOMPILE, ACTRYLINK. Most of them were fixable by just removing the extra function declaration or by switching to other autoconf macros like ACLANGSOURCE where you can actually declare your own functions if need be. The conclusion is that this issue was a combination of people not reading documentation and just copy/pasting autoconf snippets, instead of reading their documentation and using the macros in the way they were intended, and the fact that switching to a new compiler is never easy and bugs or undefined behaviour are always lurking in the dark. Thanks to everyone who helped fixing all the ports up this quickly! Hopefully all of the changes can be merged upstream, so that others can benefit as well. Interview - David Carlier - @devnexen (https://twitter.com/devnexen) Software Engineer at Afilias *** News Roundup Setting up OpenBSD's LDAP Server (ldapd) with StartTLS and SASL (http://blog.databasepatterns.com/2017/08/setting-up-openbsds-ldap-server-ldapd.html) A tutorial on setting up OpenBSD's native LDAP server with TLS encryption and SASL authentication OpenBSD has its own LDAP server, ldapd. Here's how to configure it for use with StartTLS and SASL authentication Create a certificate (acme-client anyone?) Create a basic config file listen on em0 tls certificate ldapserver This will listen on the em0 interface with tls using the certificate called ldapserver.crt / ldapserver.key Validate the configuration: /usr/sbin/ldapd -n Enable and start the service: rcctl enable ldapd rcctl start ldapd On the client machine: pkg_add openldap-client Copy the certificate to /etc/ssl/trusted.crt Add this line to /etc/openldap/ldap.conf TLS_CACERT /etc/ssl/trusted.crt Enable and start the service rcctl enable saslauthd rcctl start saslauthd Connect to ldapd (-ZZ means force TLS, use -H to specify URI): ldapsearch -H ldap://ldapserver -ZZ FreeBSD Picks Up Support for ZFS Channel Programs in -current (https://svnweb.freebsd.org/base?view=revision&revision=324163) ZFS channel programs (ZCP) adds support for performing compound ZFS administrative actions via Lua scripts in a sandboxed environment (with time and memory limits). This initial commit includes both base support for running ZCP scripts, and a small initial library of API calls which support getting properties and listing, destroying, and promoting datasets. Testing: in addition to the included unit tests, channel programs have been in use at Delphix for several months for batch destroying filesystems. Take a simple task as an example: Create a snapshot, then set a property on that snapshot. In the traditional system for this, when you issue the snapshot command, that closes the currently open transaction group (say #100), and opens a new one, #101. While #100 is being written to disk, other writes are accumulated in #101. Once #100 is flushed to disk, the ‘zfs snapshot' command returns. You can then issue the ‘zfs set' command. This actually ends up going into transaction group #102. Each administrative action needs to wait for the transaction group to flush, which under heavy loads could take multiple seconds. Now if you want to create AND set, you need to wait for two or three transaction groups. Meanwhile, during transaction group #101, the snapshot existed without the property set, which could cause all kinds of side effects. ZFS Channel programs solves this by allowing you to perform a small scripted set of actions as a single atomic operation. In Delphix's appliance, they often needed to do as many as 15 operations together, which might take multiple minutes. Now with channel programs it is much faster, far safer, and has fewer chances of side effects BSDCan 2017 - Matt Ahrens: Building products based on OpenZFS, using channel programs -- Video Soon (http://www.bsdcan.org/2017/schedule/events/854.en.html) Software Is About Storytelling (http://bravenewgeek.com/software-is-about-storytelling/) Tyler Treat writes on the brave new geek blog: Software engineering is more a practice in archeology than it is in building. As an industry, we undervalue storytelling and focus too much on artifacts and tools and deliverables. How many times have you been left scratching your head while looking at a piece of code, system, or process? It's the story, the legacy left behind by that artifact, that is just as important—if not more—than the artifact itself. And I don't mean what's in the version control history—that's often useless. I mean the real, human story behind something. Artifacts, whether that's code or tools or something else entirely, are not just snapshots in time. They're the result of a series of decisions, discussions, mistakes, corrections, problems, constraints, and so on. They're the product of the engineering process, but the problem is they usually don't capture that process in its entirety. They rarely capture it at all. They commonly end up being nothing but a snapshot in time. It's often the sign of an inexperienced engineer when someone looks at something and says, “this is stupid” or “why are they using X instead of Y?” They're ignoring the context, the fact that circumstances may have been different. There is a story that led up to that point, a reason for why things are the way they are. If you're lucky, the people involved are still around. Unfortunately, this is not typically the case. And so it's not necessarily the poor engineer's fault for wondering these things. Their predecessors haven't done enough to make that story discoverable and share that context. I worked at a company that built a homegrown container PaaS on ECS. Doing that today would be insane with the plethora of container solutions available now. “Why aren't you using Kubernetes?” Well, four years ago when we started, Kubernetes didn't exist. Even Docker was just in its infancy. And it's not exactly a flick of a switch to move multiple production environments to a new container runtime, not to mention the politicking with leadership to convince them it's worth it to not ship any new code for the next quarter as we rearchitect our entire platform. Oh, and now the people behind the original solution are no longer with the company. Good luck! And this is on the timescale of about five years. That's maybe like one generation of engineers at the company at most—nothing compared to the decades or more software usually lives (an interesting observation is that timescale, I think, is proportional to the size of an organization). Don't underestimate momentum, but also don't underestimate changing circumstances, even on a small time horizon. The point is, stop looking at technology in a vacuum. There are many facets to consider. Likewise, decisions are not made in a vacuum. Part of this is just being an empathetic engineer. The corollary to this is you don't need to adopt every bleeding-edge tech that comes out to be successful, but the bigger point is software is about storytelling. The question you should be asking is how does your organization tell those stories? Are you deliberate or is it left to tribal knowledge and hearsay? Is it something you truly value and prioritize or simply a byproduct? Documentation is good, but the trouble with documentation is it's usually haphazard and stagnant. It's also usually documentation of how and not why. Documenting intent can go a long way, and understanding the why is a good way to develop empathy. Code survives us. There's a fantastic talk by Bryan Cantrill on oral tradition in software engineering (https://youtu.be/4PaWFYm0kEw) where he talks about this. People care about intent. Specifically, when you write software, people care what you think. As Bryan puts it, future generations of programmers want to understand your intent so they can abide by it, so we need to tell them what our intent was. We need to broadcast it. Good code comments are an example of this. They give you a narrative of not only what's going on, but why. When we write software, we write it for future generations, and that's the most underestimated thing in all of software. Documenting intent also allows you to document your values, and that allows the people who come after you to continue to uphold them. Storytelling in software is important. Without it, software archeology is simply the study of puzzles created by time and neglect. When an organization doesn't record its history, it's bound to repeat the same mistakes. A company's memory is comprised of its people, but the fact is people churn. Knowing how you got here often helps you with getting to where you want to be. Storytelling is how we transcend generational gaps and the inevitable changing of the old guard to the new guard in a maturing engineering organization. The same is true when we expand that to the entire industry. We're too memoryless—shipping code and not looking back, discovering everything old that is new again, and simply not appreciating our lineage. Beastie Bits 1st BSD Users Stockholm Meetup (https://www.meetup.com/en-US/BSD-Users-Stockholm/) Absolute FreeBSD, 3rd Edition draft completed (https://blather.michaelwlucas.com/archives/3020) Absolute FreeBSD, 3rd Edition Table of Contents (https://blather.michaelwlucas.com/archives/2995) t2k17 Hackathon Report: My first time (Aaron Bieber) (https://undeadly.org/cgi?action=article;sid=20170824193521) The release of pfSense 2.4.0 will be slightly delayed to apply patches for vulnerabilities in 3rd party packages that are part of pfSense (https://www.netgate.com/blog/no-plan-survives-contact-with-the-internet.html) Feedback/Questions Ben writes in that zrepl is in ports now (http://dpaste.com/1XMJYMH#wrap) Peter asks us about Netflix on BSD (http://dpaste.com/334WY4T#wrap) meka writes in about dhclient exiting (http://dpaste.com/3GSGKD3#wrap) ***

We recap EuroBSDcon in Paris, tell the story behind a pf PR, and show you how to do screencasting with OpenBSD. This episode was brought to you by Headlines Recap of EuroBSDcon 2017 in Paris, France (https://2017.eurobsdcon.org) EuroBSDcon was held in Paris, France this year, which drew record numbers this year. With over 300 attendees, it was the largest BSD event I have ever attended, and I was encouraged by the higher than expected number of first time attendees. The FreeBSD Foundation held a board meeting on Wednesday afternoon with the members who were in Paris. Topics included future conferences (including a conference kit we can mail to people who want to represent FreeBSD) and planning for next year. The FreeBSD Devsummit started on Thursday at the beautiful Mozilla Office in Paris. After registering and picking up our conference bag, everyone gathered for a morning coffee with lots of handshaking and greeting. We then gathered in the next room which had a podium with microphone, screens as well as tables and chairs. After developers sat down, Benedict opened the devsummit with a small quiz about France for developers to win a Mogics Power Bagel (https://www.mogics.com/?page_id=3824). 45 developers participated and DES won the item in the end. After introductions and collecting topics of interest from everyone, we started with the Work in Progress (WIP) session. The WIP session had different people present a topic they are working on in 7 minute timeslots. Topics ranged from FreeBSD Forwarding Performance, fast booting options, and a GELI patch under review to attach multiple providers. See their slides on the FreeBSD wiki (https://wiki.freebsd.org/DevSummit/201709). After lunch, the FreeBSD Foundation gave a general update on staff and funding, as well as a more focused presentation about our partnership with Intel. People were interested to hear what was done so far and asked a few questions to the Intel representative Glenn Weinberg. After lunch, developers worked quietly on their own projects. The mic remained open and occasionally, people would step forward and gave a short talk without slides or motivated a discussion of common interest. The day concluded with a dinner at a nice restaurant in Paris, which allowed to continue the discussions of the day. The second day of the devsummit began with a talk about the CAM-based SDIO stack by Ilya Bakulin. His work would allow access to wifi cards/modules on embedded boards like the Raspberry Pi Zero W and similar devices as many of these are using SDIO for data transfers. Next up was a discussion and Q&A session with the FreeBSD core team members who were there (missing only Benno Rice, Kris Moore, John Baldwin, and Baptiste Daroussin, the latter being busy with conference preparations). The new FCP (FreeBSD community proposals) were introduced for those who were not at BSDCan this year and the hows and whys about it. Allan and I were asked to describe our experiences as new members of core and we encouraged people to run for core when the next election happens. After a short break, Scott Long gave an overview of the work that's been started on NUMA (Non-Uniform Memory Architecture), what the goals of the project are and who is working on it. Before lunch, Christian Schwarz presented his work on zrepl, a new ZFS replication solution he developed using Go. This sparked interest in developers, a port was started (https://reviews.freebsd.org/D12462) and people suggested to Christian that he should submit his talk to AsiaBSDcon and BSDCan next year. Benedict had to leave before lunch was done to teach his Ansible tutorial (which was well attended) at the conference venue. There were organized dinners, for those two nights, quite a feat of organization to fit over 100 people into a restaurant and serve them quickly. On Saturday, there was a social event, a river cruise down the Seine. This took the form of a ‘standing' dinner, with a wide selection of appetizer type dishes, designed to get people to walk around and converse with many different people, rather than sit at a table with the same 6-8 people. I talked to a much larger group of people than I had managed to at the other dinners. I like having both dinner formats. We would also like to thank all of the BSDNow viewers who attended the conference and made the point of introducing themselves to us. It was nice to meet you all. The recordings of the live video stream from the conference are available immediately, so you can watch the raw versions of the talks now: Auditorium Keynote 1: Software Development in the Age of Heroes (https://youtu.be/4iR8g9-39LM?t=179) by Thomas Pornin (https://twitter.com/BearSSLnews) Tuning FreeBSD for routing and firewalling (https://youtu.be/4iR8g9-39LM?t=1660) by Olivier Cochard-Labbé (https://twitter.com/ocochardlabbe) My BSD sucks less than yours, Act I (https://youtu.be/4iR8g9-39LM?t=7040) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_) My BSD sucks less than yours, Act II (https://youtu.be/4iR8g9-39LM?t=14254) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_) Reproducible builds on NetBSD (https://youtu.be/4iR8g9-39LM?t=23351) by Christos Zoulas Your scheduler is not the problem (https://youtu.be/4iR8g9-39LM?t=26845) by Martin Pieuchot Keynote 2: A French story on cybercrime (https://youtu.be/4iR8g9-39LM?t=30540) by Éric Freyssinet (https://twitter.com/ericfreyss) Case studies of sandboxing base system with Capsicum (https://youtu.be/jqdHYEH_BQY?t=731) by Mariusz Zaborski (https://twitter.com/oshogbovx) OpenBSD's small steps towards DTrace (a tale about DDB and CTF) (https://youtu.be/jqdHYEH_BQY?t=6030) by Jasper Lievisse Adriaanse The Realities of DTrace on FreeBSD (https://youtu.be/jqdHYEH_BQY?t=13096) by George Neville-Neil (https://twitter.com/gvnn3) OpenSMTPD, current state of affairs (https://youtu.be/jqdHYEH_BQY?t=16818) by Gilles Chehade (https://twitter.com/PoolpOrg) Hoisting: lessons learned integrating pledge into 500 programs (https://youtu.be/jqdHYEH_BQY?t=21764) by Theo de Raadt Keynote 3: System Performance Analysis Methodologies (https://youtu.be/jqdHYEH_BQY?t=25463) by Brendan Gregg (https://twitter.com/brendangregg) Closing Session (https://youtu.be/jqdHYEH_BQY?t=29355) Karnak “Is it done yet ?” The never ending story of pkg tools (https://youtu.be/1hjzleqGRYk?t=71) by Marc Espie (https://twitter.com/espie_openbsd) A Tale of six motherboards, three BSDs and coreboot (https://youtu.be/1hjzleqGRYk?t=7498) by Piotr Kubaj and Katarzyna Kubaj State of the DragonFly's graphics stack (https://youtu.be/1hjzleqGRYk?t=11475) by François Tigeot From NanoBSD to ZFS and Jails – FreeBSD as a Hosting Platform, Revisited (https://youtu.be/1hjzleqGRYk?t=16227) by Patrick M. Hausen Bacula – nobody ever regretted making a backup (https://youtu.be/1hjzleqGRYk?t=20069) by Dan Langille (https://twitter.com/DLangille) Never Lose a Syslog Message (https://youtu.be/qX0BS4P65cQ?t=325) by Alexander Bluhm Running CloudABI applications on a FreeBSD-based Kubernetes cluster (https://youtu.be/qX0BS4P65cQ?t=5647) by Ed Schouten (https://twitter.com/EdSchouten) The OpenBSD web stack (https://youtu.be/qX0BS4P65cQ?t=13255) by Michael W. Lucas (https://twitter.com/mwlauthor) The LLDB Debugger on NetBSD (https://youtu.be/qX0BS4P65cQ?t=16835) by Kamil Rytarowski What's in store for NetBSD 8.0? (https://youtu.be/qX0BS4P65cQ?t=21583) by Alistair Crooks Louxor A Modern Replacement for BSD spell(1) (https://youtu.be/6Nen6a1Xl7I?t=156) by Abhinav Upadhyay (https://twitter.com/abhi9u) Portable Hotplugging: NetBSD's uvm_hotplug(9) API development (https://youtu.be/6Nen6a1Xl7I?t=5874) by Cherry G. Mathew Hardening pkgsrc (https://youtu.be/6Nen6a1Xl7I?t=9343) by Pierre Pronchery (https://twitter.com/khorben) Discovering OpenBSD on AWS (https://youtu.be/6Nen6a1Xl7I?t=14874) by Laurent Bernaille (https://twitter.com/lbernail) OpenBSD Testing Infrastructure Behind bluhm.genua.de (https://youtu.be/6Nen6a1Xl7I?t=18639) by Jan Klemkow The school of hard knocks – PT1 (https://youtu.be/8wuW8lfsVGc?t=276) by Sevan Janiyan (https://twitter.com/sevanjaniyan) 7 years of maintaining firefox, and still looking ahead (https://youtu.be/8wuW8lfsVGc?t=5321) by Landry Breuil Branch VPN solution based on OpenBSD, OSPF, RDomains and Ansible (https://youtu.be/8wuW8lfsVGc?t=12385) by Remi Locherer Running BSD on AWS (https://youtu.be/8wuW8lfsVGc?t=15983) by Julien Simon and Nicolas David Getting started with OpenBSD device driver development (https://youtu.be/8wuW8lfsVGc?t=21491) by Stefan Sperling A huge thanks to the organizers, program committee, and sponsors of EuroBSDCon. Next year, EuroBSDcon will be in Bucharest, Romania. *** The story of PR 219251 (https://www.sigsegv.be//blog/freebsd/PR219251) The actual story I wanted Kristof to tell, the pf bug he fixed at the Essen Hackathon earlier this summer. As I threatened to do in my previous post, I'm going to talk about PR 219251 for a bit. The bug report dates from only a few months ago, but the first report (that I can remeber) actually came from Shawn Webb on Twitter, of all places Despite there being a stacktrace it took quite a while (nearly 6 months in fact) before I figured this one out. It took Reshad Patuck managing to distill the problem down to a small-ish test script to make real progress on this. His testcase meant that I could get core dumps and experiment. It also provided valuable clues because it could be tweaked to see what elements were required to trigger the panic. This test script starts a (vnet) jail, adds an epair interface to it, sets up pf in the jail, and then reloads the pf rules on the host. Interestingly the panic does not seem to occur if that last step is not included. Obviously not the desired behaviour, but it seems strange. The instances of pf in the jails are supposed to be separate. We try to fetch a counter value here, but instead we dereference a bad pointer. There's two here, so already we need more information. Inspection of the core dump reveals that the state pointer is valid, and contains sane information. The rule pointer (rule.ptr) points to a sensible location, but the data is mostly 0xdeadc0de. This is the memory allocator being helpful (in debug mode) and writing garbage over freed memory, to make use-after-free bugs like this one easier to find. In other words: the rule has been free()d while there was still a state pointing to it. Somehow we have a state (describing a connection pf knows about) which points to a rule which no longer exists. The core dump also shows that the problem always occurs with states and rules in the default vnet (i.e. the host pf instance), not one of the pf instances in one of the vnet jails. That matches with the observation that the test script does not trigger the panic unless we also reload the rules on the host. Great, we know what's wrong, but now we need to work out how we can get into this state. At this point we're going to have to learn something about how rules and states get cleaned up in pf. Don't worry if you had no idea, because before this bug I didn't either. The states keep a pointer to the rule they match, so when rules are changed (or removed) we can't just delete them. States get cleaned up when connections are closed or they time out. This means we have to keep old rules around until the states that use them expire. When rules are removed pfunlinkrule() adds then to the Vpfunlinkedrules list (more on that funny V prefix later). From time to time the pf purge thread will run over all states and mark the rules that are used by a state. Once that's done for all states we know that all rules that are not marked as in-use can be removed (because none of the states use it). That can be a lot of work if we've got a lot of states, so pfpurgethread() breaks that up into smaller chuncks, iterating only part of the state table on every run. We iterate over all of our virtual pf instances (VNETFOREACH()), check if it's active (for FreeBSD-EN-17.08, where we've seen this code before) and then check the expired states with pfpurgeexpiredstates(). We start at state 'idx' and only process a certain number (determined by the PFTMINTERVAL setting) states. The pfpurgeexpiredstates() function returns a new idx value to tell us how far we got. So, remember when I mentioned the odd V_ prefix? Those are per-vnet variables. They work a bit like thread-local variables. Each vnet (virtual network stack) keeps its state separate from the others, and the V_ variables use a pointer that's changed whenever we change the currently active vnet (say with CURVNETSET() or CURVNETRESTORE()). That's tracked in the 'curvnet' variable. In other words: there are as many Vpfvnetactive variables as there are vnets: number of vnet jails plus one (for the host system). Why is that relevant here? Note that idx is not a per-vnet variable, but we handle multiple pf instances here. We run through all of them in fact. That means that we end up checking the first X states in the first vnet, then check the second X states in the second vnet, the third X states in the third and so on and so on. That of course means that we think we've run through all of the states in a vnet while we really only checked some of them. So when pfpurgeunlinkedrules() runs it can end up free()ing rules that actually are still in use because pfpurgethread() skipped over the state(s) that actually used the rule. The problem only happened if we reloaded rules in the host, because the active ruleset is never free()d, even if there are no states pointing to the rule. That explains the panic, and the fix is actually quite straightforward: idx needs to be a per-vnet variable, Vpfpurge_idx, and then the problem is gone. As is often the case, the solution to a fairly hard problem turns out to be really simple. As you might expect, finding the problem takes a lot more work that fixing it Thanks to Kristof for writing up this detailed post explaining how the problem was found, and what caused it. *** vBSDcon 2017: BSD at Work (https://www.ixsystems.com/blog/vbsdcon-2017-dexter/) The third biennial vBSDcon hosted by Verisign took place September 7th through 9th with the FreeBSD Developer Summit taking place the first day. vBSDcon and iXsystems' MeetBSD event have been alternating between the East and West coasts of the U.S.A. and these two events play vital roles in reaching Washington, DC-area and Bay Area/Silicon Valley audiences. Where MeetBSD serves many BSD Vendors, vBSDcon attracts a unique government and security industry demographic that isn't found anywhere else. Conference time and travel budgets are always limited and bringing these events to their attendees is a much-appreciated service provided by their hosts. The vBSDcon FreeBSD DevSummit had a strong focus on OpenZFS, the build system and networking with the FreeBSD 12 wish list of features in mind. How to best incorporate the steady flow of new OpenZFS features into FreeBSD such as dataset-level encryption was of particular interest. This feature from a GNU/Linux-based storage vendor is tribute to the growth of the OpenZFS community which is vital in light of the recent “Death of Solaris and ZFS” at Oracle. There has never been more demand for OpenZFS on FreeBSD and the Oracle news further confirms our collective responsibility to meet that demand. The official conference opened with my talk on “Isolated BSD Build Environments” in which I explained how the bhyve hypervisor can be used to effortlessly tour FreeBSD 5.0-onward and build specific source releases on demand to trace regressions to their offending commit. I was followed by a FreeNAS user who made the good point that FreeNAS is an exemplary “entry vector” into Unix and Enterprise Storage fundamentals, given that many of the vectors our generation had are gone. Where many of us discovered Unix and the Internet via console terminals at school or work, smart phones are only delivering the Internet without the Unix. With some irony, both iOS and Android are Unix-based yet offer few opportunities for their users to learn and leverage their Unix environments. The next two talks were The History and Future of Core Dumps in FreeBSD by Sam Gwydir and Using pkgsrc for multi-platform deployments in heterogeneous environments by G. Clifford Williams. I strongly recommend that anyone wanting to speak at AsiaBSDCon read Sam's accompanying paper on core dumps because I consider it the perfect AsiaBSDCon topic and his execution is excellent. Core dumps are one of those things you rarely think about until they are a DROP EVERYTHING! priority. G. Clifford's talk was about what I consider a near-perfect BSD project: pkgsrc, the portable BSD package manager. I put it up there with OpenSSH and mandoc as projects that have provided significant value to other Open Source operating systems. G. Clifford's real-world experiences are perfectly inline with vBSDcon's goal to be more production-oriented than other BSDCons. Of the other talks, any and all Dtrace talks are always appreciated and George Neville-Neil's did not disappoint. He based it on his experiences with the Teach BSD project which is bringing FreeBSD-based computer science education to schools around the world. The security-related talks by John-Mark Gurney, Dean Freeman and Michael Shirk also represented vBSDcon's consideration of the local community and made a convincing point that the BSDs should make concerted efforts to qualify for Common Criteria, FIPS, and other Government security requirements. While some security experts will scoff at these, they are critical to the adoption of BSD-based products by government agencies. BSD Now hosts Allan Jude and Benedict Reuschling hosted an OpenZFS BoF and Ansible talk respectively and I hosted a bhyve hypervisor BoF. The Hallway Track and food at vBSDcon were excellent and both culminated with an after-dinner dramatic reading of Michael W. Lucas' latest book that raised money for the FreeBSD Foundation. A great time was had by all and it was wonderful to see everyone! News Roundup FreeBSD 10.4-RC2 Available (https://lists.freebsd.org/pipermail/freebsd-stable/2017-September/087848.html) FreeBSD 10.4 will be released soon, this is the last chance to find bugs before the official release is cut. Noteworthy Changes Since 10.4-RC1: Given that the amd64 disc1 image was overflowing, more of the base components installed into the disc1 (live) file systems had to be disabled. Most notably, this removed the compiler toolchain from the disc1 images. All disabled tools are still available with the dvd1 images, though. The aesni(4) driver now no longer shares a single FPU context across multiple sessions in multiple threads, addressing problems seen when employing aesni(4) for ipsec(4). Support for netmap(4) by the ixgbe(4) driver has been brought into line with the netmap(4) API present in stable/10. Also, ixgbe(4) now correctly handles VFs in its netmap(4) support again instead of treating these as PFs. During the creation of amd64 and i386 VM images, etcupdate(8) and mergemaster(8) databases now are bootstrapped, akin to what happens along the extraction of base.txz as part of a new installation via bsdinstall(8). This change allows for both of these tools to work out-of-box on the VM images and avoids errors seen when upgrading these images via freebsd-update(8). If you are still on the stable/10 branch, you should test upgrading to 10.4, and make sure there are no problems with your workload Additional testing specifically of the features that have changed since 10.4-BETA1 would also be most helpful This will be the last release from the stable/10 branch *** OpenBSD changes of note 628 (https://www.tedunangst.com/flak/post/openbsd-changes-of-note-628) EuroBSDCon in two weeks. Be sure to attend early and often. Many and various documentation improvements for libcrypto. New man pages, rewrites, expanded bugs sections, and more. Only allow upward migration in vmd. There's a README for the syspatch build system if you want to run your own. Move the kernel relinking code from /etc/rc into a seperate script usable by syspatch. Kernel patches can now be reduced to just the necessary files. Make the callers of sogetopt() responsible for allocating memory. Now allocation and free occur in the same place. Use waitpid() instead of wait() in most programs to avoid accidentally collecting the wrong child. Have cu call isatty() before making assumptions. Switch mandoc rendering of mathematical symbols and greek letters from trying to imitate the characters' graphical shapes, which resulted in unintelligible renderings in many cases, to transliterations conveying the characters' meanings. Update libexpat to 2.2.4. Fix copying partial UTF-8 characters. Sigh, here we go again. Work around bug in F5's handling of the supported elliptic curves extension. RFC 4492 only defines elliptic_curves for ClientHello. However, F5 is sending it in ServerHello. We need to skip over it since our TLS extension parsing code is now more strict. After a first install, run syspatch -c to check for patches. If SMAP is present, clear PSL_AC on kernel entry and interrupt so that only the code in copy{in,out}* that need it run with it set. Panic if it's set on entry to trap() or syscall(). Prompted by Maxime Villard's NetBSD work. Errata. New drivers for arm: rktemp, mvpinctrl, mvmpic, mvneta, mvmdio, mvpxa, rkiic, rkpmic. No need to exec rm from within mandoc. We know there's exactly one file and directory to remove. Similarly with running cmp. Revert to Mesa 13.0.6 to hopefully address rendering issues a handful of people have reported with xpdf/fvwm on ivy bridge with modesetting driver. Rewrite ALPN extension using CBB/CBS and the new extension framework. Rewrite SRTP extension using CBB/CBS and the new extension framework. Revisit 2q queue sizes. Limit the hot queue to 1/20th the cache size up to a max of 4096 pages. Limit the warm and cold queues to half the cache. This allows us to more effectively notice re-interest in buffers instead of losing it in a large hot queue. Add glass console support for arm64. Probably not yet for your machine, though. Replace heaps of hand-written syscall stubs in ld.so with a simpler framework. 65535 is a valid port to listen on. When xinit starts an X server that listens only on UNIX socket, prefer DISPLAY=unix:0 rather than DISPLAY=:0. This will prevent applications from ever falling back to TCP if the UNIX socket connection fails (such as when the X server crashes). Reverted. Add -z and -Z options to apmd to auto suspend or hibernate when low on battery. Remove the original (pre-IETF) chacha20-poly1305 cipher suites. Add urng(4) which supports various USB RNG devices. Instead of adding one driver per device, start bundling them into a single driver. Remove old deactivated pledge path code. A replacement mechanism is being brewed. Fix a bug from the extension parsing rewrite. Always parse ALPN even if no callback has been installed to prevent leaving unprocessed data which leads to a decode error. Clarify what is meant by syslog priorities being ordered, since the numbers and priorities are backwards. Remove a stray setlocale() from ksh, eliminating a lot of extra statically linked code. Unremove some NPN symbols from libssl because ports software thinks they should be there for reasons. Fix saved stack location after resume. Somehow clang changed it. Resume works again on i386. Improve error messages in vmd and vmctl to be more informative. Stop building the miniroot installer for OMAP3 Beagleboards. It hasn't worked in over a year and nobody noticed. Have the callers of sosetopt() free the mbuf for symmetry. On octeon, let the kernel use the hardware FPU even if emulation is compiled in. It's faster. Fix support for 486DX CPUs by not calling cpuid. I used to own a 486. Now I don't. Merge some drm fixes from linux. Defer probing of floppy drives, eliminating delays during boot. Better handling of probes and beacons and timeouts and scans in wifi stack to avoid disconnects. Move mutex, condvar, and thread-specific data routes, pthreadonce, and pthreadexit from libpthread to libc, along with low-level bits to support them. Let's thread aware (but not actually threaded) code work with just libc. New POSIX xlocale implementation. Complete as long as you only use ASCII and UTF-8, as you should. Round and round it goes; when 6.2 stops, nobody knows. A peak at the future? *** Screencasting with OpenBSD (http://eradman.com/posts/screencasting.html) USB Audio Any USB microphone should appear as a new audio device. Here is the dmesg for my mic by ART: uaudio0 at uhub0 port 2 configuration 1 interface 0 "M-One USB" rev 1.10/0.01 addr 2 uaudio0: audio rev 1.00, 8 mixer controls audio1 at uaudio0 audioctl can read off all of the specific characterisitcs of this device $ audioctl -f /dev/audio1 | grep record mode=play,record record.rate=48000 record.channels=1 record.precision=16 record.bps=2 record.msb=1 record.encoding=slinear_le record.pause=0 record.active=0 record.block_size=1960 record.bytes=0 record.errors=0 Now test the recording from the second audio device using aucat(1) aucat -f rsnd/1 -o file.wav If the device also has a headset audio can be played through the same device. aucat -f rsnd/1 -i file.wav Screen Capture using Xvfb The rate at which a framebuffer for your video card is a feature of the hardware and software your using, and it's often very slow. x11vnc will print an estimate of the banwidth for the system your running. x11vnc ... 09/05/2012 22:23:45 fb read rate: 7 MB/sec This is about 4fps. We can do much better by using a virtual framebuffer. Here I'm setting up a new screen, setting the background color, starting cwm and an instance of xterm Xvfb :1 -screen 0 720x540x16 & DISPLAY=:1 xsetroot -solid steelblue & DISPLAY=:1 cwm & DISPLAY=:1 xterm +sb -fa Hermit -fs 14 & Much better! Now we're up around 20fps. x11vnc -display :1 & ... 11/05/2012 18:04:07 fb read rate: 168 MB/sec Make a connection to this virtual screen using raw encoding to eliminate time wasted on compression. vncviewer localhost -encodings raw A test recording with sound then looks like this ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 ~/out.avi Note: always stop the recording and playback using q, not Ctrl-C so that audio inputs are shut down properly. Screen Capture using Xephyr Xephyr is perhaps the easiest way to run X with a shadow framebuffer. This solution also avoids reading from the video card's RAM, so it's reasonably fast. Xephyr -ac -br -noreset -screen 800x600 :1 & DISPLAY=:1 xsetroot -solid steelblue & DISPLAY=:1 cwm & DISPLAY=:1 xrdb -load ~/.Xdefaults & DISPLAY=:1 xterm +sb -fa "Hermit" -fs 14 & Capture works in exactally the same way. This command tries to maintain 12fps. ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi To capture keyboard and mouse input press Ctrl then Shift. This is very handy for using navigating a window manager in the nested X session. Arranging Windows I have sometimes found it helpful to launch applications and arrange them in a specific way. This will open up a web browser listing the current directory and position windows using xdotool DISPLAY=:1 midori "file:///pwd" & sleep 2 DISPLAY=:1 xdotool search --name "xterm" windowmove 0 0 DISPLAY=:1 xdotool search --class "midori" windowmove 400 0 DISPLAY=:1 xdotool search --class "midori" windowsize 400 576 This will position the window precisely so that it appears to be in a tmux window on the right. Audio/Video Sync If you find that the audio is way out of sync with the video, you can ajust the start using the -ss before the audio input to specify the number of seconds to delay. My final recording command line, that delays the audio by 0.5 seconds, writing 12fps ffmpeg -ss 0.5 -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi Sharing a Terminal with tmux If you're trying to record a terminal session, tmux is able to share a session. In this way a recording of an X framebuffer can be taken without even using the screen. Start by creating the session. tmux -2 -S /tmp/tmux0 Then on the remote side connect on the same socket tmux -2 -S /tmp/tmux0 attach Taking Screenshots Grabbing a screenshots on Xvfb server is easily accomplished with ImageMagick's import command DISPLAY=:1 import -window root screenshot.png Audio Processing and Video Transcoding The first step is to ensure that the clip begins and ends where you'd like it to. The following will make a copy of the recording starting at time 00:00 and ending at 09:45 ffmpeg -i interactive-sql.avi -vcodec copy -acodec copy -ss 00:00:00 -t 00:09:45 interactive-sql-trimmed.avi mv interactive-sql-trimmed.avi interactive-sql.avi Setting the gain correctly is very important with an analog mixer, but if you're using a USB mic there may not be a gain option; simply record using it's built-in settings and then adjust the levels afterwards using a utility such as normalize. First extact the audio as a raw PCM file and then run normalize ffmpeg -i interactive-sql.avi -c:a copy -vn audio.wav normalize audio.wav Next merge the audio back in again ffmpeg -i interactive-sql.avi -i audio.wav -map 0:0 -map 1:0 -c copy interactive-sql-normalized.avi The final step is to compress the screencast for distribution. Encoding to VP8/Vorbis is easy: ffmpeg -i interactive-sql-normalized.avi -c:v libvpx -b:v 1M -c:a libvorbis -q:a 6 interactive-sql.webm H.264/AAC is tricky. For most video players the color space needs to be set to yuv420p. The -movflags puts the index data at the beginning of the file to enable streaming/partial content requests over HTTP: ffmpeg -y -i interactive-sql-normalized.avi -c:v libx264 -preset slow -crf 14 -pix_fmt yuv420p -movflags +faststart -c:a aac -q:a 6 interactive-sql.mp4 TrueOS @ Ohio Linuxfest '17! (https://www.trueos.org/blog/trueos-ohio-linuxfest-17/) Dru Lavigne and Ken Moore are both giving presentations on Saturday the 30th. Sit in and hear about new developments for the Lumina and FreeNAS projects. Ken is offering Lumina Rising: Challenging Desktop Orthodoxy at 10:15 am in Franklin A. Hear his thoughts about the ideas propelling desktop environment development and how Lumina, especially Lumina 2, is seeking to offer a new model of desktop architecture. Elements discussed include session security, application dependencies, message handling, and operating system integration. Dru is talking about What's New in FreeNAS 11 at 2:00 pm in Franklin D. She'll be providing an overview of some of the new features added in FreeNAS 11.0, including: Alert Services Starting specific services at boot time AD Monitoring to ensure the AD service restarts if disconnected A preview of the new user interface support for S3-compatible storage and the bhyve hypervisor She's also giving a sneak peek of FreeNAS 11.1, which has some neat features: A complete rewrite of the Jails/Plugins system as FreeNAS moves from warden to iocage Writing new plugins with just a few lines of code A brand new asynchronous middleware API Who's going? Attending this year are: Dru Lavigne (dlavigne): Dru leads the technical documentation team at iX, and contributes heavily to open source documentation projects like FreeBSD, FreeNAS, and TrueOS. Ken Moore (beanpole134): Ken is the lead developer of Lumina and a core contributor to TrueOS. He also works on a number of other Qt5 projects for iXsystems. J.T. Pennington (q5sys): Some of you may be familiar with his work on BSDNow, but J.T. also contributes to the TrueOS, Lumina, and SysAdm projects, helping out with development and general bug squashing. *** Beastie Bits Lumina Development Preview: Theme Engine (https://www.trueos.org/blog/lumina-development-preview-theme-engine/) It's happening! Official retro Thinkpad lappy spotted in the wild (https://www.theregister.co.uk/2017/09/04/retro_thinkpad_spotted_in_the_wild/) LLVM libFuzzer and SafeStack ported to NetBSD (https://blog.netbsd.org/tnf/entry/llvm_libfuzzer_and_safestack_ported) Remaining 2017 FreeBSD Events (https://www.freebsdfoundation.org/news-and-events/event-calendar/2017-openzfs-developer-summit/) *** Feedback/Questions Andrew - BSD Teaching Material (http://dpaste.com/0YTT0VP) Seth - Switching to Tarsnap after Crashplan becomes no more (http://dpaste.com/1SK92ZX#wrap) Thomas - Native encryption in ZFS (http://dpaste.com/02KD5FX#wrap) Coding Cowboy - Coding Cowboy - Passwords and clipboards (http://dpaste.com/31K0E40#wrap) ***

In this episode, we take a look at the reimplementation of NetBSD using a Microkernel, check out what makes DHCP faster, and see what high-process count support for DragonflyBSD has to offer, and we answer the questions you've always wanted to ask us. This episode was brought to you by Headlines A Reimplementation Of Netbsd Using a Microkernel (http://theembeddedboard.review/a-reimplementation-of-netbsd-using-a-microkernel-part-1-of-2/) Minix author Andy Tanenbaum writes in Part 1 of a-reimplementation-of-netbsd-using-a-microkernel (http://theembeddedboard.review/a-reimplementation-of-netbsd-using-a-microkernel-part-1-of-2/) Based on the MINIX 3 microkernel, we have constructed a system that to the user looks a great deal like NetBSD. It uses pkgsrc, NetBSD headers and libraries, and passes over 80% of the KYUA tests). However, inside, the system is completely different. At the bottom is a small (about 13,000 lines of code) microkernel that handles interrupts, message passing, low-level scheduling, and hardware related details. Nearly all of the actual operating system, including memory management, the file system(s), paging, and all the device drivers run as user-mode processes protected by the MMU. As a consequence, failures or security issues in one component cannot spread to other ones. In some cases a failed component can be replaced automatically and on the fly, while the system is running, and without user processes noticing it. The talk will discuss the history, goals, technology, and status of the project. Research at the Vrije Universiteit has resulted in a reimplementation of NetBSD using a microkernel instead of the traditional monolithic kernel. To the user, the system looks a great deal like NetBSD (it passes over 80% of the KYUA tests). However, inside, the system is completely different. At the bottom is a small (about 13,000 lines of code) microkernel that handles interrupts, message passing, low-level scheduling, and hardware related details. Nearly all of the actual operating system, including memory management, the file system(s), paging, and all the device drivers run as user-mode processes protected by the MMU. As a consequence, failures or security issues in one component cannot spread to other ones. In some cases a failed component can be replaced automatically and on the fly, while the system is running. The latest work has been adding live update, making it possible to upgrade to a new version of the operating system WITHOUT a reboot and without running processes even noticing. No other operating system can do this. The system is built on MINIX 3, a derivative of the original MINIX system, which was intended for education. However, after the original author, Andrew Tanenbaum, received a 2 million euro grant from the Royal Netherlands Academy of Arts and Sciences and a 2.5 million euro grant from the European Research Council, the focus changed to building a highly reliable, secure, fault tolerant operating system, with an emphasis on embedded systems. The code is open source and can be downloaded from www.minix3.org. It runs on the x86 and ARM Cortex V8 (e.g., BeagleBones). Since 2007, the Website has been visited over 3 million times and the bootable image file has been downloaded over 600,000 times. The talk will discuss the history, goals, technology, and status of the project. Part 2 (http://theembeddedboard.review/a-reimplementation-of-netbsd-using-a-microkernel-part-2-of-2/) is also available. *** Rapid DHCP: Or, how do Macs get on the network so fast? (https://cafbit.com/post/rapid_dhcp_or_how_do/) One of life's minor annoyances is having to wait on my devices to connect to the network after I wake them from sleep. All too often, I'll open the lid on my EeePC netbook, enter a web address, and get the dreaded "This webpage is not available" message because the machine is still working on connecting to my Wi-Fi network. On some occasions, I have to twiddle my thumbs for as long as 10-15 seconds before the network is ready to be used. The frustrating thing is that I know it doesn't have to be this way. I know this because I have a Mac. When I open the lid of my MacBook Pro, it connects to the network nearly instantaneously. In fact, no matter how fast I am, the network comes up before I can even try to load a web page. My curiosity got the better of me, and I set out to investigate how Macs are able to connect to the network so quickly, and how the network connect time in other operating systems could be improved. I figure there are three main categories of time-consuming activities that occur during network initialization: Link establishment. This is the activity of establishing communication with the network's link layer. In the case of Wi-Fi, the radio must be powered on, the access point detected, and the optional encryption layer (e.g. WPA) established. After link establishment, the device is able to send and receive Ethernet frames on the network. Dynamic Host Configuration Protocol (DHCP). Through DHCP handshaking, the device negotiates an IP address for its use on the local IP network. A DHCP server is responsible for managing the IP addresses available for use on the network. Miscellaneous overhead. The operating system may perform any number of mundane tasks during the process of network initialization, including running scripts, looking up preconfigured network settings in a local database, launching programs, etc. My investigation thus far is primarily concerned with the DHCP phase, although the other two categories would be interesting to study in the future. I set up a packet capture environment with a spare wireless access point, and observed the network activity of a number of devices as they initialized their network connection. For a worst-case scenario, let's look at the network activity captured while an Android tablet is connecting: This tablet, presumably in the interest of "optimization", is initially skipping the DHCP discovery phase and immediately requesting its previous IP address. The only problem is this is a different network, so the DHCP server ignores these requests. After about 4.5 seconds, the tablet stubbornly tries again to request its old IP address. After another 4.5 seconds, it resigns itself to starting from scratch, and performs the DHCP discovery needed to obtain an IP address on the new network. In all fairness, this delay wouldn't be so bad if the device was connecting to the same network as it was previously using. However, notice that the tablet waits a full 1.13 seconds after link establishment to even think about starting the DHCP process. Engineering snappiness usually means finding lots of small opportunities to save a few milliseconds here and there, and someone definitely dropped the ball here. In contrast, let's look at the packet dump from the machine with the lightning-fast network initialization, and see if we can uncover the magic that is happening under the hood: The key to understanding the magic is the first three unicast ARP requests. It looks like Mac OS remembers certain information about not only the last connected network, but the last several networks. In particular, it must at least persist the following tuple for each of these networks: > 1. The Ethernet address of the DHCP server > 2. The IP address of the DHCP server > 3. Its own IP address, as assigned by the DHCP server During network initialization, the Mac transmits carefully crafted unicast ARP requests with this stored information. For each network in its memory, it attempts to send a request to the specific Ethernet address of the DHCP server for that network, in which it asks about the server's IP address, and requests that the server reply to the IP address which the Mac was formerly using on that network. Unless network hosts have been radically shuffled around, at most only one of these ARP requests will result in a response—the request corresponding to the current network, if the current network happens to be one of the remembered networks. This network recognition technique allows the Mac to very rapidly discover if it is connected to a known network. If the network is recognized (and presumably if the Mac knows that the DHCP lease is still active), it immediately and presumptuously configures its IP interface with the address it knows is good for this network. (Well, it does perform a self-ARP for good measure, but doesn't seem to wait more than 13ms for a response.) The DHCP handshaking process begins in the background by sending a DHCP request for its assumed IP address, but the network interface is available for use during the handshaking process. If the network was not recognized, I assume the Mac would know to begin the DHCP discovery phase, instead of sending blind requests for a former IP address as the Galaxy Tab does. The Mac's rapid network initialization can be credited to more than just the network recognition scheme. Judging by the use of ARP (which can be problematic to deal with in user-space) and the unusually regular transmission intervals (a reliable 1.0ms delay between each packet sent), I'm guessing that the Mac's DHCP client system is entirely implemented as tight kernel-mode code. The Mac began the IP interface initialization process a mere 10ms after link establishment, which is far faster than any other device I tested. Android devices such as the Galaxy Tab rely on the user-mode dhclient system (part of the dhcpcd package) dhcpcd program, which no doubt brings a lot of additional overhead such as loading the program, context switching, and perhaps even running scripts. The next step for some daring kernel hacker is to implement a similarly aggressive DHCP client system in the Linux kernel, so that I can enjoy fast sign-on speeds on my Android tablet, Android phone, and Ubuntu netbook. There already exists a minimal DHCP client implementation in the Linux kernel, but it lacks certain features such as configuring the DNS nameservers. Perhaps it wouldn't be too much work to extend this code to support network recognition and interface with a user-mode daemon to handle such auxillary configuration information received via DHCP. If I ever get a few spare cycles, maybe I'll even take a stab at it. You can also find other ways of optimizing the dhclient program and how it works in the dhclient tutorial on Calomel.org (https://calomel.org/dhclient.html). *** BSDCam Trip Report (https://www.freebsdfoundation.org/blog/bsdcam-2017-trip-report-michael-lucas/) Over the decades, FreeBSD development and coordination has shifted from being purely on-line to involving more and more in-person coordination and cooperation. The FreeBSD Foundation sponsors a devsummit right before BSDCan, EuroBSDCon, and AsiaBSDCon, so that developers traveling to the con can leverage their airfare and hammer out some problems. Yes, the Internet is great for coordination, but nothing beats a group of developers spending ten minutes together to sketch on a whiteboard and figuring out exactly how to make something bulletproof. In addition to the coordination efforts, though, conference devsummits are hierarchical. There's a rigid schedule, with topics decided in advance. Someone leads the session. Sessions can be highly informative, passionate arguments, or anything in between. BSDCam is… a little different. It's an invaluable part of the FreeBSD ecosystem. However, it's something that I wouldn't normally attend. But right now, is not normal. I'm writing a new edition of Absolute FreeBSD. To my astonishment, people have come to rely on this book when planning their deployments and operations. While I find this satisfying, it also increases the pressure on me to get things correct. When I wrote my first FreeBSD book back in 2000, a dozen mailing lists provided authoritative information on FreeBSD development. One person could read every one of those lists. Today, that's not possible—and the mailing lists are only one narrow aspect of the FreeBSD social system. Don't get me wrong—it's pretty easy to find out what people are doing and how the system works. But it's not that easy to find out what people will be doing and how the system will work. If this book is going to be future-proof, I needed to leave my cozy nest and venture into the wilds of Cambridge, England. Sadly, the BSDCam chair agreed with my logic, so I boarded an aluminum deathtrap—sorry, a “commercial airliner”—and found myself hurtled from Detroit to Heathrow. And one Wednesday morning, I made it to the William Gates building of Cambridge University, consciousness nailed to my body by a thankfully infinite stream of proper British tea. BSDCam attendance is invitation only, and the facilities can only handle fifty folks or so. You need to be actively working on FreeBSD to wrangle an invite. Developers attend from all over the world. Yet, there's no agenda. Robert Watson is the chair, but he doesn't decide on the conference topics. He goes around the room and asks everyone to introduce themselves, say what they're working on, and declare what they want to discuss during the conference. The topics of interest are tallied. The most popular topics get assigned time slots and one of the two big rooms. Folks interested in less popular topics are invited to claim one of the small breakout rooms. Then the real fun begins. I started by eavesdropping in the virtualization workshop. For two hours, people discussed FreeBSD's virtualization needs, strengths, and weaknesses. What needs help? What should this interface look like? What compatibility is important, and what isn't? By the end of the session, the couple dozen people had developed a reasonable consensus and, most importantly, some folks had added items to their to-do lists. Repeat for a dozen more topics. I got a good grip on what's really happening with security mitigation techniques, FreeBSD's cloud support, TCP/IP improvements, advances in teaching FreeBSD, and more. A BSDCan devsummit presentation on packaging the base system is informative, but eavesdropping on two dozen highly educated engineers arguing about how to nail down the final tidbits needed to make that a real thing is far more educational. To my surprise, I was able to provide useful feedback for some sessions. I speak at a lot of events outside of the FreeBSD world, and was able to share much of what I hear at Linux conferences. A tool that works well for an experienced developer doesn't necessarily work well for everyone. Every year, I leave BSDCan tired. I left BSDCam entirely exhausted. These intense, focused discussions stretched my brain. But, I have a really good idea where key parts of FreeBSD development are actually headed. This should help future-proof the new Absolute FreeBSD, as much as any computer book can be future-proof. Plus, BSDCam throws the most glorious conference dinner I've ever seen. I want to thank Robert Watson for his kind invitation, and the FreeBSD Foundation for helping defray the cost of this trip Interview - The BSDNow Crew As a kid, what did you dream of to become as an adult? JT: An Astronaut BR: I wanted to be a private detective, because of all the crime novels that I read back then. I didn't get far with it. However, I think the structured analysis skills (who did what, when, and such) help me in debugging and sysadmin work. AJ: Didn't think about it much How do you manage to stay organized day to day with so much things you're actively doing each day? (Day job, wife/girlfriend, conferences, hobbies, friends, etc.) JT: Who said I was organized? BR: A lot of stuff in my calendar as reminders, open browser tabs as “to read later” list. A few things like task switching when getting stuck helps. Also, focus on a single goal for the day, even though there will be distractions. Slowly, but steadily chip away at the things you're working on. Rather than to procrastinate and put things back to review later, get started early with easy things for a big task and then tackle the hard part. Often, things look totally chaotic and unmanageable, until you start working on them. AJ: I barely manage. Lots of Google Calendar reminders, and the entire wall of my office is covered in whiteboard sheet todo lists. I use pinboard.in to deal with finding and organizing bookmarks. Write things down, don't trust your memory. What hobbies outside of IT do you have? JT: I love photography, but I do that Professional part time, so I'm not sure if that counts as a hobby anymore. I guess it'd have to be working in the garage on my cars. BR: I do Tai Chi to relax once a week in a group, but can also do it alone, pretty much everywhere. Way too much Youtube watching and browsing the web. I did play some games before studying at the university and I'm still proud that I could control it to the bare minimum not to impact my studies. A few “lapses” from time to time, revisiting the old classics since the newer stuff won't run on my machines anyway. Holiday time is pretty much spent for BSD conferences and events, this is where I can relax and talk with like-minded people from around the world, which is fascinating. Plus, it gets me to various places and countries I never would have dared to visit on my own. AJ: I play a few video games, and I like to ski, although I don't go very often as most of my vacation time is spent hanging out with my BSD friends at various conferences How do you relax? JT: What is this word ‘relax' and what does it mean? BR: My Tai Chi plays a big part in it I guess. I really calms you and the constant stream of thoughts for a while. It also gives you better clarity of what's important in life. Watching movies, sleeping long. AJ: Usually watching TV or Movies. Although I have taken to doing most of my TV watching on my exercise bike now, but it is still mentally relaxing If FreeBSD didn't exist, which BSD flavour would you use? Why? JT: I use TrueOS, but if FreeBSD didn't exist, that project might not either… so… My other choice would be HardenedBSD, but since it's also based on FreeBSD I'm in the same dillema. BR: I once installed NetBSD to see what It can do. If FreeBSD wouldn't exist, I would probably try my luck with it. OpenBSD is also appealing, but I've never installed it. AJ: When I started using FreeBSD in 2000, the only other BSD I had heard of at the time was OpenBSD. If FreeBSD wasn't around, I don't think the world would look like it does, so it is hard to speculate. If any of the BSD's weren't around and you had to use Linux, which camp would belong to? (Redhat, SUSE, Debian, Ubuntu, Gentoo?) JT: I learned Linux in the mid 90s using Slackware, which I used consistently up until the mid 2000s, when I joined the PuppyLinux community and eventually became a developer (FYI, Puppy was/is/can be based on Slackware -- its complicated). So I'd go back to using either Slackware or PuppyLinux. BR: I tried various Linux distributions until I landed at Debian. I used is pretty extensively as my desktop OS at home, building custom kernels and packages to install them until I discovered FreeBSD. I ran both side by side for a few months for learning until one day I figured out that I had not booted Debian in a while, so I switched completely. AJ: The first Linux I played with was Slackware, and it is the most BSD like, but the bits of Linux I learned in school were Redhat and so I can somewhat wrap my head around it, although now that they are changing everything to systemd, all of that old knowledge is more harmful than useful. Are you still finding yourself in need to use Windows/Mac OS? Why? JT: I work part time as a professional Photographer, so I do use Windows for my photography work. While I can do everything I need to do in Linux, it comes down to being pragmatic about my time. What takes me several hours to accomplish in Linux I can accomplish in 20 minutes on Windows. BR: I was a long time Windows-only user before my Unix days. But back when Vista was about to come out and I needed a new laptop, my choice was basically learning to cope with Vistas awful features or learn MacOS X. I did the latter, it increased my productivity since it's really a good Unix desktop experience (at least, back then). I only have to use Windows at work from time to time as I manage our Windows Terminal server, which keeps the exposure low enough and I only connect to it to use a certain app not available for the Mac or the BSDs. AJ: I still use Windows to play games, for a lot of video conferencing, and to produce BSD Now. Some of it could be done on BSD but not as easily. I have promised myself that I will switch to 100% BSD rather than upgrade to Windows 10, so we'll see how that goes. Please describe your home networking setup. Router type, router OS, router hardware, network segmentation, wifi apparatus(es), other devices connected, and anything else that might be interesting about your home network. BR: Very simple and boring: Apple Airport Express base station and an AVM FritzBox for DNS, DHCP, and the link to my provider. A long network cable to my desktop machine. That I use less and less often. I just bought an RPI 3 for some home use in the future to replace it. Mostly my brother's and my Macbook Pro's are connected, our phones and the iPad of my mother. AJ: I have a E3-1220 v3 (dual 3.1ghz + HT) with 8 GB of ram, and 4x Intel gigabit server NICs as my router, and it runs vanilla FreeBSD (usually some snapshot of -current). I have 4 different VLANs, Home, Office, DMZ, and Guest WiFi. WiFi is served via a tiny USB powered device I bought in Tokyo years ago, it serves 3 different SSIDs, one for each VLAN except the DMZ. There are ethernet jacks in every room wired for 10 gigabit, although the only machines with 10 gigabit are my main workstation, file server, and some machines in the server rack. There are 3 switches, one for the house (in the laundry room), one for the rack, and one for 10gig stuff. There is a rack in the basement spare bedroom, it has 7 servers in it, mostly storage for live replicas of customer data for my company. How do guys manage to get your work done on FreeBSD desktops? What do you do when you need to a Linux or Windows app that isn't ported, or working? I've made several attempts to switch to FreeBSD, but each attempt failed because of tools not being available (e.g. Zoom, Dropbox, TeamViewer, Crashplan) or broken (e.g. VirtualBox). BR: I use VIrtualBox for everything that is not natively available or Windows-only. Unfortunately, that means no modern games. I mostly do work in the shell when I'm on FreeBSD and when it has to be a graphical application, then I use Fluxbox as the DE. I want to get work done, not look at fancy eye-candy that get's boring after a while. Deactivated the same stuff on my mac due to the same reason. I look for alternative software online, but my needs are relatively easy to satisfy as I'm not doing video editing/rendering and such. AJ: I generally find that I don't need these apps. I use Firefox, Thunderbird, OpenSSH, Quassel, KomodoEdit, and a few other apps, so my needs are not very demanding. It is annoying when packages are broken, but I usually work around this with boot environments, and being able to just roll back to a version that worked for a few days until the problem is solved. I do still have access to a windows machine for the odd time I need specific VPN software or access to Dell/HP etc out-of-band management tools. Which desktop environments are your favorite, and why? For example, I like i3, Xfce, and I'm drawn to Lumina's ethos, but so far always seem to end up back on Xfc because of its ease of use, flexibility, and dashing good looks. JT: As a Lumina Desktop developer, I think my preference is obvious. ;) I am also a long timeOpenBox user, so I have a soft place in my heart for that as well. BR: I use Fluxbox when I need to work with a lot of windows or an application demands X11. KDE and others are too memory heavy for me and I rarely use even 20% of the features they provide. AJ: I was a long time KDE user, but I have adopted Lumina. I find it fast, and that it gets out of my way and lets me do what I want. It had some annoyances early on, but I've nagged the developers into making it work for me. Which command-line shells do you prefer, why, and how (if at all) have you customised the environment or prompt? BR: I use zsh, but without all the fancy stuff you can find online. It might make you more productive, yes. But again, I try to keep things simple. I'm slowly learning tmux and want to work more in it in the future. I sometimes look at other BSD people's laptops and am amazed at what they do with window-management in tmux. My prompt looks like this: bcr@Voyager:~> 20:20 17-08-17 Put this in your .zshrc to get the same result: PROMPT='%n@%m:%~>' RPROMPT='%T %D' AJ: I started using tcsh early on, because it was the shell on the first box I had access to, and because one of the first things I read in “BSD Hacks” was how to enable ‘typo correction”, which made my life a lot better especially on dial up in the early days. My shell prompt looks like this: allan@CA-TOR1-02:/usr/home/allan% What is one thing (or more) missing in FreeBSD you would import from another project or community? Could be tech, process, etc. JT: AUFS from Linux BR: Nohup from Illumos where you can detach an already running process and put it in the background. I often forget that and I'm not in tmux when that happens, so I can see myself use that feature a lot. AJ: Zones (more complete Jails) from IllumOS how do you manage your time to learn about and work on FreeBSD? Does your work/employment enable what you do, or are your contributions mainly done in private time? JT: These days I'm mostly learning things I need for work, so it just falls into something I'm doing while working on work projects. BR: We have a lot of time during the semester holidays to learn on our own, it's part of the idea of being in a university to keep yourself updated, at least for me. Especially in the fast moving world of IT. I also read a lot in my free time. My interests can shift sometimes, but then I devour everything I can find on the topic. Can be a bit excessive, but has gotten me where I am now and I still need a lot to learn (and want to). Since I work with FreeBSD at work (my owndoing), I can try out many things there. AJ: My work means a spend a lot of time working with FreeBSD, but not that much time working ON it. My contributions are mostly done outside of work, but as I own the company I do get more flexibility to take time off for conferences and other FreeBSD related stuff. we know we can bribe Michael W Lucas with gelato (good gelato that is), but what can we use to bribe you guys? Like when I want to have Allan to work on fixing a bug which prevents me from running ZFS on this fancy rock64 board? BR: Desserts of various kinds. AJ: I am probably not the right person to look at your rock64 board. Most people in the project have taken to bribing me with chocolate. In general, my todo list is so long, the best way is a trade, you take this task and I'll take that task. Is your daily mobile device iOS, Android, Windows Mobile, or other? Why? JT: These days I'm using Android on my Blackberry Priv, but until recently I was still a heavy user of Sailfish OS. I would use SailfishOS everyday, if I could find a phone with a keyboard that I could run it on. BR: iOS on the iPhone 7 currently. Never used an Android phone, saw it on other people's devices and what they can do with it (much more). But the infrequent security updates (if any at all) keep me away from it. AJ: I have a Google Nexus 6 (Android 7.1). I wanted the ‘pure' Android experience, and I had been happy with my previous Nexus S. I don't run a custom OS/ROM or anything because I use the phone to verify that video streams work on an ‘average users device'. I am displeased that support for my device will end soon. I am not sure what device I will get next, but it definitely won't be an iPhone. News Roundup Beta Update - Request for (more) Testing (http://undeadly.org/cgi?action=article&sid=20170808065718&mode=flat&count=30) https://beta.undeadly.org/ has received an update. The most significant changes include: The site has been given a less antiquated "look". (As the topic icons have been eliminated, we are no longer seeking help with those graphics.) The site now uses a moderate amount of semantic HTML5. Several bugs in the HTML fragment validator (used for submissions and comments) have been fixed. To avoid generating invalid HTML, submission content which fails validation is no longer displayed in submission/comment previews. Plain text submissions are converted to HTML in a more useful fashion. (Instead of just converting each EOL to , the converter now generates proper paragraphs and interprets two or more consecutive EOLs as indicating a paragraph break.) The redevelopment remains a work-in-progress. Many thanks to those who have contributed! As before, constructive feedback would be appreciated. Of particular interest are reports of bugs in behaviour (for example, in the HTML validator or in authentication) that would preclude the adoption of the current code for the main site. High-process-count support added to master (http://lists.dragonflybsd.org/pipermail/users/2017-August/313552.html) We've fixed a number of bottlenecks that can develop when the number of user processes runs into the tens of thousands or higher. One thing led to another and I said to myself, "gee, we have a 6-digit PID, might as well make it work to a million!". With the commits made today, master can support at least 900,000 processes with just a kern.maxproc setting in /boot/loader.conf, assuming the machine has the memory to handle it. And, in fact, as today's machines start to ratchet up there in both memory capacity and core count, with fast storage (NVMe) and fast networking (10GigE and higher), even in consumer boxes, this is actually something that one might want to do. With AMD's threadripper and EPYC chips now out, the IntelAMD cpu wars are back on! Boasting up to 32 cores (64 threads) per socket and two sockets on EPYC, terabytes of ram, and motherboards with dual 10GigE built-in, the reality is that these numbers are already achievable in a useful manner. In anycase, I've tested these changes on a dual-socket xeon. I can in-fact start 900,000 processes. They don't get a whole lot of cpu and running 'ps' would be painful, but it works and the system is still responsive from the shell with all of that going on. xeon126# uptime 1:42PM up 9 mins, 3 users, load averages: 890407.00, 549381.40, 254199.55 In fact, judging from the memory use, these minimal test processes only eat around 60KB each. 900,000 of them ate only 55GB on a 128GB machine. So even a million processes is not out of the question, depending on the cpu requirements for those processes. Today's modern machines can be stuffed with enormous amounts of memory. Of course, our PIDs are currently limited to 6 digits, so a million is kinda the upper limit in terms of discrete user processes (verses pthreads which are less restricted). I'd rather not go to 7 digits (yet). CFT: Driver for generic MS Windows 7/8/10 - compatible USB HID multi-touch touchscreens (https://lists.freebsd.org/pipermail/freebsd-current/2017-August/066783.html) Following patch [1] adds support for generic MS Windows 7/8/10 - compatible USB HID multi-touch touchscreens via evdev protocol. It is intended to be a native replacement of hid-multitouch.c driver found in Linux distributions and multimedia/webcamd port. Patch is made for 12-CURRENT and most probably can be applied to recent 11-STABLE and 11.1-RELEASE (not tested) How to test" 1. Apply patch [1] 2. To compile this driver into the kernel, place the following lines into your kernel configuration file: device wmt device usb device evdev Alternatively, to load the driver as a module at boot time, place the following line in loader.conf(5): wmt_load="YES" 3. Install x11-drivers/xf86-input-evdev or x11-drivers/xf86-input-libinput port 4. Tell XOrg to use evdev or libinput driver for the device: ``` Section "ServerLayout" InputDevice "TouchScreen0" "SendCoreEvents" EndSection Section "InputDevice" Identifier "TouchScreen0" Driver "evdev" # Driver "libinput" Option "Device" "/dev/input/eventXXX" EndSection ``` Exact value of "/dev/input/eventXXX" can be obtained with evemu-record utility from devel/evemu. Note1: Currently, driver does not support pens or touchpads. Note2: wmt.ko should be kld-loaded before uhid driver to take precedence over it! Otherwise uhid can be kld-unloaded after loading of wmt. wmt review: https://reviews.freebsd.org/D12017 Raw diff: https://reviews.freebsd.org/D12017.diff *** Beastie Bits BSDMag Programing Languages Infographic (https://bsdmag.org/programm_history/) t2k17 Hackathon Report: Bob Beck on buffer cache tweaks, libressl and pledge progress (http://undeadly.org/cgi?action=article&sid=20170815171854) New FreeBSD Journal (https://www.freebsdfoundation.org/past-issues/resource-control/) NetBSD machines at Open Source Conference 2017 Kyoto (http://mail-index.netbsd.org/netbsd-advocacy/2017/08/10/msg000744.html) *** Feedback/Questions Dan - HDD question (http://dpaste.com/3H6TDJV) Benjamin - scrub of death (http://dpaste.com/10F086V) Jason - Router Opinion (http://dpaste.com/2D9102K) Sohrab - Thanks (http://dpaste.com/1XYYTWF) ***

This week on BSD Now, we clear up some ZFS FUD, show you how to write a NetBSD kernel module, and cover DragonflyBSD on the desktop. This episode was brought to you by Headlines ZFS is the best file system (for now) (http://blog.fosketts.net/2017/07/10/zfs-best-filesystem-now/) In my ongoing effort to fight misinformation and FUD about ZFS, I would like to go through this post in detail and share my thoughts on the current state and future of OpenZFS. The post starts with: ZFS should have been great, but I kind of hate it: ZFS seems to be trapped in the past, before it was sidelined it as the cool storage project of choice; it's inflexible; it lacks modern flash integration; and it's not directly supported by most operating systems. But I put all my valuable data on ZFS because it simply offers the best level of data protection in a small office/home office (SOHO) environment. Here's why. When ZFS first appeared in 2005, it was absolutely with the times, but it's remained stuck there ever since. The ZFS engineers did a lot right when they combined the best features of a volume manager with a “zettabyte-scale” filesystem in Solaris 10 The skies first darkened in 2007, as NetApp sued Sun, claiming that their WAFL patents were infringed by ZFS. Sun counter-sued later that year, and the legal issues dragged on. The lawsuit was resolved, and it didn't really impede ZFS. Some say it is the reason that Apple didn't go with ZFS, but there are other theories too. By then, Sun was hitting hard times and Oracle swooped in to purchase the company. This sowed further doubt about the future of ZFS, since Oracle did not enjoy wide support from open source advocates. Yes, Oracle taking over Sun and closing the source for ZFS definitely seemed like a setback at the time, but the OpenZFS project was started and active development has continued as an ever increasing pace. As of today, more than half of the code in OpenZFS has been written since the fork from the last open version of Oracle ZFS. the CDDL license Sun applied to the ZFS code was https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/ (judged incompatible) with the GPLv2 that covers Linux, making it a non-starter for inclusion in the world's server operating system. That hasn't stopped the ZFS-on-Linux project, or Ubuntu… Although OpenSolaris continued after the Oracle acquisition, and FreeBSD embraced ZFS, this was pretty much the extent of its impact outside the enterprise. Sure, NexentaStor and http://blog.fosketts.net/2008/09/15/greenbytes-embraces-extends-zfs/ (GreenBytes) helped push ZFS forward in the enterprise, but Oracle's lackluster commitment to Sun in the datacenter started having an impact. Lots of companies have adopted OpenZFS for their products. Before OpenZFS, there were very few non-Sun appliances that used ZFS, now there are plenty. OpenZFS Wiki: Companies with products based on OpenZFS (http://open-zfs.org/wiki/Companies) OpenZFS remains little-changed from what we had a decade ago. Other than the fact that half of the current code did not exist a decade ago… Many remain skeptical of deduplication, which hogs expensive RAM in the best-case scenario. This is one of the weaker points in ZFS. As it turns out, the demand for deduplication is actually not that strong. Most of the win can be had with transparent compression. However, there are a number of suggested designs to work around the dedup problems: Dedup Ceiling: Set a limit on the side of the DDT and just stop deduping new unique blocks when this limit is reached. Allocation Classes: A feature being developed by Intel for a supercomputer, will allow different types of data to be classified, and dedicated vdevs (or even metaslabs within a vdev), to be dedicated to that class of data. This could be extended to having the DDT live on a fast device like an PCIe NVMe, combined with the Dedup Ceiling when the device is full. DDT Pruning: Matt Ahrens described a design where items in the DDT with only a single reference, would be expired in an LRU type fashion, to allow newer blocks to live in the DDT in hopes that they would end up with more than a single reference. This doesn't cause bookkeeping problems since when a block is about to be freed, if it is NOT listed in the DDT, ZFS knows it was never deduplicated, so the current block must be the only reference, and it can safely be freed. This provides a best case scenario compared to Dedup Ceiling, since blocks that will deduplicate well, are likely to be written relatively close together, whereas the chance to a dedup match on a very old block is much lower. And I do mean expensive: Pretty much every ZFS FAQ flatly declares that ECC RAM is a must-have and 8 GB is the bare minimum. In my own experience with FreeNAS, 32 GB is a nice amount for an active small ZFS server, and this costs $200-$300 even at today's prices. As we talked about a few weeks ago, ECC is best, but it is not required. If you want your server to stay up for a long time, to be highly available, you'll put ECC in it. Don't let a lack of ECC stop you from using ZFS, you are just putting your data at more risk. The scrub of death is a myth. ZFS does not ‘require' lots of ram. Your NAS will work happily with 8 GB instead of 32 GB of RAM. Its cache hit ratio will be much lower, so performance will be worse. It won't be able to buffer as many writes, so performance will be worse. Copy-on-Write has some drawbacks, data tends to get scattered and fragmented across the drives when it is written gradually. The ARC (RAM Cache) lessens the pain of this, and allows ZFS to batch incoming writes up into nice contiguous writes. ZFS purposely alternates between reading and writing, since both are faster when the other is not happening. So writes are batched up until there is too much dirty data, or the timeout expires. Then reads are held off while the bulk linear write finishes as quickly as possible, and reads are resumed. Obviously all of this works better and more efficiently in larger batches, which you can do if you have more RAM. ZFS can be tuned to use less RAM, and if you do not have a lot of RAM, or you have a lot of other demand on your RAM, you should do that tuning. And ZFS never really adapted to today's world of widely-available flash storage: Although flash can be used to support the ZIL and L2ARC caches, these are of dubious value in a system with sufficient RAM, and ZFS has no true hybrid storage capability. It's laughable that the ZFS documentation obsesses over a few GB of SLC flash when multi-TB 3D NAND drives are on the market. And no one is talking about NVMe even though it's everywhere in performance PC's. Make up your mind, is 32GB of ram too expensive or not… the L2ARC exists specifically for the case where it is not possible to just install more RAM. Be it because there are no more slots, of limits of the processor, or limits of your budget. The SLOG is optional, but it never needs to be very big. A number of GBs of SLC flash is all you need, it is only holding writes that have not been flushed to the regular storage devices yet. The reason the documentation talks about SLC specifically is because your SLOG needs a very high write endurance, something never the newest NVMe devices cannot yet provide. Of course you can use NVMe devices with ZFS, lots of people do. All flash ZFS arrays are for sale right now. Other than maybe a little tuning of the device queue depths, ZFS just works and there is nothing to think about. However, to say there is nothing happening in this space is woefully inaccurate. The previously mentioned allocation classes code can be used to allocate metadata (4 KB blocks) on SSD or NVMe, while allocating bulk storage data (up to 16 MB blocks) on spinning disks. Extended a bit beyond what Intel is building for their super computer, this will basically create hybrid storage for ZFS. With the metaslab classes feature, it will even be possible to mix classes on the same device, grouping small allocations and large allocations in different areas, decreasing fragmentation. Then there's the question of flexibility, or lack thereof. Once you build a ZFS volume, it's pretty much fixed for life. There are only three ways to expand a storage pool: Replace each and every drive in the pool with a larger one (which is great but limiting and expensive) It depends on your pool layout. If you design with this in mind using ZFS Mirrors, it can be quite useful Add a stripe on another set of drives (which can lead to imbalanced performance and redundancy and a whole world of potential stupid stuff) The unbalanced LUNs performance issues were sorted out in 2013-2016. 2014: OpenZFS Allocation Performance (http://open-zfs.org/w/images/3/31/Performance-George_Wilson.pdf) 2016: OpenZFS space allocation: doubling performance on large and fragmented pools (http://www.bsdcan.org/2016/schedule/events/710.en.html) These also mostly solved the performance issues when a pool gets full, you can run a lot closer to the edge now Build a new pool and “zfs send” your datasets to it (which is what I do, even though it's kind of tricky) This is one way to do it, yes. There is another way coming, but I can't talk about it just yet. Look for big news later this year. Apart from option 3 above, you can't shrink a ZFS pool. Device removal is arriving now. It will not work for RAIDZ*, but for Mirrors and Stripes you will be able to remove a device. I've probably made ZFS sound pretty unappealing right about now. It was revolutionary but now it's startlingly limiting and out of touch with the present solid-state-dominated storage world. I don't feel like ZFS is out of touch with solid state. Lots of people are running SSD only pools. I will admit the tiered storage options in ZFS are a bit limited still, but there is a lot of work being done to overcome this. After all, reliably storing data is the only thing a storage system really has to do. All my important data goes on ZFS, from photos to music and movies to office files. It's going to be a long time before I trust anything other than ZFS! + I agree. + ZFS has a great track record of doing its most important job, keeping your data safe. + Work is ongoing to make ZFS more performance, and more flexible. The import thing is that this work is never allowed to compromise job #1, keeping your data safe. + Hybrid/tiered storage features, re-RAID-ing, are coming + There is a lot going on with OpenZFS, check out the notes from the last two OpenZFS Developer Summits just to get an idea of what some of those things are: 2015 (http://open-zfs.org/wiki/OpenZFS_Developer_Summit_2015) & 2016 (http://open-zfs.org/wiki/OpenZFS_Developer_Summit_2016) Some highlights: Compressed ARC Compressed send/recv ABD (arc buf scatter/gather) ZFS Native Encryption (scrub/resilver, send/recv, etc without encryption keys loaded) Channel Programs (do many administrative operations as one atomic transaction) Device Removal Redacted send/recv ZStandard Compression TRIM Support (FreeBSD has its own, but this will be more performant and universal) Faster Scrub/Resilver (https://youtu.be/SZFwv8BdBj4) Declustered RAID (https://youtu.be/MxKohtFSB4M) Allocation Classes (https://youtu.be/28fKiTWb2oM) Multi-mount protection (for Active/Passive failover) Zpool Checkpoint (undo almost anything) Even more Improved Allocator Performance vdev spacemap log ZIL performance improvements (w/ or w/o SLOG) Persistent L2ARC What I don't think the author of this article understands is how far behind every other filesystem is. 100s of Engineer years have gone into OpenZFS, and the pace is accelerating. I don't see how BtrFS can ever catch up, without a huge cash infusion. Writing a NetBSD kernel module (https://saurvs.github.io/post/writing-netbsd-kern-mod/) Kernel modules are object files used to extend an operating system's kernel functionality at run time. In this post, we'll look at implementing a simple character device driver as a kernel module in NetBSD. Once it is loaded, userspace processes will be able to write an arbitrary byte string to the device, and on every successive read expect a cryptographically-secure pseudorandom permutation of the original byte string. You will need the NetBSD Source Code. This doc (https://www.netbsd.org/docs/guide/en/chap-fetch.html) will explain how you can get it. The article gives an easy line by line walkthrough which is easy to follow and understand. The driver implements the bare minimum: open, close, read, and write, plus the module initialization function It explains the differences in how memory is allocated and freed in the kernel It also describes the process of using UIO to copy data back and forth between userspace and the kernel Create a Makefile, and compile the kernel module Then, create a simple userspace program to use the character device that the kernel module creates All the code is available here (https://github.com/saurvs/rperm-netbsd) *** DragonFlyBSD Desktop! (https://functionallyparanoid.com/2017/07/11/dragonflybsd-desktop/) If you read my last post (https://functionallyparanoid.com/2017/06/30/boot-all-the-things/), you know that I set up a machine (Thinkpad x230) with UEFI and four operating systems on it. One, I had no experience with – DragonFlyBSD (other than using Matthew Dillon's C compiler for the Amiga back in the day!) and so it was uncharted territory for me. After getting the install working, I started playing around inside of DragonFlyBSD and discovered to my delight that it was a great operating system with some really unique features – all with that BSD commitment to good documentation and a solid coupling of kernel and userland that doesn't exist (by design) in Linux. So my goal for my DragonFlyBSD desktop experience was to be as BSD as I possibly could. Given that (and since I'm the maintainer of the port on OpenBSD ), I went with Lumina as the desktop environment and XDM as the graphical login manager. I have to confess that I really like the xfce terminal application so I wanted to make sure I had that as well. Toss in Firefox, libreOffice and ownCloud sync client and I'm good to go! OK. So where to start. First, we need to get WiFi and wired networking happening for the console at login. To do that, I added the following to /etc/rc.conf: wlans_iwn0=”wlan0″ ifconfig_wlan0=”WPA DHCP” ifconfig_em0=”DHCP” I then edited /etc/wpa_supplicant.conf to put in the details of my WiFi network: network={ ssid=”MY-NETWORK-NAME” psk=”my-super-secret-password” } A quick reboot showed that both wired and wireless networking were functional and automatically were assigned IP addresses via DHCP. Next up is to try getting into X with whatever DragonFlyBSD uses for its default window manager. A straight up “startx” met with, shall we say, less than stellar results. Therefore, I used the following command to generate a simple /etc/X11/xorg.conf file: # Xorg -configure # cp /root/xorg.conf.new /etc/X11/xorg.conf With that file in place, I could get into the default window manager, but I had no mouse. After some searching and pinging folks on the mailing list, I was able to figure out what I needed to do. I added the following to my /etc/rc.conf file: moused_enable=”YES” moused_type=”auto” moused_port=”/dev/psm0″ I rebooted (I'm sure there is an easier way to get the changes but I don't know it… yet) and was able to get into a basic X session and have a functional mouse. Next up, installing and configuring Lumina! To do that, I went through the incredibly torturous process of installing Lumina: # pkg install lumina Wow! That was really, really hard. I might need to pause here to catch my breath.

This week on BSD Now, we review the EuroBSDcon schedule, we explore the mysteries of Docker on OpenBSD, and show you how to run PostgreSQL on ZFS. This episode was brought to you by Headlines EuroBSDcon 2017 - Talks & Schedule published (https://2017.eurobsdcon.org/2017/05/26/talks-schedule-published/) The EuroBSDcon website was updated with the tutorial and talk schedule for the upcoming September conference in Paris, France. Tutorials on the 1st day: Kirk McKusick - An Introduction to the FreeBSD Open-Source Operating System, George Neville-Neil - DTrace for Developers, Taylor R Campbell - How to untangle your threads from a giant lock in a multiprocessor system Tutorials on the 2nd day: Kirk continues his Introduction lecture, Michael Lucas - Core concepts of ZFS (half day), Benedict Reuschling - Managing BSD systems with Ansible (half day), Peter Hessler - BGP for developers and sysadmins Talks include 3 keynotes (2 on the first day, beginning and end), another one at the end of the second day by Brendan Gregg Good mixture of talks of the various BSD projects Also, a good amount of new names and faces Check out the full talk schedule (https://2017.eurobsdcon.org/talks-schedule/). Registration is not open yet, but will be soon. *** OpenBSD on the Xiaomi Mi Air 12.5" (https://jcs.org/2017/05/22/xiaomiair) The Xiaomi Mi Air 12.5" (https://xiaomi-mi.com/notebooks/xiaomi-mi-notebook-air-125-silver/) is a basic fanless 12.5" Ultrabook with good build quality and decent hardware specs, especially for the money: while it can usually be had for about $600, I got mine for $489 shipped to the US during a sale about a month ago. Xiaomi offers this laptop in silver and gold. They also make a 13" version but it comes with an NVidia graphics chip. Since these laptops are only sold in China, they come with a Chinese language version of Windows 10 and only one or two distributors that carry them ship to the US. Unfortunately that also means they come with practically no warranty or support. Hardware > The Mi Air 12.5" has a fanless, 6th generation (Skylake) Intel Core m3 processor, 4Gb of soldered-on RAM, and a 128Gb SATA SSD (more on that later). It has a small footprint of 11.5" wide, 8" deep, and 0.5" thick, and weighs 2.3 pounds. > A single USB-C port on the right-hand side is used to charge the laptop and provide USB connectivity. A USB-C ethernet adapter I tried worked fine in OpenBSD. Whether intentional or not, a particular design touch I appreciated was that the USB-C port is placed directly to the right of the power button on the keyboard, so you don't have to look or feel around for the port when plugging in the power cable. > A single USB 3 type-A port is also available on the right side next to the USB-C port. A full-size HDMI port and a headphone jack are on the left-hand side. It has a soldered-on Intel 8260 wireless adapter and Bluetooth. The webcam in the screen bezel attaches internally over USB. > The chassis is all aluminum and has sufficient rigidity in the keyboard area. The 12.5" 1920x1080 glossy IPS screen has a fairly small bezel and while its hinge is properly weighted to allow opening the lid with one hand (if you care about that kind of thing), the screen does have a bit of top-end wobble when open, especially when typing on another laptop on the same desk. > The keyboard has a roomy layout and a nice clicky tactile with good travel. It is backlit, but with only one backlight level. When enabled via Fn+F10 (which is handled by the EC, so no OpenBSD support required), it will automatically shut off after not typing for a short while, automatically turning back once a key is pressed. Upgrades > An interesting feature of the Mi Air is that it comes with a 128Gb SATA SSD but also includes an open PCI-e slot ready to accept an NVMe SSD. > I upgraded mine with a Samsung PM961 256Gb NVMe SSD (left), and while it is possible to run with both drives in at the same time, I removed the Samsung CM871a 128Gb SATA (right) drive to save power. > The bottom case can be removed by removing the seven visible screws, in addition to the one under the foot in the middle back of the case, which just pries off. A spudger tool is needed to release all of the plastic attachment clips along the entire edge of the bottom cover. > Unfortunately this upgrade proved to be quite time consuming due to the combination of the limited UEFI firmware on the Mi Air and a bug in OpenBSD. A Detour into UEFI Firmware Variables > Unlike a traditional BIOS where one can boot into a menu and configure the boot order as well as enabling and disabling options such as "USB Hard Drive", the InsydeH2O UEFI firmware on the Xiaomi Air only provides the ability to adjust the boot order of existing devices. Any change or addition of boot devices must be done from the operating system, which is not possible under OpenBSD. > I booted to a USB key with OpenBSD on it and manually partitioned the new NVME SSD, then rsynced all of the data over from the old drive, but the laptop would not boot to the new NVME drive, instead showing an error message that there was no bootable OS. > Eventually I figured out that the GPT table that OpenBSD created on the NVMe disk was wrong due to a [one-off bug in the nvme driver](https://github.com/openbsd/src/commit/dc8298f669ea2d7e18c8a8efea509eed200cb989) which was causing the GPT table to be one sector too large, causing the backup GPT table to be written in the wrong location (and other utilities under Linux to write it over the OpenBSD area). I'm guessing the UEFI firmware would fail to read the bad GPT table on the disk that the boot variable pointed to, then declare that disk as missing, and then remove any variables that pointed to that disk. OpenBSD Support > The Mi Air's soldered-on Intel 8260 wireless adapter is supported by OpenBSD's iwm driver, including 802.11n support. The Intel sound chip is recognized by the azalia driver. > The Synaptics touchpad is connected via I2C, but is not yet supported. I am actively hacking on my dwiic driver to make this work and the touchpad will hopefully operate as a Windows Precision Touchpad via imt so I don't have to write an entirely new Synaptics driver. > Unfortunately since OpenBSD's inteldrm support that is ported from Linux is lagging quite a bit behind, there is no kernel support for Skylake and Kaby Lake video chips. Xorg works at 1920x1080 through efifb so the machine is at least usable, but X is not very fast and there is a noticeable delay when doing certain redrawing operations in xterm. Screen backlight can be adjusted through my OpenBSD port of intel_backlight. Since there is no hardware graphics support, this also means that suspend and resume do not work because nothing is available to re-POST the video after resume. Having to use efifb also makes it impossible to adjust the screen gamma, so for me, I can't use redshift for comfortable night-time hacking. Flaws > Especially taking into account the cheap price of the laptop, it's hard to find faults with the design. One minor gripe is that the edges of the case along the bottom are quite sharp, so when carrying the closed laptop, it can feel uncomfortable in one's hands. > While all of those things could be overlooked, unfortunately there is also a critical flaw in the rollover support in the keyboard/EC on the laptop. When typing certain combinations of keys quickly, such as holding Shift and typing "NULL", one's fingers may actually hold down the Shift, N, and U keys at the same time for a very brief moment before releasing N. Normally the keyboard/EC would recognize U being pressed after N is already down and send an interrupt for the U key. Unfortunately on this laptop, particular combinations of three keys do not interrupt for the third key at all until the second key is lifted, usually causing the third key not to register at all if typed quickly. I've been able to reproduce this problem in OpenBSD, Linux, and Windows, with the combinations of at least Shift+N+U and Shift+D+F. Holding Shift and typing the two characters in sequence quickly enough will usually fail to register the final character. Trying the combinations without Shift, using Control or Alt instead of Shift, or other character pairs does not trigger the problem. This might be a problem in the firmware on the Embedded Controller, or a defect in the keyboard circuitry itself. As I mentioned at the beginning, getting technical support for this machine is difficult because it's only sold in China. Docker on OpenBSD 6.1-current (https://medium.com/@dave_voutila/docker-on-openbsd-6-1-current-c620513b8110) Dave Voutila writes: So here's the thing. I'm normally a macOS user…all my hardware was designed in Cupertino, built in China. But I'm restless and have been toying with trying to switch my daily machine over to a non-macOS system sort of just for fun. I find Linux messy, FreeBSD not as Apple-laptop-friendly as it should be, and Windows a non-starter. Luckily, I found a friend in Puffy. Switching some of my Apple machines over to dual-boot OpenBSD left a gaping hole in my workflow. Luckily, all the hard work the OpenBSD team has done over the last year seems to have plugged it nicely! OpenBSD's hypervisor support officially made it into the 6.1 release, but after some experimentation it was rather time consuming and too fragile to get a Linux guest up and running (i.e. basically the per-requisite for Docker). Others had reported some success starting with QEMU and doing lots of tinkering, but after a wasted evening I figured I'd grab the latest OpenBSD snapshot and try what the openbsd-misc list suggested was improved Linux support in active development. 10 (11) Steps to docker are provided Step 0 — Install the latest OpenBSD 6.1 snapshot (-current) Step 1 — Configure VMM/VMD Step 2 — Grab an Alpine Linux ISO Step 3 — Make a new virtual disk image Step 4 — Boot Alpine's ISO Step 5 — Inhale that fresh Alpine air Step 6 — Boot Alpine for Reals Step 7 — Install Docker Step 8 — Make a User Step 9 — Ditch the Serial Console Step 10 — Test out your Docker instance I haven't done it yet, but I plan on installing docker-compose via Python's pip package manager. I prefer defining containers in the compose files. PostgreSQL + ZFS Best Practices and Standard Procedures (https://people.freebsd.org/~seanc/postgresql/scale15x-2017-postgresql_zfs_best_practices.pdf) Slides from Sean Chittenden's talk about PostgreSQL and ZFS at Scale 15x this spring Slides start with a good overview of Postgres and ZFS, and how to use them together To start, it walks through the basics of how PostgreSQL interacts with the filesystem (any filesystem) Then it shows the steps to take a good backup of PostgreSQL, then how to do it even better with ZFS Then an intro to ZFS, and how Copy-on-Write changes host PostgreSQL interacts with the filesystem Overview of how ZFS works ZFS Tuning tips: Compression, Recordsize, atime, when to use mostly ARC vs sharedbuffer, plus pgrepack Followed by a discussion of the reliability of SSDs, and their Bit Error Rate (BER) A good SSD has a 4%/year chance of returning the wrong data. A cheap SSD 34% If you put 20 SSDs in a database server, that means 58% (Good SSDs) to 99.975% (Lowest quality commercially viable SSD) chance of an error per year Luckily, ZFS can detect and correct these errors This applies to all storage, not just SSDs, every device fails More Advice: Use quotas and reservations to avoid running out of space Schedule Periodic Scrubs One dataset per database Backups: Live demo of rm -rf'ing the database and getting it back Using clones to test upgrades on real data Naming Conventions: Use a short prefix not on the root filesystem (e.g. /db) Encode the PostgreSQL major version into the dataset name Give each PostgreSQL cluster its own dataset (e.g. pgdb01) Optional but recommended: one database per cluster Optional but recommended: one app per database Optional but recommended: encode environment into DB name Optional but recommended: encode environment into DB username using ZFS Replication Check out the full detailed PDF and implement a similar setup for your database needs *** News Roundup TrueOS Evolving Its "Stable" Release Cycle (https://www.trueos.org/blog/housekeeping-update-infrastructure-trueos-changes/) TrueOS is reformulating its Stable branch based on feedback from users. The goal is to have a “release” of the stable branch every 6 months, for those who do not want to live on the edge with the rapid updates of the full rolling release Most of the TrueOS developers work for iX Systems in their Tennessee office. Last month, the Tennessee office was moved to a different location across town. As part of the move, we need to move all our servers. We're still getting some of the infrastructure sorted before moving the servers, so please bear with us as we continue this process. As we've continued working on TrueOS, we've heard a significant portion of the community asking for a more stable “STABLE” release of TrueOS, maybe something akin to an old PC-BSD version release. In order to meet that need, we're redefining the TrueOS STABLE branch a bit. STABLE releases are now expected to follow a six month schedule, with more testing and lots of polish between releases. This gives users the option to step back a little from the “cutting edge” of development, but still enjoy many of the benefits of the “rolling release” style and the useful elements of FreeBSD Current. Critical updates like emergency patches and utility bug fixes are still expected to be pushed to STABLE on a case-by-case basis, but again with more testing and polish. This also applies to version updates of the Lumina and SysAdm projects. New, released work from those projects will be tested and added to STABLE outside the 6 month window as well. The UNSTABLE branch continues to be our experimental “cutting edge” track, and users who want to follow along with our development and help us or FreeBSD test new features are still encouraged to follow the UNSTABLE track by checking that setting in their TrueOS Update Manager. With boot environments, it will be easy to switch back and forth, so you can have the best of both worlds. Use the latest bleeding edge features, but knowing you can fall back to the stable branch with just a reboot As TrueOS evolves, it is becoming clearer that one role of the system is to function as a “test platform” for FreeBSD. In order to better serve this role, TrueOS will support both OpenRC and the FreeBSD RC init systems, giving users the choice to use either system. While the full functionality isn't quite ready for the next STABLE update, it is planned for addition after the last bit of work and testing is complete. Stay tuned for an upcoming blog post with all the details of this change, along with instructions how to switch between RC and OpenRC. This is the most important change for me. I used TrueOS as an easy way to run the latest version of -CURRENT on my laptop, to use it as a user, but also to do development. When TrueOS deviates from FreeBSD too much, it lessens the power of my expertise, and complicates development and debugging. Being able to switch back to RC, even if it takes another minute to boot, will bring TrueOS back to being FreeBSD + GUI and more by default, instead of a science project. We need both of those things, so having the option, while more work for the TrueOS team, I think will be better for the entire community *** Logical Domains on SunFire T2000 with OpenBSD/sparc64 (http://www.h-i-r.net/2017/05/logical-domains-on-sunfire-t2000-with.html) A couple of years ago, I picked up a Sun Fire T2000. This is a 2U rack mount server. Mine came with four 146GB SAS drives, a 32-core UltraSPARC T1 CPU and 32GB of RAM. Sun Microsystems incorporated Logical Domains (LDOMs) on this class of hardware. You don't often need 32 threads and 32GB of RAM in a single server. LDOMs are a kind of virtualization technology that's a bit closer to bare metal than vmm, Hyper-V, VirtualBox or even Xen. It works a bit like Xen, though. You can allocate processor, memory, storage and other resources to virtual servers on-board, with a blend of firmware that supports the hardware allocation, and some software in userland (on the so-called primary or control domain, similar to Xen DomU) to control it. LDOMs are similar to what IBM calls Logical Partitions (LPARs) on its Mainframe and POWER series computers. My day job from 2006-2010 involved working with both of these virtualization technologies, and I've kind of missed it. While upgrading OpenBSD to 6.1 on my T2000, I decided to delve into LDOM support under OpenBSD. This was pretty easy to do, but let's walk through it Resources: The ldomctl(8) man page (http://man.openbsd.org/OpenBSD-current/man8/sparc64/ldomctl.8) tedu@'s write-up on Flak (for a different class of server) (http://www.tedunangst.com/flak/post/OpenBSD-on-a-Sun-T5120) A Google+ post by bmercer@ (https://plus.google.com/101694200911870273983/posts/jWh4rMKVq97) Once you get comfortable with the fact that there's a little-tiny computer (the ALOM) powered by VXWorks inside that's acting as the management system and console (there's no screen or keyboard/mouse input), Installing OpenBSD on the base server is pretty straightforward. The serial console is an RJ-45 jack, and, yes, the ubiquitous blue-colored serial console cables you find for certain kinds of popular routers will work fine. OpenBSD installs quite easily, with the same installer you find on amd64 and i386. I chose to install to /dev/sd0, the first SAS drive only, leaving the others unused. It's possible to set them up in a hardware RAID configuration using tools available only under Solaris, or use softraid(4) on OpenBSD, but I didn't do this. I set up the primary LDOM to use the first ethernet port, em0. I decided I wanted to bridge the logical domains to the second ethernet port. You could also use a bridge and vether interface, with pf and dhcpd to create a NAT environment, similar to how I networked the vmm(4) systems. Create an LDOM configuration file. You can put this anywhere that's convenient. All of this stuff was in a "vm" subdirectory of my home. I called it ldom.conf: domain primary { vcpu 8 memory 8G } domain puffy { vcpu 8 memory 4G vdisk "/home/axon/vm/ldom1" vnet } Make as many disk images as you want, and make as many additional domain clauses as you wish. Be mindful of system resources. I couldn't actually allocate a full 32GB of RAM across all the LDOMs I eventually provisioned seven LDOMs (in addition to the primary) on the T2000, each with 3GB of RAM and 4 vcpu cores. If you get creative with use of network interfaces, virtual ethernet, bridges and pf rules, you can run a pretty complex environment on a single chassis, with services that are only exposed to other VMs, a DMZ segment, and the internal LAN. A nice tutorial, and an interesting look at an alternative platform that was ahead of its time *** documentation is thoroughly hard (http://www.tedunangst.com/flak/post/documentation-is-thoroughly-hard) Ted Unangst has a new post this week about documentation: Documentation is good, so therefore more documentation must be better, right? A few examples where things may have gotten out of control A fine example is the old OpenBSD install instructions. Once you've installed OpenBSD once or twice, the process is quite simple, but you'd never know this based on reading the instructions. Compare the files for 4.8 INSTALL and 5.8 INSTALL. Both begin with a brief intro to the project. Then 4.8 has an enormous list of mirrors, which seems fairly redundant if you've already found the install file. Followed by an enormous list of every supported variant of every supported device. Including a table of IO port configurations for ISA devices. Finally, after 1600 lines of introduction we get to the actual installation instructions. (Compared to line 231 for 5.8.) This includes a full page of text about how to install from tape, which nobody ever does. It took some time to recognize that all this documentation was actually an impediment to new users. Attempting to answer every possible question floods the reader with information for questions they were never planning to ask. Part of the problem is how the information is organized. Theoretically it makes sense to list supported hardware before instructions. After all, you can't install anything if it's not supported, right? I'm sure that was considered when the device list was originally inserted above the install instructions. But as a practical matter, consulting a device list is neither the easiest nor fastest way to determine what actually works. In the FreeBSD docs tree, we have been doing a facelift project, trying to add ‘quick start' sections to each chapter to let you get to the more important information first. It is also helpful to move data in the forms of lists and tables to appendices or similar, where they can easily be references, but are not blocking your way to the information you are actually hunting for An example of nerdview signage (http://languagelog.ldc.upenn.edu/nll/?p=29866). “They have in effect provided a sign that will tell you exactly what the question is provided you can already supply the answer.” That is, the logical minds of technical people often decide to order information in an order that makes sense to them, rather than in the order that will be most useful to the reader In the end, I think “copy diskimage to USB and follow prompts” is all the instructions one should need, but it's hard to overcome the unease of actually making the jump. What if somebody is confused or uncertain? Why is this paragraph more redundant than that paragraph? (And if we delete both, are we cutting too much?) Sometimes we don't need to delete the information. Just hide it. The instructions to upgrade to 4.8 and upgrade to 5.8 are very similar, with a few differences because every release is a little bit different. The pages look very different, however, because the not at all recommended kernel free procedure, which takes up half the page, has been hidden from view behind some javascript and only expanded on demand. A casual browser will find the page and figure the upgrade process will be easy, as opposed to some long ordeal. This is important as well, it was my original motivation for working on the FreeBSD Handbook's ZFS chapter. The very first section of the chapter was the custom kernel configuration required to run ZFS on i386. That scared many users away. I moved that to the very end, and started with why you might want to use ZFS. Much more approachable. Sometimes it's just a tiny detail that's overspecified. The apmd manual used to explain exactly which CPU idle time thresholds were used to adjust frequency. Those parameters, and the algorithm itself, were adjusted occasionally in response to user feedback, but sometimes the man page lagged behind. The numbers are of no use to a user. They're not adjustable without recompiling. Knowing that the frequency would be reduced at 85% idle vs 90% idle doesn't really offer much guidance as to whether to enable auto scaling or not. Deleting this detail ensured the man page was always correct and spares the user the cognitive load of trying to solve an unnecessary math problem. For fun: For another humorous example, it was recently observed that the deja-dup package provides man page translations for Australia, Canada, and Great Britain. I checked, the pages are in fact not quite identical. Some contain typo fixes that didn't propagate to other translations. Project idea: attempt to identify which country has the most users, or most fastidious users, by bug fixes to localized man pages. lldb on BeagleBone Black (https://lists.freebsd.org/pipermail/freebsd-arm/2017-May/016260.html) I reliably managed to build (lldb + clang/lld) from the svn trunk of LLVM 5.0.0 on my Beaglebone Black running the latest snapshot (May 20th) of FreeBSD 12.0-CURRENT, and the lldb is working very well, and this includes single stepping and ncurses-GUI mode, while single stepping with the latest lldb 4.0.1 from the ports does not work. In order to reliably build LLVM 5.0.0 (svn), I set up a 1 GB swap partition for the BBB on a NFSv4 share on a FreeBSD fileserver in my network - I put a howto of the procedure on my BLog: https://obsigna.net/?p=659 The prerequesites on the Beaglebone are: ``` pkg install tmux pkg install cmake pkg install python pkg install libxml2 pkg install swig30 pkg install ninja pkg install subversion ``` On the FreeBSD fileserver: ``` /pathtothe/bbb_share svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm cd llvm/tools svn co http://llvm.org/svn/llvm-project/cfe/trunk clang svn co http://llvm.org/svn/llvm-project/lld/trunk lld svn co http://llvm.org/svn/llvm-project/lldb/trunk lldb ``` + On the Beaglebone Black: # mount_nfs -o noatime,readahead=4,intr,soft,nfsv4 server:/path_to_the/bbb_share /mnt # cd /mnt # mkdir build # cmake -DLLVM_TARGETS_TO_BUILD="ARM" -DCMAKE_BUILD_TYPE="MinSizeRel" -DLLVM_PARALLEL_COMPILE_JOBS="1" -DLLVM_PARALLEL_LINK_JOBS="1" -G Ninja .. I execute the actual build command from within a tmux session, so I may disconnect during the quite long (40 h) build: ``` tmux new "ninja lldb install" ``` When debugging in GUI mode using the newly build lldb 5.0.0-svn, I see only a minor issue, namely UTF8 strings are not displayed correctly. This happens in the ncurses-GUI only, and this is an ARM issue, since it does not occur on x86 machines. Perhaps this might be related to the signed/unsigned char mismatch between ARM and x86. Beastie Bits Triangle BSD Meetup on June 27th (https://www.meetup.com/Triangle-BSD-Users-Group/events/240247251/) Support for Controller Area Networks (CAN) in NetBSD (http://www.feyrer.de/NetBSD/bx/blosxom.cgi/nb_20170521_0113.html) Notes from Monday's meeting (http://mailman.uk.freebsd.org/pipermail/ukfreebsd/2017-May/014104.html) RunBSD - A site about the BSD family of operating systems (http://runbsd.info/) BSDCam(bridge) 2017 Travel Grant Application Now Open (https://www.freebsdfoundation.org/blog/bsdcam-2017-travel-grant-application-now-open/) New BSDMag has been released (https://bsdmag.org/download/nearly-online-zpool-switching-two-freebsd-machines/) *** Feedback/Questions Philipp - A show about byhve (http://dpaste.com/390F9JN#wrap) Jake - byhve Support on AMD (http://dpaste.com/0DYG5BD#wrap) CY - Pledge and Capsicum (http://dpaste.com/1YVBT12#wrap) CY - OpenSSL relicense Issue (http://dpaste.com/3RSYV23#wrap) Andy - Laptops (http://dpaste.com/0MM09EX#wrap) ***

This week on BSD Now we cover the latest FreeBSD Status Report, a plan for Open Source software development, centrally managing bhyve with Ansible, libvirt, and pkg-ssh, and a whole lot more. This episode was brought to you by Headlines FreeBSD Project Status Report (January to March 2017) (https://www.freebsd.org/news/status/report-2017-01-2017-03.html) While a few of these projects indicate they are a "plan B" or an "attempt III", many are still hewing to their original plans, and all have produced impressive results. Please enjoy this vibrant collection of reports, covering the first quarter of 2017. The quarterly report opens with notes from Core, The FreeBSD Foundation, the Ports team, and Release Engineering On the project front, the Ceph on FreeBSD project had made considerable advances, and is now usable as the net/ceph-devel port via the ceph-fuse module. Eventually they hope to have a kernel RADOS block device driver, so fuse is not required CloudABI update, including news that the Bitcoin reference implementation is working on a port to CloudABI eMMC Flash and SD card updates, allowing higher speeds (max speed changes from ~40 to ~80 MB/sec). As well, the MMC Stack can now also be backed by the CAM framework. Improvements to the Linuxulator More detail on the pNFS Server plan B that we discussed in a previous week Snow B.V. is sponsoring a dutch translation of the FreeBSD Handbook using the new .po system *** A plan for open source software maintainers (http://www.daemonology.net/blog/2017-05-11-plan-for-foss-maintainers.html) Colin Percival describes in his blog “a plan for open source software maintainers”: I've been writing open source software for about 15 years now; while I'm still wet behind the ears compared to FreeBSD greybeards like Kirk McKusick and Poul-Henning Kamp, I've been around for long enough to start noticing some patterns. In particular: Free software is expensive. Software is expensive to begin with; but good quality open source software tends to be written by people who are recognized as experts in their fields (partly thanks to that very software) and can demand commensurate salaries. While that expensive developer time is donated (either by the developers themselves or by their employers), this influences what their time is used for: Individual developers like doing things which are fun or high-status, while companies usually pay developers to work specifically on the features those companies need. Maintaining existing code is important, but it is neither fun nor high-status; and it tends to get underweighted by companies as well, since maintenance is inherently unlikely to be the most urgent issue at any given time. Open source software is largely a "throw code over the fence and walk away" exercise. Over the past 15 years I've written freebsd-update, bsdiff, portsnap, scrypt, spiped, and kivaloo, and done a lot of work on the FreeBSD/EC2 platform. Of these, I know bsdiff and scrypt are very widely used and I suspect that kivaloo is not; but beyond that I have very little knowledge of how widely or where my work is being used. Anecdotally it seems that other developers are in similar positions: At conferences I've heard variations on "you're using my code? Wow, that's awesome; I had no idea" many times. I have even less knowledge of what people are doing with my work or what problems or limitations they're running into. Occasionally I get bug reports or feature requests; but I know I only hear from a very small proportion of the users of my work. I have a long list of feature ideas which are sitting in limbo simply because I don't know if anyone would ever use them — I suspect the answer is yes, but I'm not going to spend time implementing these until I have some confirmation of that. A lot of mid-size companies would like to be able to pay for support for the software they're using, but can't find anyone to provide it. For larger companies, it's often easier — they can simply hire the author of the software (and many developers who do ongoing maintenance work on open source software were in fact hired for this sort of "in-house expertise" role) — but there's very little available for a company which needs a few minutes per month of expertise. In many cases, the best support they can find is sending an email to the developer of the software they're using and not paying anything at all — we've all received "can you help me figure out how to use this" emails, and most of us are happy to help when we have time — but relying on developer generosity is not a good long-term solution. Every few months, I receive email from people asking if there's any way for them to support my open source software contributions. (Usually I encourage them to donate to the FreeBSD Foundation.) Conversely, there are developers whose work I would like to support (e.g., people working on FreeBSD wifi and video drivers), but there isn't any straightforward way to do this. Patreon has demonstrated that there are a lot of people willing to pay to support what they see as worthwhile work, even if they don't get anything directly in exchange for their patronage. It seems to me that this is a case where problems are in fact solutions to other problems. To wit: Users of open source software want to be able to get help with their use cases; developers of open source software want to know how people are using their code. Users of open source software want to support the the work they use; developers of open source software want to know which projects users care about. Users of open source software want specific improvements; developers of open source software may be interested in making those specific changes, but don't want to spend the time until they know someone would use them. Users of open source software have money; developers of open source software get day jobs writing other code because nobody is paying them to maintain their open source software. I'd like to see this situation get fixed. As I envision it, a solution would look something like a cross between Patreon and Bugzilla: Users would be able sign up to "support" projects of their choosing, with a number of dollars per month (possibly arbitrary amounts, possibly specified tiers; maybe including $0/month), and would be able to open issues. These could be private (e.g., for "technical support" requests) or public (e.g., for bugs and feature requests); users would be able to indicate their interest in public issues created by other users. Developers would get to see the open issues, along with a nominal "value" computed based on allocating the incoming dollars of "support contracts" across the issues each user has expressed an interest in, allowing them to focus on issues with higher impact. He poses three questions to users about whether or not people (users and software developers alike) would be interested in this and whether payment (giving and receiving, respectively) is interesting Check out the comments (and those on https://news.ycombinator.com/item?id=14313804 (reddit.com)) as well for some suggestions and discussion on the topic *** OpenBSD vmm hypervisor: Part 2 (http://www.h-i-r.net/2017/04/openbsd-vmm-hypervisor-part-2.html) We asked for people to write up their experience using OpenBSD's VMM. This blog post is just that This is going to be a (likely long-running, infrequently-appended) series of posts as I poke around in vmm. A few months ago, I demonstrated some basic use of the vmm hypervisor as it existed in OpenBSD 6.0-CURRENT around late October, 2016. We'll call that video Part 1. Quite a bit of development was done on vmm before 6.1-RELEASE, and it's worth noting that some new features made their way in. Work continues, of course, and I can only imagine the hypervisor technology will mature plenty for the next release. As it stands, this is the first release of OpenBSD with a native hypervisor shipped in the base install, and that's exciting news in and of itself To get our virtual machines onto the network, we have to spend some time setting up a virtual ethernet interface. We'll run a DHCP server on that, and it'll be the default route for our virtual machines. We'll keep all the VMs on a private network segment, and use NAT to allow them to get to the network. There is a way to directly bridge VMs to the network in some situations, but I won't be covering that today. Create an empty disk image for your new VM. I'd recommend 1.5GB to play with at first. You can do this without doas or root if you want your user account to be able to start the VM later. I made a "vmm" directory inside my home directory to store VM disk images in. You might have a different partition you wish to store these large files in. Boot up a brand new vm instance. You'll have to do this as root or with doas. You can download a -CURRENT install kernel/ramdisk (bsd.rd) from an OpenBSD mirror, or you can simply use the one that's on your existing system (/bsd.rd) like I'll do here. The command will start a VM named "test.vm", display the console at startup, use /bsd.rd (from our host environment) as the boot image, allocate 256MB of memory, attach the first network interface to the switch called "local" we defined earlier in /etc/vm.conf, and use the test image we just created as the first disk drive. Now that the VM disk image file has a full installation of OpenBSD on it, build a VM configuration around it by adding the below block of configuration (with modifications as needed for owner, path and lladdr) to /etc/vm.conf I've noticed that VMs with much less than 256MB of RAM allocated tend to be a little unstable for me. You'll also note that in the "interface" clause, I hard-coded the lladdr that was generated for it earlier. By specifying "disable" in vm.conf, the VM will show up in a stopped state that the owner of the VM (that's you!) can manually start without root access. Let us know how VMM works for you *** News Roundup openbsd changes of note 621 (http://www.tedunangst.com/flak/post/openbsd-changes-of-note-621) More stuff, more fun. Fix script to not perform tty operations on things that aren't ttys. Detected by pledge. Merge libdrm 2.4.79. After a forced unmount, also unmount any filesystems below that mount point. Flip previously warm pages in the buffer cache to memory above the DMA region if uvm tells us it is available. Pages are not automatically promoted to upper memory. Instead it's used as additional memory only for what the cache considers long term buffers. I/O still requires DMA memory, so writing to a buffer will pull it back down. Makefile support for systems with both gcc and clang. Make i386 and amd64 so. Take a more radical approach to disabling colours in clang. When the data buffered for write in tmux exceeds a limit, discard it and redraw. Helps when a fast process is running inside tmux running inside a slow terminal. Add a port of witness(4) lock validation tool from FreeBSD. Use it with mplock, rwlock, and mutex in the kernel. Properly save and restore FPU context in vmm. Remove KGDB. It neither compiles nor works. Add a constant time AES implementation, from BearSSL. Remove SSHv1 from ssh. and more... *** Digging into BSD's choice of Unix group for new directories and files (https://utcc.utoronto.ca/~cks/space/blog/unix/BSDDirectoryGroupChoice) I have to eat some humble pie here. In comments on my entry on an interesting chmod failure, Greg A. Woods pointed out that FreeBSD's behavior of creating everything inside a directory with the group of the directory is actually traditional BSD behavior (it dates all the way back to the 1980s), not some odd new invention by FreeBSD. As traditional behavior it makes sense that it's explicitly allowed by the standards, but I've also come to think that it makes sense in context and in general. To see this, we need some background about the problem facing BSD. In the beginning, two things were true in Unix: there was no mkdir() system call, and processes could only be in one group at a time. With processes being in only one group, the choice of the group for a newly created filesystem object was easy; it was your current group. This was felt to be sufficiently obvious behavior that the V7 creat(2) manpage doesn't even mention it. Now things get interesting. 4.1c BSD seems to be where mkdir(2) is introduced and where creat() stops being a system call and becomes an option to open(2). It's also where processes can be in multiple groups for the first time. The 4.1c BSD open(2) manpage is silent about the group of newly created files, while the mkdir(2) manpage specifically claims that new directories will have your effective group (ie, the V7 behavior). This is actually wrong. In both mkdir() in sysdirectory.c and maknode() in ufssyscalls.c, the group of the newly created object is set to the group of the parent directory. Then finally in the 4.2 BSD mkdir(2) manpage the group of the new directory is correctly documented (the 4.2 BSD open(2) manpage continues to say nothing about this). So BSD's traditional behavior was introduced at the same time as processes being in multiple groups, and we can guess that it was introduced as part of that change. When your process can only be in a single group, as in V7, it makes perfect sense to create new filesystem objects with that as their group. It's basically the same case as making new filesystem objects be owned by you; just as they get your UID, they also get your GID. When your process can be in multiple groups, things get less clear. A filesystem object can only be in one group, so which of your several groups should a new filesystem object be owned by, and how can you most conveniently change that choice? One option is to have some notion of a 'primary group' and then provide ways to shuffle around which of your groups is the primary group. Another option is the BSD choice of inheriting the group from context. By far the most common case is that you want your new files and directories to be created in the 'context', ie the group, of the surrounding directory. If you fully embrace the idea of Unix processes being in multiple groups, not just having one primary group and then some number of secondary groups, then the BSD choice makes a lot of sense. And for all of its faults, BSD tended to relatively fully embrace its changes While it leads to some odd issues, such as the one I ran into, pretty much any choice here is going to have some oddities. Centrally managed Bhyve infrastructure with Ansible, libvirt and pkg-ssh (http://www.shellguardians.com/2017/05/centrally-managed-bhyve-infrastructure.html) At work we've been using Bhyve for a while to run non-critical systems. It is a really nice and stable hypervisor even though we are using an earlier version available on FreeBSD 10.3. This means we lack Windows and VNC support among other things, but it is not a big deal. After some iterations in our internal tools, we realised that the installation process was too slow and we always repeated the same steps. Of course, any good sysadmin will scream "AUTOMATION!" and so did we. Therefore, we started looking for different ways to improve our deployments. We had a look at existing frameworks that manage Bhyve, but none of them had a feature that we find really important: having a centralized repository of VM images. For instance, SmartOS applies this method successfully by having a backend server that stores a catalog of VMs and Zones, meaning that new instances can be deployed in a minute at most. This is a game changer if you are really busy in your day-to-day operations. The following building blocks are used: The ZFS snapshot of an existing VM. This will be our VM template. A modified version of oneoff-pkg-create to package the ZFS snapshots. pkg-ssh and pkg-repo to host a local FreeBSD repo in a FreeBSD jail. libvirt to manage our Bhyve VMs. The ansible modules virt, virtnet and virtpool. Once automated, the installation process needs 2 minutes at most, compared with the 30 minutes needed to manually install VM plus allowing us to deploy many guests in parallel. NetBSD maintainer in the QEMU project (https://blog.netbsd.org/tnf/entry/netbsd_maintainer_in_the_qemu) QEMU - the FAST! processor emulator - is a generic, Open Source, machine emulator and virtualizer. It defines state of the art in modern virtualization. This software has been developed for multiplatform environments with support for NetBSD since virtually forever. It's the primary tool used by the NetBSD developers and release engineering team. It is run with continuous integration tests for daily commits and execute regression tests through the Automatic Test Framework (ATF). The QEMU developers warned the Open Source community - with version 2.9 of the emulator - that they will eventually drop support for suboptimally supported hosts if nobody will step in and take the maintainership to refresh the support. This warning was directed to major BSDs, Solaris, AIX and Haiku. Thankfully the NetBSD position has been filled - making NetBSD to restore official maintenance. Beastie Bits OpenBSD Community Goes Gold (http://undeadly.org/cgi?action=article&sid=20170510012526&mode=flat&count=0) CharmBUG's Tor Hack-a-thon has been pushed back to July due to scheduling difficulties (https://www.meetup.com/CharmBUG/events/238218840/) Direct Rendering Manager (DRM) Driver for i915, from the Linux kernel to Haiku with the help of DragonflyBSD's Linux Compatibility layer (https://www.haiku-os.org/blog/vivek/2017-05-05_[gsoc_2017]_3d_hardware_acceleration_in_haiku/) TomTom lists OpenBSD in license (https://twitter.com/bsdlme/status/863488045449977864) London Net BSD Meetup on May 22nd (https://mail-index.netbsd.org/regional-london/2017/05/02/msg000571.html) KnoxBUG meeting May 30th, 2017 - Introduction to FreeNAS (http://knoxbug.org/2017-05-30) *** Feedback/Questions Felix - Home Firewall (http://dpaste.com/35EWVGZ#wrap) David - Docker Recipes for Jails (http://dpaste.com/0H51NX2#wrap) Don - GoLang & Rust (http://dpaste.com/2VZ7S8K#wrap) George - OGG feed (http://dpaste.com/2A1FZF3#wrap) Roller - BSDCan Tips (http://dpaste.com/3D2B6J3#wrap) ***

This week on BSD Now, Adrian Chadd on bringing up 802.11ac in FreeBSD, a PFsense and OpenVPN tutorial, and we talk about an interesting ZFS storage pool checkpoint project. This episode was brought to you by Headlines Bringing up 802.11ac on FreeBSD (http://adrianchadd.blogspot.com/2017/04/bringing-up-80211ac-on-freebsd.html) Adrian Chadd has a new blog post about his work to bring 802.11ac support to FreeBSD 802.11ac allows for speeds up to 500mbps and total bandwidth into multiple gigabits The FreeBSD net80211 stack has reasonably good 802.11n support, but no 802.11ac support. I decided a while ago to start adding basic 802.11ac support. It was a good exercise in figuring out what the minimum set of required features are and another excuse to go find some of the broken corner cases in net80211 that needed addressing. 802.11ac introduces a few new concepts that the stack needs to understand. I decided to use the QCA 802.11ac parts because (a) I know the firmware and general chip stuff from the first generation 11ac parts well, and (b) I know that it does a bunch of stuff (like rate control, packet scheduling, etc) so I don't have to do it. If I chose, say, the Intel 11ac parts then I'd have to implement a lot more of the fiddly stuff to get good behaviour. Step one - adding VHT channels. I decided in the shorter term to cheat and just add VHT channels to the already very large ieee80211channel map. The linux way of there being a channel context rather than hundreds of static channels to choose from is better in the long run, but I wanted to get things up and running. So, that's what I did first - I added VHT flags for 20, 40, 80, 80+80 and 160MHz operating modes and I did the bare work required to populate the channel lists with VHT channels as well. Then I needed to glue it into an 11ac driver. My ath10k port was far enough along to attempt this, so I added enough glue to say "I support VHT" to the iccaps field and propagated it to the driver for monitor mode configuration. And yes, after a bit of dancing, I managed to get a VHT channel to show up in ath10k in monitor mode and could capture 80MHz wide packets. Success! By far the most fiddly was getting channel promotion to work. net80211 supports the concept of dumb NICs (like atheros 11abgn parts) very well, where you can have multiple virtual interfaces but the "driver" view of the right configuration is what's programmed into the hardware. For firmware NICs which do this themselves (like basically everything sold today) this isn't exactly all that helpful. So, for now, it's limited to a single VAP, and the VAP configuration is partially derived from the global state and partially derived from the negotiated state. It's annoying, but it is adding to the list of things I will have to fix later. the QCA chips/firmware do 802.11 crypto offload. They actually pretend that there's no key - you don't include the IV, you don't include padding, or anything. You send commands to set the crypto keys and then you send unencrypted 802.11 frames (or 802.3 frames if you want to do ethernet only.) This means that I had to teach net80211 a few things: + frames decrypted by the hardware needed to have a "I'm decrypted" bit set, because the 802.11 header field saying "I'm decrypted!" is cleared + frames encrypted don't have the "i'm encrypted" bit set + frames encrypted/decrypted have no padding, so I needed to teach the input path and crypto paths to not validate those if the hardware said "we offload it all." Now comes the hard bit of fixing the shortcomings before I can commit the driver. There are .. lots. The first one is the global state. The ath10k firmware allows what they call 'vdevs' (virtual devices) - for example, multiple SSID/BSSID support is implemented with multiple vdevs. STA+WDS is implemented with vdevs. STA+P2P is implemented with vdevs. So, technically speaking I should go and find all of the global state that should really be per-vdev and make it per-vdev. This is tricky though, because a lot of the state isn't kept per-VAP even though it should be. Anyway, so far so good. I need to do some of the above and land it in FreeBSD-HEAD so I can finish off the ath10k port and commit what I have to FreeBSD. There's a lot of stuff coming - including all of the wave-2 stuff (like multiuser MIMO / MU-MIMO) which I just plainly haven't talked about yet. Viva la FreeBSD wireless! pfSense and OpenVPN Routing (http://www.terrafoundry.net/blog/2017/04/12/pfsense-openvpn/) This article tries to be a simple guide on how to enable your home (or small office) https://www.pfsense.org/ (pfSense) setup to route some traffic via the vanilla Internet, and some via a VPN site that you've setup in a remote location. Reasons to Setup a VPN: Control Security Privacy Fun VPNs do not instantly guarantee privacy, they're a layer, as with any other measure you might invoke. In this example I used a server that's directly under my name. Sure, it was a country with strict privacy laws, but that doesn't mean that the outgoing IP address wouldn't be logged somewhere down the line. There's also no reason you have to use your own OpenVPN install, there are many, many personal providers out there, who can offer the same functionality, and a degree of anonymity. (If you and a hundred other people are all coming from one IP, it becomes extremely difficult to differentiate, some VPN providers even claim a ‘logless' setup.) VPNs can be slow. The reason I have a split-setup in this article, is because there are devices that I want to connect to the internet quickly, and that I'm never doing sensitive things on, like banking. I don't mind if my Reddit-browsing and IRC messages are a bit slower, but my Nintendo Switch and PS4 should have a nippy connection. Services like Netflix can and do block VPN traffic in some cases. This is more of an issue for wider VPN providers (I suspect, but have no proof, that they just blanket block known VPN IP addresses.) If your VPN is in another country, search results and tracking can be skewed. This is arguable a good thing, who wants to be tracked? But it can also lead to frustration if your DuckDuckGo results are tailored to the middle of Paris, rather than your flat in Birmingham. The tutorial walks through the basic setup: Labeling the interfaces, configuring DHCP, creating a VPN: Now that we have our OpenVPN connection set up, we'll double check that we've got our interfaces assigned With any luck (after we've assigned our OPENVPN connection correctly, you should now see your new Virtual Interface on the pfSense Dashboard We're charging full steam towards the sections that start to lose people. Don't be disheartened if you've had a few issues up to now, there is no “right” way to set up a VPN installation, and it may be that you have to tweak a few things and dive into a few man-pages before you're set up. NAT is tricky, and frankly it only exists because we stretched out IPv4 for much longer than we should have. That being said it's a necessary evil in this day and age, so let's set up our connection to work with it. We need NAT here because we're going to masque our machines on the LAN interface to show as coming from the OpenVPN client IP address, to the OpenVPN server. Head over to Firewall -> NAT -> Outbound. The first thing we need to do in this section, is to change the Outbound NAT Mode to something we can work with, in this case “Hybrid.” Configure the LAN interface to be NAT'd to the OpenVPN address, and the INSECURE interface to use your regular ISP connection Configure the firewall to allow traffic from the LAN network to reach the INSECURE network Then add a second rule allowing traffic from the LAN network to any address, and set the gateway the the OPENVPN connection And there you have it, traffic from the LAN is routed via the VPN, and traffic from the INSECURE network uses the naked internet connection *** Switching to OpenBSD (https://mndrix.blogspot.co.uk/2017/05/switching-to-openbsd.html) After 12 years, I switched from macOS to OpenBSD. It's clean, focused, stable, consistent and lets me get my work done without any hassle. When I first became interested in computers, I thought operating systems were fascinating. For years I would reinstall an operating system every other weekend just to try a different configuration: MS-DOS 3.3, Windows 3.0, Linux 1.0 (countless hours recompiling kernels). In high school, I settled down and ran OS/2 for 5 years until I graduated college. I switched to Linux after college and used it exclusively for 5 years. I got tired of configuring Linux, so I switched to OS X for the next 12 years, where things just worked. But Snow Leopard was 7 years ago. These days, OS X is like running a denial of service attack against myself. macOS has a dozen apps I don't use but can't remove. Updating them requires a restart. Frequent updates to the browser require a restart. A minor XCode update requires me to download a 4.3 GB file. My monitors frequently turn off and require a restart to fix. A system's availability is a function (http://techthoughts.typepad.com/managing_computers/2007/11/availability-mt.html) of mean time between failure and mean time to repair. For macOS, both numbers are heading in the wrong direction for me. I don't hold any hard feelings about it, but it's time for me to get off this OS and back to productive work. I found OpenBSD very refreshing, so I created a bootable thumb drive and within an hour had it up and running on a two-year old laptop. I've been using it for my daily work for the past two weeks and it's been great. Simple, boring and productive. Just the way I like it. The documentation is fantastic. I've been using Unix for years and have learned quite a bit just by reading their man pages. OS releases come like clockwork every 6 months and are supported for 12. Security and other updates seem relatively rare between releases (roughly one small patch per week during 6.0). With syspatch in 6.1, installing them should be really easy too. ZFS Storage Pool Checkpoint Project (https://sdimitro.github.io/post/zpool-checkpoint) During the OpenZFS summit last year (2016), Dan Kimmel and I quickly hacked together the zpool checkpoint command in ZFS, which allows reverting an entire pool to a previous state. Since it was just for a hackathon, our design was bare bones and our implementation far from complete. Around a month later, we had a new and almost complete design within Delphix and I was able to start the implementation on my own. I completed the implementation last month, and we're now running regression tests, so I decided to write this blog post explaining what a storage pool checkpoint is, why we need it within Delphix, and how to use it. The Delphix product is basically a VM running DelphixOS (a derivative of illumos) with our application stack on top of it. During an upgrade, the VM reboots into the new OS bits and then runs some scripts that update the environment (directories, snapshots, open connections, etc.) for the new version of our app stack. Software being software, failures can happen at different points during the upgrade process. When an upgrade script that makes changes to ZFS fails, we have a corresponding rollback script that attempts to bring ZFS and our app stack back to their previous state. This is very tricky as we need to undo every single modification applied to ZFS (including dataset creation and renaming, or enabling new zpool features). The idea of Storage Pool Checkpoint (aka zpool checkpoint) deals with exactly that. It can be thought of as a “pool-wide snapshot” (or a variation of extreme rewind that doesn't corrupt your data). It remembers the entire state of the pool at the point that it was taken and the user can revert back to it later or discard it. Its generic use case is an administrator that is about to perform a set of destructive actions to ZFS as part of a critical procedure. She takes a checkpoint of the pool before performing the actions, then rewinds back to it if one of them fails or puts the pool into an unexpected state. Otherwise, she discards it. With the assumption that no one else is making modifications to ZFS, she basically wraps all these actions into a “high-level transaction”. I definitely see value in this for the appliance use case Some usage examples follow, along with some caveats. One of the restrictions is that you cannot attach, detach, or remove a device while a checkpoint exists. However, the zpool add operation is still possible, however if you roll back to the checkpoint, the device will no longer be part of the pool. Rather than a shortcoming, this seems like a nice feature, a way to help users avoid the most common foot shooting (which I witnessed in person at Linux Fest), adding a new log or cache device, but missing a keyword and adding it is a storage vdev rather than a aux vdev. This operation could simply be undone if a checkpoint where taken before the device was added. *** News Roundup Review of TrueOS (https://distrowatch.com/weekly.php?issue=20170501#trueos) TrueOS, which was formerly named PC-BSD, is a FreeBSD-based operating system. TrueOS is a rolling release platform which is based on FreeBSD's "CURRENT" branch, providing TrueOS with the latest drivers and features from FreeBSD. Apart from the name change, TrueOS has deviated from the old PC-BSD project in a number of ways. The system installer is now more streamlined (and I will touch on that later) and TrueOS is a rolling release platform while PC-BSD defaulted to point releases. Another change is PC-BSD used to allow the user to customize which software was installed at boot time, including the desktop environment. The TrueOS project now selects a minimal amount of software for the user and defaults to using the Lumina desktop environment. From the conclusions: What I took away from my time with TrueOS is that the project is different in a lot of ways from PC-BSD. Much more than just the name has changed. The system is now more focused on cutting edge software and features in FreeBSD's development branch. The install process has been streamlined and the user begins with a set of default software rather than selecting desired packages during the initial setup. The configuration tools, particularly the Control Panel and AppCafe, have changed a lot in the past year. The designs have a more flat, minimal look. It used to be that PC-BSD did not have a default desktop exactly, but there tended to be a focus on KDE. With TrueOS the project's in-house desktop, Lumina, serves as the default environment and I think it holds up fairly well. In all, I think TrueOS offers a convenient way to experiment with new FreeBSD technologies and ZFS. I also think people who want to run FreeBSD on a desktop computer may want to look at TrueOS as it sets up a graphical environment automatically. However, people who want a stable desktop platform with lots of applications available out of the box may not find what they want with this project. A simple guide to install Ubuntu on FreeBSD with byhve (https://www.davd.eu/install-ubuntu-on-freebsd-with-bhyve/) David Prandzioch writes in his blog: For some reasons I needed a Linux installation on my NAS. bhyve is a lightweight virtualization solution for FreeBSD that makes that easy and efficient. However, the CLI of bhyve is somewhat bulky and bare making it hard to use, especially for the first time. This is what vm-bhyve solves - it provides a simple CLI for working with virtual machines. More details follow about what steps are needed to setup vm_bhyve on FreeBSD Also check out his other tutorials on his blog: https://www.davd.eu/freebsd/ (https://www.davd.eu/freebsd/) *** Graphical Overview of the Architecture of FreeBSD (https://dspinellis.github.io/unix-architecture/arch.pdf) This diagram tries to show the different components that make up the FreeBSD Operating Systems It breaks down the various utilities, libraries, and components into some categories and sub-categories: User Commands: Development (cc, ld, nm, as, etc) File Management (ls, cp, cmp, mkdir) Multiuser Commands (login, chown, su, who) Number Processing (bc, dc, units, expr) Text Processing (cut, grep, sort, uniq, wc) User Messaging (mail, mesg, write, talk) Little Languages (sed, awk, m4) Network Clients (ftp, scp, fetch) Document Preparation (*roff, eqn, tbl, refer) Administrator and System Commands Filesystem Management (fsck, newfs, gpart, mount, umount) Networking (ifconfig, route, arp) User Management (adduser, pw, vipw, sa, quota*) Statistics (iostat, vmstat, pstat, gstat, top) Network Servers (sshd, ftpd, ntpd, routed, rpc.*) Scheduling (cron, periodic, rc.*, atrun) Libraries (C Standard, Operating System, Peripheral Access, System File Access, Data Handling, Security, Internationalization, Threads) System Call Interface (File I/O, Mountable Filesystems, File ACLs, File Permissions, Processes, Process Tracing, IPC, Memory Mapping, Shared Memory, Kernel Events, Memory Locking, Capsicum, Auditing, Jails) Bootstrapping (Loaders, Configuration, Kernel Modules) Kernel Utility Functions Privilege Management (acl, mac, priv) Multitasking (kproc, kthread, taskqueue, swi, ithread) Memory Management (vmem, uma, pbuf, sbuf, mbuf, mbchain, malloc/free) Generic (nvlist, osd, socket, mbuf_tags, bitset) Virtualization (cpuset, crypto, device, devclass, driver) Synchronization (lock, sx, sema, mutex, condvar_, atomic_*, signal) Operations (sysctl, dtrace, watchdog, stack, alq, ktr, panic) I/O Subsystem Special Devices (line discipline, tty, raw character, raw disk) Filesystems (UFS, FFS, NFS, CD9660, Ext2, UDF, ZFS, devfs, procfs) Sockets Network Protocols (TCP, UDP, UCMP, IPSec, IP4, IP6) Netgraph (50+ modules) Drivers and Abstractions Character Devices CAM (ATA, SATA, SAS, SPI) Network Interface Drivers (802.11, ifae, 100+, ifxl, NDIS) GEOM Storage (stripe, mirror, raid3, raid5, concat) Encryption / Compression (eli, bde, shsec, uzip) Filesystem (label, journal, cache, mbr, bsd) Virtualization (md, nop, gate, virtstor) Process Control Subsystems Scheduler Memory Management Inter-process Communication Debugging Support *** Official OpenBSD 6.1 CD - There's only One! (http://undeadly.org/cgi?action=article&sid=20170503203426&mode=expanded) Ebay auction Link (http://www.ebay.com/itm/The-only-Official-OpenBSD-6-1-CD-set-to-be-made-For-auction-for-the-project-/252910718452) Now it turns out that in fact, exactly one CD set was made, and it can be yours if you are the successful bidder in the auction that ends on May 13, 2017 (About 3 days from when this episode was recorded). The CD set is hand made and signed by Theo de Raadt. Fun Fact: The winning bidder will have an OpenBSD CD set that even Theo doesn't have. *** Beastie Bits Hardware Wanted by OpenBSD developers (https://www.openbsd.org/want.html) Donate hardware to FreeBSD developers (https://www.freebsd.org/donations/index.html#components) Announcing NetBSD and the Google Summer of Code Projects 2017 (https://blog.netbsd.org/tnf/entry/announcing_netbsd_and_the_google) Announcing FreeBSD GSoC 2017 Projects (https://wiki.freebsd.org/SummerOfCode2017Projects) LibreSSL 2.5.4 Released (https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.5.4-relnotes.txt) CharmBUG Meeting - Tor Browser Bundle Hack-a-thon (https://www.meetup.com/CharmBUG/events/238218840/) pkgsrcCon 2017 CFT (https://mail-index.netbsd.org/netbsd-advocacy/2017/05/01/msg000735.html) Experimental Price Cuts (https://blather.michaelwlucas.com/archives/2931) Linux Fest North West 2017: Three Generations of FreeNAS: The World's most popular storage OS turns 12 (https://www.youtube.com/watch?v=x6VznQz3VEY) *** Feedback/Questions Don - Reproducible builds & gcc/clang (http://dpaste.com/2AXX75X#wrap) architect - C development on BSD (http://dpaste.com/0FJ854X#wrap) David - Linux ABI (http://dpaste.com/2CCK2WF#wrap) Tom - ZFS (http://dpaste.com/2Z25FKJ#wrap) RAIDZ Stripe Width Myth, Busted (https://www.delphix.com/blog/delphix-engineering/zfs-raidz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz) Ivan - Jails (http://dpaste.com/1Z173WA#wrap) ***

Today on BSD Now, the latest Dragonfly BSD release, RaidZ performance, another OpenSSL Vulnerability, and more; all this week on BSD Now. This episode was brought to you by Headlines DragonFly BSD 4.8 is released (https://www.dragonflybsd.org/release48/) Improved kernel performance This release further localizes cache lines and reduces/removes cache ping-ponging on globals. For bulk builds on many-cores or multi-socket systems, we have around a 5% improvement, and certain subsystems such as namecache lookups and exec()s see massive focused improvements. See the corresponding mailing list post with details. Support for eMMC booting, and mobile and high-performance PCIe SSDs This kernel release includes support for eMMC storage as the boot device. We also sport a brand new SMP-friendly, high-performance NVMe SSD driver (PCIe SSD storage). Initial device test results are available. EFI support The installer can now create an EFI or legacy installation. Numerous adjustments have been made to userland utilities and the kernel to support EFI as a mainstream boot environment. The /boot filesystem may now be placed either in its own GPT slice, or in a DragonFly disklabel inside a GPT slice. DragonFly, by default, creates a GPT slice for all of DragonFly and places a DragonFly disklabel inside it with all the standard DFly partitions, such that the disk names are roughly the same as they would be in a legacy system. Improved graphics support The i915 driver has been updated to match the version found with the Linux 4.6 kernel. Broadwell and Skylake processor users will see improvements. Other user-affecting changes Kernel is now built using -O2. VKernels now use COW, so multiple vkernels can share one disk image. powerd() is now sensitive to time and temperature changes. Non-boot-filesystem kernel modules can be loaded in rc.conf instead of loader.conf. *** #8005 poor performance of 1MB writes on certain RAID-Z configurations (https://github.com/openzfs/openzfs/pull/321) Matt Ahrens posts a new patch for OpenZFS Background: RAID-Z requires that space be allocated in multiples of P+1 sectors,because this is the minimum size block that can have the required amount of parity. Thus blocks on RAIDZ1 must be allocated in a multiple of 2 sectors; on RAIDZ2 multiple of 3; and on RAIDZ3 multiple of 4. A sector is a unit of 2^ashift bytes, typically 512B or 4KB. To satisfy this constraint, the allocation size is rounded up to the proper multiple, resulting in up to 3 "pad sectors" at the end of some blocks. The contents of these pad sectors are not used, so we do not need to read or write these sectors. However, some storage hardware performs much worse (around 1/2 as fast) on mostly-contiguous writes when there are small gaps of non-overwritten data between the writes. Therefore, ZFS creates "optional" zio's when writing RAID-Z blocks that include pad sectors. If writing a pad sector will fill the gap between two (required) writes, we will issue the optional zio, thus doubling performance. The gap-filling performance improvement was introduced in July 2009. Writing the optional zio is done by the io aggregation code in vdevqueue.c. The problem is that it is also subject to the limit on the size of aggregate writes, zfsvdevaggregationlimit, which is by default 128KB. For a given block, if the amount of data plus padding written to a leaf device exceeds zfsvdevaggregation_limit, the optional zio will not be written, resulting in a ~2x performance degradation. The solution is to aggregate optional zio's regardless of the aggregation size limit. As you can see from the graphs, this can make a large difference in performance. I encourage you to read the entire commit message, it is well written and very detailed. *** Can you spot the OpenSSL vulnerability (https://guidovranken.wordpress.com/2017/01/28/can-you-spot-the-vulnerability/) This code was introduced in OpenSSL 1.1.0d, which was released a couple of days ago. This is in the server SSL code, ssl/statem/statemsrvr.c, sslbytestocipherlist()), and can easily be reached remotely. Can you spot the vulnerability? So there is a loop, and within that loop we have an ‘if' statement, that tests a number of conditions. If any of those conditions fail, OPENSSLfree(raw) is called. But raw isn't the address that was allocated; raw is increment every loop. Hence, there is a remote invalid free vulnerability. But not quite. None of those checks in the ‘if' statement can actually fail; earlier on in the function, there is a check that verifies that the packet contains at least 1 byte, so PACKETget1 cannot fail. Furthermore, earlier in the function it is verified that the packet length is a multiple of 3, hence PACKETcopybytes and PACKET_forward cannot fail. So, does the code do what the original author thought, or expected it to do? But what about the next person that modifies that code, maybe changing or removing one of the earlier checks, allowing one of those if conditions to fail, and execute the bad code? Nonetheless OpenSSL has acknowledged that the OPENSSL_free line needs a rewrite: Pull Request #2312 (https://github.com/openssl/openssl/pull/2312) PS I'm not posting this to ridicule the OpenSSL project or their programming skills. I just like reading code and finding corner cases that impact security, which is an effort that ultimately works in everybody's best interest, and I like to share what I find. Programming is a very difficult enterprise and everybody makes mistakes. Thanks to Guido Vranken for the sharp eye and the blog post *** Research Debt (http://distill.pub/2017/research-debt/) I found this article interesting as it relates to not just research, but a lot of technical areas in general Achieving a research-level understanding of most topics is like climbing a mountain. Aspiring researchers must struggle to understand vast bodies of work that came before them, to learn techniques, and to gain intuition. Upon reaching the top, the new researcher begins doing novel work, throwing new stones onto the top of the mountain and making it a little taller for whoever comes next. People expect the climb to be hard. It reflects the tremendous progress and cumulative effort that's gone into the research. The climb is seen as an intellectual pilgrimage, the labor a rite of passage. But the climb could be massively easier. It's entirely possible to build paths and staircases into these mountains. The climb isn't something to be proud of. The climb isn't progress: the climb is a mountain of debt. Programmers talk about technical debt: there are ways to write software that are faster in the short run but problematic in the long run. Poor Exposition – Often, there is no good explanation of important ideas and one has to struggle to understand them. This problem is so pervasive that we take it for granted and don't appreciate how much better things could be. Undigested Ideas – Most ideas start off rough and hard to understand. They become radically easier as we polish them, developing the right analogies, language, and ways of thinking. Bad abstractions and notation – Abstractions and notation are the user interface of research, shaping how we think and communicate. Unfortunately, we often get stuck with the first formalisms to develop even when they're bad. For example, an object with extra electrons is negative, and pi is wrong Noise – Being a researcher is like standing in the middle of a construction site. Countless papers scream for your attention and there's no easy way to filter or summarize them. We think noise is the main way experts experience research debt. There's a tradeoff between the energy put into explaining an idea, and the energy needed to understand it. On one extreme, the explainer can painstakingly craft a beautiful explanation, leading their audience to understanding without even realizing it could have been difficult. On the other extreme, the explainer can do the absolute minimum and abandon their audience to struggle. This energy is called interpretive labor Research distillation is the opposite of research debt. It can be incredibly satisfying, combining deep scientific understanding, empathy, and design to do justice to our research and lay bare beautiful insights. Distillation is also hard. It's tempting to think of explaining an idea as just putting a layer of polish on it, but good explanations often involve transforming the idea. This kind of refinement of an idea can take just as much effort and deep understanding as the initial discovery. + The distillation can often times require an entirely different set of skills than the original creation of the idea. Almost all of the BSD projects have some great ideas or subsystems that just need distillation into easy to understand and use platforms or tools. Like the theoretician, the experimentalist or the research engineer, the research distiller is an integral role for a healthy research community. Right now, almost no one is filling it. Anyway, if that bit piqued your interest, go read the full article and the suggested further reading. *** News Roundup And then the murders began. (https://blather.michaelwlucas.com/archives/2902) A whole bunch of people have pointed me at articles like this one (http://thehookmag.com/2017/03/adding-murders-began-second-sentence-book-makes-instantly-better-125462/), which claim that you can improve almost any book by making the second sentence “And then the murders began.” It's entirely possible they're correct. But let's check, with a sampling of books. As different books come in different tenses and have different voices, I've made some minor changes. “Welcome to Cisco Routers for the Desperate! And then the murders begin.” — Cisco Routers for the Desperate, 2nd ed “Over the last ten years, OpenSSH has become the standard tool for remote management of Unix-like systems and many network devices. And then the murders began.” — SSH Mastery “The Z File System, or ZFS, is a complicated beast, but it is also the most powerful tool in a sysadmin's Batman-esque utility belt. And then the murders begin.” — FreeBSD Mastery: Advanced ZFS “Blood shall rain from the sky, and great shall be the lamentation of the Linux fans. And then, the murders will begin.” — Absolute FreeBSD, 3rd Ed Netdata now supports FreeBSD (https://github.com/firehol/netdata) netdata is a system for distributed real-time performance and health monitoring. It provides unparalleled insights, in real-time, of everything happening on the system it runs (including applications such as web and database servers), using modern interactive web dashboards. From the release notes: apps.plugin ported for FreeBSD Check out their demo sites (https://github.com/firehol/netdata/wiki) *** Distrowatch Weekly reviews RaspBSD (https://distrowatch.com/weekly.php?issue=20170220#raspbsd) RaspBSD is a FreeBSD-based project which strives to create a custom build of FreeBSD for single board and hobbyist computers. RaspBSD takes a recent snapshot of FreeBSD and adds on additional components, such as the LXDE desktop and a few graphical applications. The RaspBSD project currently has live images for Raspberry Pi devices, the Banana Pi, Pine64 and BeagleBone Black & Green computers. The default RaspBSD system is quite minimal, running a mere 16 processes when I was logged in. In the background the operating system runs cron, OpenSSH, syslog and the powerd power management service. Other than the user's shell and terminals, nothing else is running. This means RaspBSD uses little memory, requiring just 16MB of active memory and 31MB of wired or kernel memory. I made note of a few practical differences between running RaspBSD on the Pi verses my usual Raspbian operating system. One minor difference is RaspBSD turns off the Pi's external power light after booting. Raspbian leaves the light on. This means it looks like the Pi is off when it is running RaspBSD, but it also saves a little electricity. Conclusions: Apart from these little differences, running RaspBSD on the Pi was a very similar experience to running Raspbian and my time with the operating system was pleasantly trouble-free. Long-term, I think applying source updates to the base system might be tedious and SD disk operations were slow. However, the Pi usually is not utilized for its speed, but rather its low cost and low-energy usage. For people who are looking for a small home server or very minimal desktop box, RaspBSD running on the Pi should be suitable. Research UNIX V8, V9 and V10 made public by Alcatel-Lucent (https://media-bell-labs-com.s3.amazonaws.com/pages/20170327_1602/statement%20regarding%20Unix%203-7-17.pdf) Alcatel-Lucent USA Inc. (“ALU-USA”), on behalf of itself and Nokia Bell Laboratories agrees, to the extent of its ability to do so, that it will not assert its copyright rights with respect to any non-commercial copying, distribution, performance, display or creation of derivative works of Research Unix®1 Editions 8, 9, and 10. Research Unix is a term used to refer to versions of the Unix operating system for DEC PDP-7, PDP-11, VAX and Interdata 7/32 and 8/32 computers, developed in the Bell Labs Computing Science Research Center. The version breakdown can be viewed on its Wikipedia page (https://en.wikipedia.org/wiki/Research_Unix) It only took 30+ years, but now they're public You can grab them from here (http://www.tuhs.org/Archive/Distributions/Research/) If you're wondering what happened with Research Unix, After Version 10, Unix development at Bell Labs was stopped in favor of a successor system, Plan 9 (http://plan9.bell-labs.com/plan9/); which itself was succeeded by Inferno (http://www.vitanuova.com/inferno/). *** Beastie Bits The BSD Family Tree (https://github.com/freebsd/freebsd/blob/master/share/misc/bsd-family-tree) Unix Permissions Calculator (http://permissions-calculator.org/) NAS4Free release 11.0.0.4 now available (https://sourceforge.net/projects/nas4free/files/NAS4Free-11.0.0.4/11.0.0.4.4141/) Another BSD Mag released for free downloads (https://bsdmag.org/download/simple-quorum-drive-freebsd-ctl-ha-beast-storage-system/) OPNsense 17.1.4 released (https://forum.opnsense.org/index.php?topic=4898.msg19359) *** Feedback/Questions gozes asks via twitter about how get involved in FreeBSD (https://twitter.com/gozes/status/846779901738991620) ***

This is a very special BSD Now! New exciting changes are coming to the show and we're gonna cover them, so stick around or you'll miss it! Interview – Kris Moore – kris@trueos.org / @pcbsdKrisTrueOS founder, FreeNAS developer, BSD Now co-hostBenedict Reuschling – bcr@freebsd.org / @bsdbcrFreeBSD commiter & FreeBSD Foundation Vice President, BSD Now co-host Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) ***

This week on BSD Now, we will be discussing a wide variety of topics including Routers, Run-Controls, the “Rule” of silence and some This episode was brought to you by Headlines Ports no longer build on EOL FreeBSD versions (https://www.reddit.com/r/freebsd/comments/5ouvmp/ports_no_longer_build_on_eol_freebsd_versions/) The FreeBSD ports tree has been updated to automatically fail if you try to compile ports on EOL versions of FreeBSD (any version of 9.x or earlier, 10.0 - 10.2, or 11 from before 11.0) This is to prevent shooting yourself in the food, as the compatibility code for those older OSes has been removed now that they are no longer supported. If you use pkg, you will also run into problems on old releases. Packages are always built on the oldest supported release in a branch. Until recently, this meant packages for 10.1, 10.2, and 10.3 were compiled on 10.1. Now that 10.1 and 10.2 are EOL, packages for 10.x are compiled on 10.3. This matters because 10.3 supports the new openat() and various other *at() functions used by capsicum. Now that pkg and packages are built on a version that supports this new feature, they will not run on systems that do not support it. So pkg will exit with an error as soon as it tries to open a file. You can work around this temporarily by using the pkg-static command, but you should upgrade to a supported release immediately. *** Improving TrueOS: OpenRC (https://www.trueos.org/blog/improving-trueos-openrc/) With TrueOS moving to a rolling-release model, we've decided to be a bit more proactive in sharing news about new features that are landing. This week we've posted an article talking about the transition to OpenRC In past episodes you've heard me mention OpenRC, but hopefully today we can help answer any of those lingering questions you may still have about it The first thing always asked, is “What is OpenRC?” OpenRC is a dependency-based init system working with the system provided init program. It is used with several Linux distributions, including Gentoo and Alpine Linux. However, OpenRC was created by the NetBSD developer Roy Marples in one of those interesting intersections of Linux and BSD development. OpenRC's development history, portability, and 2-clause BSD license make its integration into TrueOS an easy decision. Now that we know a bit about what it is, how does it behave differently than traditional RC? TrueOS now uses OpenRC to manage all system services, as opposed to FreeBSD's RC. Instead of using rc.d for base system rc scripts, OpenRC uses init.d. Also, every service in OpenRC has its own user configuration file, located in /etc/conf.d/ for the base system and /usr/local/etc.conf.d/ for ports. Finally, OpenRC uses runlevels, as opposed to the FreeBSD single- or multi- user modes. You can view the services and their runlevels by typing $ rc-update show -v in a CLI. Also, TrueOS integrates OpenRC service management into SysAdm with the Service Manager tool One of the prime benefits of OpenRC is much faster boot-times, which is important in a portable world of laptops (and desktops as well). But service monitoring and crash detection are also important parts of what make OpenRC a substantial upgrade for TrueOS. Lastly people have asked us about migration, what is done, what isn't? As of now almost all FreeBSD base system services have been migrated over. In addition most desktop-facing services required to run Lumina and the like are also ported. We are still going through the ports tree and converting legacy rc.d scripts to init.d, but the process takes time. Several new folks have begun contributing OpenRC scripts and we hope to have all the roughly 1k ports converted over this year. BSDRP Releases 1.70 (https://sourceforge.net/projects/bsdrp/files/BSD_Router_Project/1.70/) A new release of the BSD Router Project This distro is designed to replace high end routers, like those from Cisco and Juniper, with FreeBSD running on regular off-the-shelf server. Highlights: Upgraded to FreeBSD 11.0-STABLE r312663 (skip 11.0 for massive performance improvement) Re-Added: netmap-fwd (https://github.com/Netgate/netmap-fwd) Add FIBsync patch to netmap-fwd from Zollner Robert netmap pkt-gen supports IPv6, thanks to Andrey V. Elsukov (ae@freebsd.org) bird 1.6.3 (add BGP Large communities support) OpenVPN 2.4.0 (adds the high speed AEAD GCM cipher) All of the other packages have also been upgraded A lot of great work has been done on BSDRP, and it has also generated a lot of great benchmarks and testing that have resulted in performance increases and improved understanding of how FreeBSD networking scales across different CPU types and speeds *** DragonFlyBSD gets UEFI support (http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/7b1aa074fcd99442a1345fb8a695b62d01d9c7fd) This commit adds support for UEFI to the Dragonfly Installer, allowing new systems to be installed to boot from UEFI This script (http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/9d53bd00e9be53d6b893afd79111370ee0c053b0) provides a way to build a HAMMER filesystem that works with UEFI There is also a UEFI man page (http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/d195d5099328849c500d4a1b94d6915d3c72c71e) The install media (http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/5fa778d7b36ab0981ff9dcbd96c71ebf653a6a19) has also been updated to support booting from either UEFI or MBR, in the same way that the FreeBSD images work *** News Roundup The Rule of Silence (http://www.linfo.org/rule_of_silence.html) “The rule of silence, also referred to as the silence is golden rule, is an important part of the Unix philosophy that states that when a program has nothing surprising, interesting or useful to say, it should say nothing. It means that well-behaved programs should treat their users' attention and concentration as being valuable and thus perform their tasks as unobtrusively as possible. That is, silence in itself is a virtue.” This doesn't mean a program cannot be verbose, it just means you have to ask it for the additional output, rather than having it by default “There is no single, standardized statement of the Unix philosophy, but perhaps the simplest description would be: "Write programs that are small, simple and transparent. Write them so that they do only one thing, but do it well and can work together with other programs." That is, the philosophy centers around the concepts of smallness, simplicity, modularity, craftsmanship, transparency, economy, diversity, portability, flexibility and extensibility.” “This philosophy has been fundamental to the the fact that Unix-like operating systems have been thriving for more than three decades, far longer than any other family of operating systems, and can be expected to see continued expansion of use in the years to come” “The rule of silence is one of the oldest and most persistent design rules of such operating systems. As intuitive as this rule might seem to experienced users of such systems, it is frequently ignored by the developers of other types of operating systems and application programs for them. The result is often distraction, annoyance and frustration for users.” “There are several very good reasons for the rule of silence: (1) One is to avoid cluttering the user's mind with information that might not be necessary or might not even be desired. That is, unnecessary information can be a distraction. Moreover, unnecessary messages generated by some operating systems and application programs are sometimes poorly worded, and can cause confusion or needless worry on the part of users.” No news is good news. When there is bad news, error messages should be descriptive, and ideally tell the user what they might do about the error. “A third reason is that command line programs (i.e., all-text mode programs) on Unix-like operating systems are designed to work together with pipes, i.e., the output from one program becomes the input of another program. This is a major feature of such systems, and it accounts for much of their power and flexibility. Consequently, it is important to have only the truly important information included in the output of each program, and thus in the input of the next program.” Have you ever had to try to strip out useless output so you could feed that data into another program? “The rule of silence originally applied to command line programs, because all programs were originally command line programs. However, it is just as applicable to GUI (graphical user interfaces) programs. That is, unnecessary and annoying information should be avoided regardless of the type of user interface.” “A example is the useless and annoying dialog boxes (i.e., small windows) that pop up on the display screen with with surprising frequency on some operating systems and programs. These dialog boxes contain some obvious, cryptic or unnecessary message and require the user to click on them in order to close them and proceed with work. This is an interruption of concentration and a waste of time for most users. Such dialog boxes should be employed only in situations in which some unexpected result might occur or to protect important data.” It goes on to make an analogy about Public Address systems. If too many unimportant messages, like advertisements, are sent over the PA system, people will start to ignore them, and miss the important announcements. *** The Tao of tmux (https://leanpub.com/the-tao-of-tmux/read) An interesting article floated across my news feed a few weeks back. It's what essentially boils down to a book called the “Tao of tmux”, which immediately piqued my interest. My story may be similar to many of yours. I was initially raised on using screen, and screen only for my terminal session and multiplexing needs. Since then I've only had a passing interest in tmux, but its always been one of those utilities I felt was worthy of investing some more time into. (Especially when seeing some of the neat setups some of my peers have with it) Needless to say, this article has been bookmarked, and I've started digesting some of it, but thought it would be good to share with anybody else who finds them-self in a similar situation. The book starts off well, explaining in the simplest terms possible what Tmux really is, by comparing and contrasting it to something we are all familiar with, GUIS! Helpfully they also include a chart which explains some of the terms we will be using frequently when discussing tmux (https://leanpub.com/the-tao-of-tmux/read#leanpub-auto-window-manager-for-the-terminal) One of the things the author does recommend is also making sure you are up to speed on your Terminal knowledge. Before getting into tmux, a few fundamentals of the command line should be reviewed. Often, we're so used to using these out of street smarts and muscle memory a great deal of us never see the relation of where these tools stand next to each other. Seasoned developers are familiar with zsh, Bash, iTerm2, konsole, /dev/tty, shell scripting, and so on. If you use tmux, you'll be around these all the time, regardless of whether you're in a GUI on a local machine or SSH'ing into a remote server. If you want to learn more about how processes and TTY's work at the kernel level (data structures and all) the book The Design and Implementation of the FreeBSD Operating System (2nd Edition) by Marshall Kirk McKusick is nice. In particular, Chapter 4, Process Management and Section 8.6, Terminal Handling. The TTY demystified by Linus Åkesson (available online) dives into the TTY and is a good read as well. We had to get that shout-out of Kirk's book in here ;) From here the boot/article takes us on a whirlwind journey of Sessions, Windows, Panes and more. Every control- command is covered, information on how to customize your statusbar, tips, tricks and the like. There's far more here than we can cover in a single segment, but you are highly encouraged to bookmark this one and start your own adventure into the world of tmux. *** SDF Celebrates 30 years of service in 2017 (https://sdf.org/) HackerNews thread on SDF (https://news.ycombinator.com/item?id=13453774) “Super Dimension Fortress (SDF, also known as freeshell.org) is a non-profit public access UNIX shell provider on the Internet. It has been in continual operation since 1987 as a non-profit social club. The name is derived from the Japanese anime series The Super Dimension Fortress Macross; the original SDF server was a BBS for anime fans[1]. From its BBS roots, which have been well documented as part of the BBS: The Documentary project, SDF has grown into a feature-rich provider serving members around the world.” A public access UNIX system, it was many people's first access to a UNIX shell. In the 90s, Virtual Machines were rare, the software to run them usually cost a lot of money and no one had very much memory to try to run two operating systems at the same time. So for many people, these type of shell accounts were the only way they could access UNIX without having to replace the OS on their only computer This is how I first started with UNIX, eventually moving to paying for access to bigger machines, and then buying my own servers and renting out shell accounts to host IRC servers and channel protection bots. “On June 16th, 1987 Ted Uhlemann (handle: charmin, later iczer) connected his Apple ][e's 300 baud modem to the phone line his mother had just given him for his birthday. He had published the number the night before on as many BBSes around the Dallas Ft. Worth area that he could and he waited for the first caller. He had a copy of Magic Micro BBS which was written in Applesoft BASIC and he named the BBS "SDF-1" after his favorite Japanimation series ROBOTECH (Macross). He hoped to draw users who were interested in anime, industrial music and the Church of the Subgenius.” I too started out in the world of BBSes before I had access to the internet. My parents got my a dedicated phone line for my birthday, so I wouldn't tie up their line all the time. I quickly ended up running my own BBS, the Sudden Death BBS (Renegade (https://en.wikipedia.org/wiki/Renegade_(BBS)) on MS DOS) I credit this early experience for my discovery of a passion for Systems Administration, that lead me to my current career “Slowly, SDF has grown over all these years, never forgetting our past and unlike many sites on the internet, we actually have a past. Some people today may come here and see us as outdated and "retro". But if you get involved, you'll see it is quite alive with new ideas and a platform for opportunity to try many new things. The machines are often refreshed, the quotas are gone, the disk space is expanding as are the features (and user driven features at that) and our cabinets have plenty of space for expansion here in the USA and in Europe (Germany).” “Think about ways you'd like to celebrate SDF's 30th and join us on the 'bboard' to discuss what we could do. I realize many of you have likely moved on yourselves, but I just wanted you to know we're still here and we'll keep doing new and exciting things with a foundation in the UNIX shell.” *** Getting Minecraft to Run on NetBSD (https://www.reddit.com/r/NetBSD/comments/5mtsy1/getting_minecraft_to_run_on_netbsd/) One thing that doesn't come up often on BSDNow is the idea of gaming. I realize most of us are server folks, or perhaps don't play games (The PC is for work, use your fancy-smanzy PS4 and get off my lawn you kids) Today I thought it would be fun to highlight this post over at Reddit talking about running MineCraft on NetBSD Now I realize this may not be news to some of you, but perhaps it is to others. For the record my kids have been playing Minecraft on PC-BSD / TrueOS for years. It's the primary reason they are more often booted into that instead of Windows. (Funny story behind that - Got sick of all the 3rd party mods, which more often than not came helpfully bundled with viruses and malware) On NetBSD the process looks a bit different than on FreeBSD. First up, you'll need to enable Linux Emulation and install Oracle JRE (Not OpenJDK, that path leads to sadness here) The guide will then walk us through the process of fetching the Linux runtime packages, extracting and then enabling bits such as ‘procfs' that is required to run the Linux binaries. Once that's done, minecraft is only a simple “oracle8-jre /path/to/minecraft.jar” command away from starting up, and you'll be “crafting” in no time. (Does anybody even play survival anymore?) *** Beastie Bits UNIX on the Computer Chronicals (https://youtu.be/g7P16mYDIJw) FreeBSD: Atheros AR9380 and later, maximum UDP TX goes from 250mbit to 355mbit. (https://twitter.com/erikarn/status/823298416939659264) Capsicumizing traceroute with casper (https://reviews.freebsd.org/D9303) Feedback/Questions Jason - TarSnap on Windows (http://pastebin.com/Sr1BTzVN) Mike - OpenRC & DO (http://pastebin.com/zpHyhHQG) Anonymous - Old Machines (http://pastebin.com/YnjkrDmk) Matt - Iocage (http://pastebin.com/pBUXtFak) Hjalti - Rclone & FreeNAS (http://pastebin.com/zNkK3epM)

This week on BSDNow, we've got all sorts of post-holiday goodies to share. New OpenSSL APIs, Dtrace, OpenBSD This episode was brought to you by Headlines OpenSSL 1.1 API migration path, or the lack thereof (https://www.mail-archive.com/tech@openbsd.org/msg36437.html) As many of you will already be aware, the OpenSSL 1.1.0 release intentionally introduced significant API changes from the previous release. In summary, a large number of data structures that were previously publically visible have been made opaque, with accessor functions being added in order to get and set some of the fields within these now opaque structs. It is worth noting that the use of opaque data structures is generally beneficial for libraries, since changes can be made to these data structures without breaking the ABI. As such, the overall direction of these changes is largely reasonable. However, while API change is generally necessary for progression, in this case it would appear that there is NO transition plan and a complete disregard for the impact that these changes would have on the overall open source ecosystem. So far it seems that the only approach is to place the migration burden onto each and every software project that uses OpenSSL, pushing significant code changes to each project that migrates to OpenSSL 1.1, while maintaining compatibility with the previous API. This is forcing each project to provide their own backwards compatibility shims, which is practically guaranteeing that there will be a proliferation of variable quality implementations; it is almost a certainty that some of these will contain bugs, potentially introducing security issues or memory leaks. I think this will be a bigger issue for other operating systems that do not have the flexibility of the ports tree to deliver a newer version of OpenSSL. If a project switches from the old API to the new API, and the OS only provides the older branch of OpenSSL, how can the application work? Of course, this leaves the issue, if application A wants OpenSSL 1.0, and application B only works with OpenSSL 1.1, how does that work? Due to a number of factors, software projects that make use of OpenSSL cannot simply migrate to the 1.1 API and drop support for the 1.0 API - in most cases they will need to continue to support both. Firstly, I am not aware of any platform that has shipped a production release with OpenSSL 1.1 - any software that supported OpenSSL 1.1 only, would effectively be unusable on every platform for the time being. Secondly, the OpenSSL 1.0.2 release is supported until the 31st of December 2019, while OpenSSL 1.1.0 is only supported until the 31st of August 2018 - any LTS style release is clearly going to consider shipping with 1.0.2 as a result. Platforms that are attempting to ship with OpenSSL 1.1 are already encountering significant challenges - for example, Debian currently has 257 packages (out of 518) that do not build against OpenSSL 1.1. There are also hidden gotchas for situations where different libraries are linked against different OpenSSL versions and then share OpenSSL data structures between them - many of these problems will be difficult to detect since they only fail at runtime. It will be interesting to see what happens with OpenSSL, and LibreSSL Hopefully, most projects will decide to switch to the cleaner APIs provided by s2n or libtls, although they do not provide the entire functionality of the OpenSSL API. Hacker News comments (https://news.ycombinator.com/item?id=13284648) *** exfiltration via receive timing (http://www.tedunangst.com/flak/post/exfiltration-via-receive-timing) Another similar way to create a backchannel but without transmitting anything is to introduce delays in the receiver and measure throughput as observed by the sender. All we need is a protocol with transmission control. Hmmm. Actually, it's easier (and more reliable) to code this up using a plain pipe, but the same principle applies to networked transmissions. For every digit we want to “send” back, we sleep a few seconds, then drain the pipe. We don't care about the data, although if this were a video file or an OS update, we could probably do something useful with it. Continuously fill the pipe with junk data. If (when) we block, calculate the difference between before and after. This is a our secret backchannel data. (The reader and writer use different buffer sizes because on OpenBSD at least, a writer will stay blocked even after a read depending on the space that opens up. Even simple demos have real world considerations.) In this simple example, the secret data (argv) is shared by the processes, but we can see that the writer isn't printing them from its own address space. Nevertheless, it works. Time to add random delays and buffering to firewalls? Probably not. An interesting thought experiment that shows just how many ways there are to covertly convey a message *** OpenBSD Desktop in about 30 Minutes (https://news.ycombinator.com/item?id=13223351) Over at hackernews we have a very non-verbose, but handy guide to getting to a OpenBSD desktop in about 30 minutes! First, the guide will assume you've already installed OpenBSD 6.0, so you'll need to at least be at the shell prompt of your freshly installed system to begin. With that, now its time to do some tuning. Editing some resource limits in login.conf will be our initial task, upping some datasize tunables to 2GB Next up, we will edit some of the default “doas” settings to something a bit more workable for desktop computing Another handy trick, editing your .profile to have your PKG_PATH variables set automatically will make One thing some folks may overlook, but disabling atime can speed disk performance (which you probably don't care about atime on your desktop anyway), so this guide will show you what knobs to tweak in /etc/fstab to do so After some final WPA / Wifi configuration, we then drop to “mere mortal” mode and begin our package installations. In this particular guide, he will be setting up Lumina Desktop (Which yes, it is on OpenBSD) A few small tweaks later for xscreensaver and your xinitrc file, then you are ready to run “startx” and begin your desktop session! All in all, great guide which if you are fast can probably be done in even less than 30 minutes and will result in a rock-solid OpenBSD desktop rocking Lumina none-the-less. *** How DTrace saved Christmas (https://hackernoon.com/dtrace-at-home-145ba773371e) Adam Leventhal, one of the co-creators of DTrace, wrote up this post about how he uses DTrace at home, to save Christmas I had been procrastinating making the family holiday card. It was a combination of having a lot on my plate and dreading the formulation of our annual note recapping the year; there were some great moments, but I'm glad I don't have to do 2016 again. It was just before midnight and either I'd make the card that night or leave an empty space on our friends' refrigerators. Adobe Illustrator had other ideas: “Unable to set maximum number of files to be opened” I'm not the first person to hit this. The problem seems to have existed since CS6 was released in 2016. None of the solutions were working for me, and — inspired by Sara Mauskopf's excellent post (https://medium.com/startup-grind/how-to-start-a-company-with-no-free-time-b70fbe7b918a#.uujdblxc6) — I was rapidly running out of the time bounds for the project. Enough; I'd just DTrace it. A colleague scoffed the other day, “I mean, how often do you actually use DTrace?” In his mind DTrace was for big systems, critical system, when dollars and lives were at stake. My reply: I use DTrace every day. I can't imagine developing software without DTrace, and I use it when my laptop (not infrequently) does something inexplicable (I'm forever grateful to the Apple team that ported it to Mac OS X) Illustrator is failing on setrlimit(2) and blowing up as result. Let's confirm that it is in fact returning -1:$ sudo dtrace -n 'syscall::setrlimit:return/execname == "Adobe Illustrato"/{ printf("%d %d", arg1, errno); }' dtrace: description 'syscall::setrlimit:return' matched 1 probe CPU ID FUNCTION:NAME 0 532 setrlimit:return -1 1 There it is. And setrlimit(2) is failing with errno 1 which is EPERM (value too high for non-root user). I already tuned up the files limit pretty high. Let's confirm that it is in fact setting the files limit and check the value to which it's being set. To write this script I looked at the documentation for setrlimit(2) (hooray for man pages!) to determine that the position of the resource parameter (arg0) and the type of the value parameter (struct rlimit). I needed the DTrace copyin() subroutine to grab the structure from the process's address space: $ sudo dtrace -n 'syscall::setrlimit:entry/execname == "Adobe Illustrato"/{ this->r = *(struct rlimit *)copyin(arg1, sizeof (struct rlimit)); printf("%x %x %x", arg0, this->r.rlimcur, this->r.rlimmax); }' dtrace: description 'syscall::setrlimit:entry' matched 1 probe CPU ID FUNCTION:NAME 0 531 setrlimit:entry 1008 2800 7fffffffffffffff Looking through /usr/include/sys/resource.h we can see that 1008 corresponds to the number of files (RLIMITNOFILE | _RLIMITPOSIX_FLAG) The quickest solution was to use DTrace again to whack a smaller number into that struct rlimit. Easy: $ sudo dtrace -w -n 'syscall::setrlimit:entry/execname == "Adobe Illustrato"/{ this->i = (rlimt *)alloca(sizeof (rlimt)); *this->i = 10000; copyout(this->i, arg1 + sizeof (rlimt), sizeof (rlimt)); }' dtrace: description 'syscall::setrlimit:entry' matched 1 probe dtrace: could not enable tracing: Permission denied Oh right. Thank you SIP (System Integrity Protection). This is a new laptop (at least a new motherboard due to some bizarre issue) which probably contributed to Illustrator not working when once it did. Because it's new I haven't yet disabled the part of SIP that prevents you from using DTrace on the kernel or in destructive mode (e.g. copyout()). It's easy enough to disable, but I'm reboot-phobic — I hate having to restart my terminals — so I went to plan B: lldb + After using DTrace to get the address of the setrlimit function, Adam used lldb to change the result before it got back to the application: (lldb) break set -n _init Breakpoint 1: 47 locations. (lldb) run … (lldb) di -s 0x1006e5b72 -c 1 0x1006e5b72: callq 0x1011628e0 ; symbol stub for: setrlimit (lldb) memory write 0x1006e5b72 0x31 0xc0 0x90 0x90 0x90 (lldb) di -s 0x1006e5b72 -c 4 0x1006e5b72: xorl %eax, %eax 0x1006e5b74: nop 0x1006e5b75: nop 0x1006e5b76: nop Next I just did a process detach and got on with making that holiday card… DTrace was designed for solving hard problems on critical systems, but the need to understand how systems behave exists in development and on consumer systems. Just because you didn't write a program doesn't mean you can't fix it. News Roundup Say my Blog's name! (https://functionallyparanoid.com/2016/12/22/say-my-blogs-name/) Brian Everly over at functionally paranoid has a treat for us today. Let us give you a moment to get the tin-foil hats on… Ok, done? Let's begin! He starts off with a look at physical security. He begins by listing your options: BIOS passwords – Not something I'm typically impressed with. Most can be avoided by opening up the machine, closing a jumper and powering it up to reset the NVRAM to factory defaults. I don't even bother with them. Full disk encryption – This one really rings my bell in a positive way. If you can kill power to the box (either because the bad actor has to physically steal it and they aren't carrying around a pile of car batteries and an inverter or because you can interrupt power to it some other way), then the disk will be encrypted. The other beauty of this is that if a drive fails (and they all do eventually) you don't have to have any privacy concerns about chucking it into an electronics recycler (or if you are a bad, bad person, into a landfill) because that data is effectively gibberish without the key (or without a long time to brute force it). Two factor auth for logins – I like this one as well. I'm not a fan of biometrics because if your fingerprint is compromised (yes, it can happen – read (https://www.washingtonpost.com/news/federal-eye/wp/2015/07/09/hack-of-security-clearance-system-affected-21-5-million-people-federal-authorities-say/) about the department of defense background checks that were extracted by a bad agent – they included fingerprint images) you can't exactly send off for a new finger. Things like the YubiKey (https://www.yubico.com/) are pretty slick. They require that you have the physical hardware key as well as the password so unless the bad actor lifted your physical key, they would have a much harder time with physical access to your hardware. Out of those options, Brian mentions that he uses disk encryption and yubi-key for all his secure network systems. Next up is network segmentation, in this case the first thing to do is change your admin password for any ISP supplied modem / router. He goes on to scare us of javascript attacks being used not against your local machine, but instead non WAN exposed router admin interface. Scary Stuff! For added security, naturally he firewalls the router by plugging in the LAN port to a OpenBSD box which does the 2nd layer of firewall / router protection. What about privacy and browsing? Here's some more of his tips: I use Unbound as my DNS resolver on my local network (with all UDP port 53 traffic redirected to it by pf so I don't have to configure anything on the clients) and then forward the traffic to DNSCrypt Proxy, caching the results in Unbound. I notice ZERO performance penalty for this and it greatly enhances privacy. This combination of Unbound and DNSCrypt Proxy works very well together. You can even have redundancy by having multiple upstream resolvers running on different ports (basically run the DNSCrypt Proxy daemon multiple times pointing to different public resolvers). I also use Firefox exclusively for my web browsing. By leveraging the tips on this page (https://www.privacytools.io/), you can lock it down to do a great job of privacy protection. The fact that your laptop's battery drain rate can be used to fingerprint your browser completely trips me out but hey – that's the world we live in.' What about the cloud you may ask? Well Brian has a nice solution for that as well: I recently decided I would try to live a cloud-free life and I'll give you a bit of a synopsis on it. I discovered a wonderful Open Source project called FreeNAS (http://www.freenas.org/). What this little gem does is allow you to install a FreeBSD/zfs file server appliance on amd64 hardware and have a slick administrative web interface for managing it. I picked up a nice SuperMicro motherboard and chassis that has 4 hot swap drive bays (and two internal bays that I used to mirror the boot volume on) and am rocking the zfs lifestyle! (Thanks Alan Jude!) One of the nicest features of the FreeNAS is that it provides the ability to leverage the FreeBSD jail functionality in an easy to use way. It also has plugins but the security on those is a bit sketchy (old versions of libraries, etc.) so I decided to roll my own. I created two jails – one to run OwnCloud (yeah, I know about NextCloud and might switch at some point) and the other to run a full SMTP/IMAP email server stack. I used Lets Encrypt (https://letsencrypt.org/) to generate the SSL certificates and made sure I hit an A on SSLLabs (https://www.ssllabs.com/) before I did anything else. His post then goes in to talk about Backups and IoT devices, something else you need to consider in this truely paranoid world we are forced to live in. We even get a nice shout-out near the end! Enter TarSnap (http://www.tarsnap.com/) – a company that advertises itself as “Online Backups for the Truly Paranoid”. It brings a tear to my eye – a kindred spirit! :-) Thanks again to Alan Jude and Kris Moore from the BSD Now podcast (http://www.bsdnow.tv/) for turning me onto this company. It has a very easy command syntax (yes, it isn't a GUI tool – suck it up buttercup, you wanted to learn the shell didn't you?) and even allows you to compile the thing from source if you want to.” We've only covered some of the highlights here, but you really should take a few moments of your time today and read this top to bottom. Lots of good tips here, already thinking how I can secure my home network better. The open source book: “Producing Open Source Software” (http://producingoss.com/en/producingoss.pdf) “How to Run a Successful Free Software Project” by Karl Fogel 9 chapters and over 200 pages of content, plus many appendices Some interesting topics include: Choosing a good name version control bug tracking creating developer guidelines setting up communications channels choosing a license (although this guide leans heavily towards the GPL) setting the tone of the project joining or creating a Non-Profit Organization the economics of open source release engineering, packaging, nightly builds, etc how to deal with forks A lot of good information packaged into this ebook This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License *** DTrace Flamegraphs for node.js on FreeBSD (http://www.venshare.com/dtrace-flamegraphs-for-freebsd-and-node-js-2/) One of the coolest tools built on top of DTrace is flamegraphs They are a very accurate, and visual way to see where a program is spending its time, which can tell you why it is slow, or where it could be improved. Further enhancements include off-cpu flame graphs, which tell you when the program is doing nothing, which can also be very useful > Recently BSD UNIXes are being acknowledged by the application development community as an interesting operating system to deploy to. This is not surprising given that FreeBSD had jails, the original container system, about 17 years ago and a lot of network focused businesses such as netflix see it as the best way to deliver content. This developer interest has led to hosting providers supporting FreeBSD. e.g. Amazon, Azure, Joyent and you can get a 2 months free instance at Digital Ocean. DTrace is another vital feature for anyone who has had to deal with production issues and has been in FreeBSD since version 9. As of FreeBSD 11 the operating system now contains some great work by Fedor Indutny so you can profile node applications and create flamegraphs of node.js processes without any additional runtime flags or restarting of processes. This is one of the most important things about DTrace. Many applications include some debugging functionality, but they require that you stop the application, and start it again in debugging mode. Some even require that you recompile the application in debugging mode. Being able to attach DTrace to an application, while it is under load, while the problem is actively happening, can be critical to figuring out what is going on. In order to configure your FreeBSD instance to utilize this feature make the following changes to the configuration of the server. Load the DTrace module at boot Increase some DTrace limits Install node with the optional DTrace feature compiled in Follow the generic node.js flamegraph tutorial (https://nodejs.org/en/blog/uncategorized/profiling-node-js/) > I hope you find this article useful. The ability to look at a runtime in this manor has saved me twice this year and I hope it will save you in the future too. My next post on freeBSD and node.js will be looking at some scenarios on utilising the ZFS features. Also check out Brendan Gregg's ACM Queue Article (http://queue.acm.org/detail.cfm?id=2927301) “The Flame Graph: This visualization of software execution is a new necessity for performance profiling and debugging” SSHGuard 2.0 Call for Testing (https://sourceforge.net/p/sshguard/mailman/message/35580961/) SSHGuard is a tool for monitoring brute force attempts and blocking them It has been a favourite of mine for a while because it runs as a pipe from syslogd, rather than reading the log files from the disk A lot of work to get SSHGuard working with new log sources (journalctl, macOS log) and backends (firewalld, ipset) has happened in 2.0. The new version also uses a configuration file. Most importantly, SSHGuard has been split into several processes piped into one another (sshg-logmon | sshg-parser | sshg-blocker | sshg-fw). sshg-parser can run with capsicum(4) and pledge(2). sshg-blocker can be sandboxed in its default configuration (without pid file, whitelist, blacklisting) and has not been tested sandboxed in other configurations. Breaking the processes up so that the sensitive bits can be sandboxes is very nice to see *** Beastie Bits pjd's 2007 paper from AsiaBSDCon: “Porting the ZFS file system to the FreeBSD operating system” (https://2007.asiabsdcon.org/papers/P16-paper.pdf) A Message From the FreeBSD Foundation (https://vimeo.com/user60888329) Remembering Roger Faulkner, Unix Champion (http://thenewstack.io/remembering-roger-faulkner/) and A few HN comments (including Bryan Cantrill) (https://news.ycombinator.com/item?id=13293596) Feedback/Questions Peter - TrueOS Network (http://pastebin.com/QtyJeHMk) Chris - Remote Desktop (http://pastebin.com/ru726VTV) Goetz - Geli on Serial (http://pastebin.com/LQZPgF5g) Joe - BGP (http://pastebin.com/jFeL8zKX) Alejandro - BSD Router (http://pastebin.com/Xq9cbmfn) ***

A look back at 2016 This episode was brought to you by LinksZFS in the trenches | BSD Now 123One small step for DRM, one giant leap for BSD | BSD Now 143The Laporte has landed! | BSD Now 152Ham, Radio & Pie, Oh My! | BSD Now 158The Foundation of NetBSD | BSD Now 162Return of the Cantrill | BSD Now 163

This week on BSDNow, we have some big breaking news about another major switcher to FreeBSD, plus early information about the pending This episode was brought to you by Headlines Leo Laporte tries FreeBSD (http://www.leolaporte.com/blog/a-grand-experiment) Leo Laporte, formerly of TechTV, and now of TWiT.tv, is switching to FreeBSD “The latest debacle over the "forced" upgrade to Windows 10 and Apple's increasingly locked-in ecosystem has got me thinking. Do I really need to use a proprietary operating system to get work done? And while I'm at it, do I need to use commercial cloud services to store my data?” A sometimes Linux user since the mid 90s, Leo talks about his motivations: “But as time went by, even Ubuntu began to seem too commercial to me” “So now for the grand experiment. Is it possible, I wonder, to do everything I need to do on an even more venerable, more robust system: a true UNIX OS, FreeBSD? Here are my requirements” Browsing Email with PGP signing and encryption Coding - I'm a hobbyist programmer requiring support for lisp/scheme/racket, rust, and python (and maybe forth and clojure and meteor and whatever else is cool and new) Writing A password vault. I currently use Lastpass because it syncs with mobile but eventually I'll need to find a FOSS replacement for that, too Photo editing - this is the toughest to replace. I love Photoshop and Lightroom. Can I get by with, say, GIMP and Darktable? I do all of those things on my PCBSD machine all the time “I love Linux and will continue to use it on my laptops, but for my main workhorse desktop I think FreeBSD will be a better choice. I also look forward to learning and administering a true UNIX system.” He got a nice SuperMicro based workstation, with an Intel Xeon E3-1275v5 and an NVIDIA GeForce GTX 960 GPU I have a server with one of those Skylake E3s, it is very nice “450Mbps Wireless N Dual Band PCI-e Adapter w/ 3x 2dBi Antennas (Yes, sad to say, unless I rewire my house I'll have to use Wi-Fi with this beast. I'll probably rewire my house.)” He plans to have a 4x 1TB ZFS pool, plus a second pool backed by a 512 GB NVMe m.2 for the OS “And I'll continue to chronicle my journey into the land of FOSS here when The Beast arrives. But in the meantime, please excuse me, I've got some reading to do.” Leo went so far as to slap a “Power By FreeBSD” sticker (https://youtu.be/vNVst_rxxm0?t=270) on the back of his new Tesla *** OpenBSD 6.0 to be released on Sept 1st, 2016 (http://undeadly.org/cgi?action=article&sid=20160725100831) OpenBSD 6.0 Tenative Released Notes (https://www.openbsd.org/60.html) OpenBSD 6.0 is just around the corner, currently slated for Sept 1st and brings with it a whole slew of exciting new features First up, and let's get this right out of the way.. VAX support has been dropped!! Oh no! However to make up for this devastating loss, armv7 has been added to this release. The tentative release notes are very complete and marks 6.0 as quite an exciting release OpenBSD 6.0 Pre-orders up (http://undeadly.org/cgi?action=article&sid=20160726230851) OpenBSD 6.0 tightens security by losing Linux compatibility (http://www.infoworld.com/article/3099038/open-source-tools/openbsd-60-tightens-security-by-losing-linux-compatibility.html) In related news, infoworld picked up on the pending removal of Linux compat from OpenBSD 6.0. Touted as a security feature, you will soon be unable to run legacy linux binaries on OpenBSD. This has both positives and negatives depending upon your use case. Ironically we're excitedly awaiting improved Linux Compat support in FreeBSD, to allow running some various closed-source applications. (Netflix DRM, Steam, Skype to name a few) *** EuroBSDCon 2016 Schedule released (https://2016.eurobsdcon.org/talks-schedule/) EuroBSDCon 2016 Tutorial Schedule released (https://2016.eurobsdcon.org/tutorials/) EuroBSDCon has announced the list of talks and tutorials for September 22nd-25th's conference! George Neville Neil (Who we've interviewed in the past) is giving the keynote about “The Coming Decades of BSD” *** News Roundup Blast from the past No interview again this week, we're working on getting some people lined up. The Leo Laporte story brought these old gem from TechTV into my youtube playlist: Matt Olander and Murrey Stokey explain FreeBSD on TechTV (https://www.youtube.com/watch?v=d0UsXwRvaIg) Matt Olander and Brooks Davis explain building a cluster with FreeBSD on TechTV (https://www.youtube.com/watch?v=bAsYz5pVwyc) FreeBSD vs Linux Part 1 (https://www.youtube.com/watch?v=91igg2UX7o8) FreeBSD vs Linux Part 2 (https://www.youtube.com/watch?v=oU88fQkwfws) *** Running FreeBSD on the LibreM (https://ericmccorkleblog.wordpress.com/2016/07/16/freebsd-librem-update/) Eric McCorkle (Who has worked on the EFI loader for a while now) has written an update on his efforts to get FreeBSD working properly on the LibreM 13 laptop. Since April the work seems to be progressing nicely Matt Macy's i915 graphics patch works well on the Librem 13, and I personally made sure that the suspend/resume support works. The patch is very stable on the Librem, and I've only had one kernel panic the entire time testing it. The HDMI output Just Works™ with the i915 driver. Even better, it works for both X11 and console modes. Full support for the Atheros 9462 card has been merged in. I've had some occasional issues, but it works for the most part. The vesa weirdness is obviated by i915 support, but it was resolved by using the scfb driver. Some of the outstanding issues still being worked on are support for Synaptics on this particular touchpad, as well as hotkey support for the keyboard, and brightness controls. In addition Eric is still working on the EFI + Geli support, with the eventual goal of getting EFI secure-boot working out of box as well. More OpenBSD syscall fuzzing (http://seclists.org/oss-sec/2016/q3/157) NCC Group's Project Triforce continues its work of fuzzing OpenBSD This time they have found a flaw that allows any user to panic the kernel Attempting to read from the tmpfs_vfsops sysctl tree will panic the system: “attempt to execute user address 0x0 in supervisor mode” This is actually a “good” thing… “Impact: Any user can panic the kernel by using the sysctl call. If a user can manage to map a page at address zero, they may be able to gain kernel code execution and escalate privileges” OpenBSD's default configuration prevents mapping a page at address zero, so the code execution is prevented So while a panic is a bad outcome, it is a lot better than it could have been *** Root privilege escalation on NetBSD (http://akat1.pl/?id=2) This post described a root privilege escalation in NetBSD mail.local is a utility included in the base system for delivering mail to other users on the same system, rather than invoking a mail client and going through the mail server. The mail.local utility contains a ‘time of check / time of use' vulnerability. This means that it checks if a file or permission is valid, and then later accesses that file. If an attacker can change that file between the time when it is checked, and the time when it is used, they may be able to exploit the system by evading the check This is exactly what happens in this case mail.local appends a message to the indicated user's mailbox It first checks if the target user already has an existing mailbox file. If the file exists, but is a link, mail.local exits with an error (to prevent exploits) If the file does not exist, it is created The message is then appended to the file If the file needed to be created, it is chown'd to the owner of the mailbox This is where the problem lies, if mail.local checks and does not find the mailbox, but an attacker then creates a link from the target mailbox to some other file mail.local then appends to that file instead, thinking it is creating the new mailbox Then, mail.local chown's the target file to the user the attacker was trying to send mail to The article explains how this could be used to replace /etc/master.passwd etc, but opts for an easier proof of concept, replacing /usr/bin/atrun, which is run as root every 5 minutes from crontab with a script that will copy the shell to /tmp and mark it setuid The attacker can then run that shell out of /tmp, and be root NetBSD fixed the vulnerability by changing the code flow, separating the cases for opening an existing file from creating a new file. In the case where an existing file is opened, the code then verifies that the file that was opened has the same inode number and is on the same device, as the file that was checked earlier, to ensure it was not a link *** FreeBSD Heap vulnerability in bspatch (https://www.freebsd.org/security/advisories/FreeBSD-SA-16:25.bspatch.asc) An important vuln has been found and fixed in FreeBSD this past week, specifically relating to the ‘bspatch' utility. “Upstream's bspatch.c implementation doesn't check for negative values on the number of bytes to read from the "diff" and "extra" streams, allowing an attacker controlling the patch file to write at arbitrary locations in the heap.” This could result in a crash, or running arbitrary code as the user running bspatch. (Often root) “bspatch's main loop reads three numbers from the "control" stream in the patch: X, Y and Z. The first two are the number of bytes to read from "diff" and "extra" (and thus only non-negative), while the third one could be positive or negative and moves the oldpos pointer on the source image. These 3 values are 64bits signed ints (encoded somehow on the file) that are later passed the function that reads from the streams, but those values are not verified to be non-negative.” “Chrome[OS] has four different implementations of this program, all derived from the same original code by Colin Percival.” Chromium Issue Tracker (https://bugs.chromium.org/p/chromium/issues/detail?id=372525) Patch your systems now! *** Beastie Bits: If you're a BUG member or Organizer, please contact BSD Now (https://twitter.com/q5sys/status/758087886927388673) TedU writes about some interesting localizations to gcc in openbsd, and why they are there (http://www.tedunangst.com/flak/post/one-reason-to-hate-openbsd) List of Products based on FreeBSD -- Help complete the list (https://en.wikipedia.org/wiki/List_of_products_based_on_FreeBSD) Virtualbox v5 hits the FreeBSD Ports tree (http://www.freshports.org/emulators/virtualbox-ose/) Skull Canyon NUC booting FreeBSD 11.0-BETA2 (https://gist.github.com/gonzopancho/b71be467f45594822131f4816d6cb718) 2016 BSDCan Trip Report : Trent Thompson (https://www.freebsdfoundation.org/blog/2016-bsdcan-trip-report-trent-thompson/) August London BSD Meetup (http://mail-index.netbsd.org/regional-london/2016/07/25/msg000542.html) Feedback/Questions Michael Open-Source Alts (http://pastebin.com/eiWbDXTd) Herminio - AP Troubles (http://pastebin.com/w9aCDBut) Jake - Plasma (http://pastebin.com/d15QpVFw) Morgan - Clean DO Droplets (http://pastebin.com/Wj1P7jq8) Chris - Auditd (http://pastebin.com/U9PYEH6K) ***

This week on the show, Allan and I are going to be showing you a very interesting interview we did talking about using FreeBSD to drive This episode was brought to you by Headlines FreeBSD Core Team Election (https://www.freebsd.org/administration.html#t-core) Core.9 has been elected, and will officially take over from Core.8 on Wednesday, 6 July 2016 Many thanks to the outgoing members of the core team for their service over the last 2 years 214 out of 325 eligible voters (65.8%) cast their votes in an election counting 14 candidates. The top nine candidates are, in descending order of votes received: 180 84.1% Ed Maste (incumbent) 176 82.2% George V. Neville-Neil (incumbent) 171 79.9% Baptiste Daroussin (incumbent) 168 78.5% John Baldwin 166 77.6% Hiroki Sato (incumbent) 147 68.7% Allan Jude 132 61.7% Kris Moore 121 56.5% Benedict Reuschling 108 50.5% Benno Rice There was no tie for ninth. BSDNow and the entire community would also like to extend their thanks to all those who stood for election to the core team Next week's core meeting will encompass the members of Core.8 and Core.9, as responsibility for any outstanding items will be passed from outgoing members of core to the new incoming members *** Why I run OpenBSD (http://deftly.net/posts/2016-05-31-why-i-run-openbsd.html) This week we have a good article / blog post talking about why the posted has moved to OpenBSD from Linux. “One thing I learned during my travels between OSs: consistency is everything. Most operating systems seem to, at least, keep a consistent interface between themselves and binaries / applications. They do this by keeping consistent APIs (Application Programming Interfaces) and ABIs (Application Binary Interfaces). If you take a binary from a really old version of Linux and run or build it on a brand-spanking new install of Linux, it will likely Just Work™. This is great for applications and developers of applications. Vendors can build binaries for distribution and worry less about their product working when it gets out in the wild (sure this binary built in 2016 will run on RedHat AS2.1!!).“ The author then goes through another important part of the consistency argument, with what he calls “UPI” or “User Program Interfaces”. In other words, while the ABI may be stable, what about the end-user tooling that the user directly has to interact with on a daily basis? “This inconsistency seems to have come to be when Linux started getting wireless support. For some reason someone (vendors, maybe?) decided that ifconfig wasn't a good place to let users interact with their wireless device. Maybe they felt their device was special? Maybe there were technical reasons? The bottom line is, someone decided to create a new utility to manage a wireless device… and then another one came along… pretty soon there was iwconfig(8), iw(8), ifconfig(8), some funky thing that let windows drivers interface with Linux.. and one called ip(8) I am sure there are others I am forgetting, but I prefer to forget. I have moved onto greener pastures and the knowledge of these programs no longer serves me.” The article then goes through the rundown of how he evaluated the various BSD's and ultimately settled on OpenBSD: “OpenBSD won the showdown. It was the most complete, simple, and coherent system. The documentation was thorough, the code was easy to follow and understand. It had one command to configure all of the network interfaces! I didn't have wireless, but I was able to find a cheap USB adapter that worked by simply running man -k wireless and reading about the USB entries. It didn't have some of the applications I use regularly, so I started reading about ports (intuitively, via man ports!).” The ultimate NetBSD Router (http://blog.tbrodel.me/2016/#netbsd-router) “So yesterday I spent the day setting up a new firewall at home here, based off of this BSD Now tutorial. Having set up a couple of OpenBSD routers before, either based on old laptops, bulky old power-sucking desktops or completely over-specced machines like the Intel NUC, I wanted to get some kind of BSD onto a low-powered ARM board and use that instead.” “I've had a couple of Cubietrucks lying around for a while now, I've used them in a couple of art installations, running Debian and Pure Data, but over all they've been a bit disappointing. It's more the manufacturer's fault but they require blobs for the graphics and audio, which Debian won't allow, so as a multimedia board they're dud for video, and only passable for audio work with a usb sound card. So they've been collecting dust.” “Only thing missing is a second NIC, luckily I had an Apple USB->Ethernet dongle lying around, which when I bought it was the cheapest thing I could find on eBay that OpenBSD definitely supported. There, and on NetBSD, it's supported by the axe(4) driver. USB 2.0 works fine for me as I live in Australia and my ISP can only give me 30Mbps, so this should do for the forseeable future.” + The article then walks through installing and configuring NetBSD + Configuration includes: pf, unbound, and dhcpd “This project has been really fun, I started with basically no experience with NetBSD and have finished with a really useful, low-powered and robust appliance. It's a testament to the simplicity of the NetBSD system, and the BSD design principles in general, that such a novice as myself could figure this out. The NetBSD project has easily the most polished experience on Allwinner ARM boards, even Debian doesn't make it this easy. It's been a joy running the system, it has the bits I love from OpenBSD; ksh(1), tmux(1), an http daemon in base and of course, pf(4). This is mixed with some of the pragmatism I see in FreeBSD; a willingness to accept blobs if that really is the only way to boot, or get audio, or a video console.” bhyve-Bootable Boot Environments (http://callfortesting.org/bhyve-boot-environments/) We have a lengthy article also today from our friend Michael Dexter, who asks the basic question “What if multibooting and OS upgrades weren't horrible?” No doubt if you've been a frequent listener to this show, you've heard Allan or Myself talking about ZFS Boot Environments, and how they can “change your life”. Well today Michael goes further into detail on how the BE's work, and how they can be leveraged to do neat things, like installing other versions of an operating system from the original running system. “If you are reading this, you have probably used a personal computer with a BSD or GNU/Linux operating system and at some point attempted to multiboot between multiple operating systems on the same computer. This goal is typically attempted with complex disk partitioning and a BSD or GNU/Linux boot loader like LILO or GRUB, plus several hours of frustrating experimentation and perhaps data loss. While exotic OS experimentation has driven my virtualization work since the late 1990s, there are very pragmatic reasons for multibooting the same OS on the same hardware, notable for updates and failback to "known good" versions. To its credit, FreeBSD has long had various strategies including the NanoBSD embedded system framework with primary and secondary root partitions, plus the nextboot(8) utility for selecting the "next" kernel with various boot parameters. Get everything set correctly and you can multiboot "with impunity". “That's a good start, and over time we have seen ZFS "boot environments" be used by PC-BSD and FreeNAS to allow for system updates that allow one to fall back to previous versions should something go wrong. Hats off to these efforts but they exist in essentially purpose-built appliance environments. I have long sensed that there is more fun to be had here and a wonderful thing happened with FreeBSD 10.3 and 11.0: Allan Jude added a boot environment menu to the FreeBSD loader” From here Michael takes us through the mechanical bits of actually creating a new ZFS dataset (BE) and performing a fresh FreeBSD 10.3 installation into this new boot-environment. The twist comes at the end, where he next sets up the BE to be a root NFS for booting in bhyve! This is interesting and gives you a way to test booting into your new environment via a VM, before rebooting the host directly into it. *** Interview - Edicarla Andrade & Vinícius Zavam - @egypcio (https://twitter.com/egypcio) BSD-Powered Robots News Roundup Tomohiro Kasumi explains what “@@” means, in the context of the Hammer filesystem (http://lists.dragonflybsd.org/pipermail/users/2016-June/249717.html) A post from the Dragonfly users' mailing list about what the @@ construct means in the Hammer filesystem “@@ represents the existence of a PFS which is logically separated pseudo filesystem space within HAMMER's B-Tree” “HAMMER only has 1 large B-Tree per filesystem (not per PFS), so all the PFS exist within that single B-Tree. PFS are separated by localization parameter which is one of the B-Tree keys used to lookup the tree.” Each substring in "@@-1:00001" means: "@@" means it's a PFS or snapshot. "-1" means it's a master. ":" is just a separator. "00001" means it's PFS#1, where PFS#0 is the default PFS created on newfs. There is no "00000" because that's what's mounted on /HAMMER. PFS# is used for localization parameter. “Localization parameter has the highest priority when inserting or looking up B-Tree elements, so fs elements that belong to the same PFS# tend to be localized (clustered) within the B-Tree” There is also a note about how snapshots are named: "@@0x00..." A user points out that having : in the path can confuse some applications, such as in the case of adding the current directory or a relative path to the $PATH environment variable, which is a colon delimited list of paths This seems quite a bit more confusing that the datasets created by ZFS, but they might have other useful properties *** FreeBSD 11.0 nearing RC1 (https://www.freebsd.org/releases/11.0R/schedule.html) We've all been eagerly awaiting the pending release of FreeBSD 11.0, and the schedule has now been updated! The first release candidate is slated for July 29th! If all goes well (and we stick to schedule) there will be another RC2 and possible RC3 release, before 11.0 officially drops near the end of August. Start playing with those builds folks, be sure to send your feedback to the team to make this the best .0 release ever! *** TensorFlow on FreeBSD (http://ecc-comp.blogspot.com/2016/06/tensorflow-on-freebsd.html) Next we have a blog post about the experience of a “new” FreeBSD user trying to deploy some non-ported software to his new system. Specifically he was interested in running TensorFlow, but not doing a port himself, because in his words: “First, I apologize for not supplying a port archive myself. After reading the FreeBSD handbook for creating a port, it's too complex of a task for me right now. I've only been using FreeBSD for two weeks. I would also not like to waste anyone's time giving them a terrible port archive and mess up their system.” First of all, good ports are often born out of bad ports! Don't let the porting framework daunt you, give it a go, since that's the only way you are going to learn how to write “good” ports over time. The porters-handbook is a good first place to start, plus the community usually is very helpful in providing feedback. He then walks us through the changes made to the TensorFlow code (starting with the assumption that OSX was a good “flavor” to begin porting from) and ultimately compiling. This ends up with the creation of a pip package which works! A good tutorial, and also very similar to what goes on in the porting process. With this write-up perhaps somebody will take up creating a port of it… hint hint! *** NetBSD: A New Beginning? (http://jamesdeagle.blogspot.ca/2016/06/netbsd-new-beginning.html) We don't get enough NetBSD news at times, but this post by James Deagle talks about his adventure with NetBSD 7.0 and making it his “new beginning” “After a few months of traipsing around the worlds of SunOS and Linux, I'm back to NetBSD for what I hope will be a lengthy return engagement. And while I'm enamored of NetBSD for all the previously-mentioned reasons, I'm already thinking ahead to some problems to solve, some of which have also been mentioned before.” He then goes through and lists some of the small nits he's still running into during the daily workflow YouTube audio - Specifically he mentions that no audio is playing, but wonders if Flash plays some part. (Ideally you're not using Flash though, in which case you need to check the audio backend FF is using. Try PulseAudio since it seems the best supported. If pulse is already enabled, install ‘pavucontrol' to make sure audio is playing to the correct sound device) Slow gaming performance (TuxKart and Celestia) - Check DRI / Xorg? Or is it CPU bound? Lastly some unspecified Wireless issues, which typically end up being driver related. (Or use another chipset) Beastie Bits Reproducible NetBSD? 77.7% of the way there (https://reproducible.debian.net/netbsd/netbsd.html) Create FreeBSD virtual machine using qemu. Run the VM using xhyve. (https://gist.github.com/zg/38a3afa112ddf7de4912aafc249ec82f) FreeBSD PowerPC 32bit pkg repository (unofficial). ~19,500 packages, more to come (https://joshcummings.net/pub/FreeBSD) NetBSD machines at Open Source Conference 2016 Gunma (http://mail-index.netbsd.org/netbsd-advocacy/2016/05/16/msg000706.html) Adam Leventhal (of ZFS and DTrace) does an analysis of APFS (http://arstechnica.com/apple/2016/06/a-zfs-developers-analysis-of-the-good-and-bad-in-apples-new-apfs-file-system/) SemiBug June meeting summary (http://lists.nycbug.org/pipermail/semibug/2016-June/000106.html) KnoxBug Meeting (http://knoxbug.org/content/2016-07-26) Feedback/Questions Andrew - iocage (http://pastebin.com/nuYTzaG6) Florian - Arm + GitHub (http://pastebin.com/PzY68hNS) Clint - Synth (http://pastebin.com/JESGZjLu) Leonardo - Translations (http://pastebin.com/b4LAiPs4) Zachary - Moving things to VMs (http://pastebin.com/VRc8fvBk) ***

This week on BSD Now, I'm out of town for the week, but we have a special unboxing video to share with you, that you won't want to miss. That, plus the latest BSD news, is coming your way right now! This episode was brought to you by Headlines Example of a FreeBSD bug hunting session by a simple user (http://blog.cochard.me/2016/01/example-of-freebsd-bug-hunting-session.html) Don't be fooled, Olivier Cochard-Labbé is a bit more than just a FreeBSD user Original founder of the FreeNAS project many years ago, and currently leads the BSD Router Project (designed as a replacement for “Big Iron” routers like Cisco's etc) However, he is not actually a committer on any of the BSD projects, and is mostly focused on networking, rather than development, so it is fair to call him a user He walks us through a bug hunting session that started when he updated his wireless router “My wireless-router configuration was complex: it involves routing, wireless in hostap mode, ipfw, snort, bridge, openvpn, etc.” Provides helpful advice on writing problem reports to developers, including trying to reproduce your issue with as minimal a setup as possible. This both reduces the amount of setup a developer has to do to try to recreate your issue, and can often make it more obvious where the problem actually lies As you might expect, the more he researched the problem, the more questions he had The journey goes through the kernel debugger, learning dtrace, and reading some source code In the end it seems the problem is that the bridge interface marks itself as down if none of the interfaces are in an ‘UP' state. The wireless interface was in the unknown state, and was actually up, but when the wired interface was disconnected, this caused the bridge to mark it self as down. *** How-to Install OpenBSD 5.9 plus XFCE desktop and basic applications (http://ribalinux.blogspot.com/2016/04/how-to-install-openbsd-59-plus-xfce.html) Now this is the way to do videos. Over at the RibaLinux blogspot site, we have a great video showing how to setup and install OpenBSD 5.9 with XFCE and basic desktop applications. Along with the video tutorial, another nicety is the commands-used script, so you can see exactly how the setup was done, without having to pause/rewind the video to keep up. How to install PC-BSD 10.3 (http://ribalinux.blogspot.com/2016/04/how-to-install-pc-bsd-103.html) In addition to the OpenBSD 5.9 setup video, they just published a PC-BSD 10.3 installation video as well, check it out! *** FreeBSD on xhyve tutorial (https://gist.github.com/tanb/f8fefa22332edc7a641d) Originally only able to boot linux, xhyve, a “sort of” port of bhyve to OS X, can now run FreeBSD This tutorial makes it much easier, providing a script There are a few small command line flag differences from bhyve on FreeBSD The tutorial also covers sharing a directory between the guest and the host, resizing and growing the disk for the guest, and converting a QEMU image to be run under xhyve *** How to Configure SSHguard With IPFW Firewall On FreeBSD (http://www.unixmen.com/configure-sshguard-ipfw-firewall-freebsd) It's been a while, but UNIXMen has dropped on us another FreeBSD tutorial, this time on how to setup IPFW and ‘sshguard' to protect your system. In this tutorial they first lay down the rationale for picking IPFW as the firewall, but the reasons mainly boil down to IPFW being developed primarily on FreeBSD, and as such isn't lagging behind when it comes to features / support. Interestingly enough, they also go the route of adding their own /usr/local/etc/rc.firewall script which will be used to specify TCP/UDP ports to open through IPFW via the rc.conf file Once that setup is complete (which you can just copy-n-paste) they then move onto ‘sshguard' setup. Specifically you'll need to be sure to install the correct port/pkg, sshguard-ipfw in order to work in this setup, although sshguard-pf and friends are available also. The article mentions that the name ‘sshguard' can also be misleading, since it can be used to detect brute force attempts into a number of services. From there a bunch of configuration is thrown at you, which will allow you to start making the most out of sshguard's potential, well worth your read if you are using IPFW, or even PF and want to get the basics down of using sshguard properly. *** FreeNAS Mini XL Video Unboxing Beastie Bits Amazon lists FreeBSD as 'Other Linux' (https://i.imgur.com/NJ7lpso.png) sbin/hammer: Make hammer commands print root volume path (http://lists.dragonflybsd.org/pipermail/commits/2016-April/459667.html) sbin/hammer: Print volume list after volume-add|del (http://lists.dragonflybsd.org/pipermail/commits/2016-April/459674.html) Front cover reveal for the upcoming 'FreeBSD Mastery: Advanced ZFS" book (https://twitter.com/mwlauthor/status/716328414072872960) If you don't already have one, get your FreeBSD Pillow (http://linuxpillow.blogspot.com/2016/03/world-backup-day.html) Feedback/Questions Daniel - SysVIPC (http://pastebin.com/raw/JBbMj87t) Shane - OpenToonz (http://pastebin.com/raw/54ngYVEN) ***

This week on BSDNow, Allan is back from the Storage Summit in Silicon Valley! We are going to get his thoughts on how the conference went, plus bring you the latest ZFS info discussed. That plus the usual BSD news is This episode was brought to you by Headlines OpenBSD website operators urged to fix mind-alteringly bad bug (http://www.theregister.co.uk/2016/02/21/openbsd_website_operator_patch_now_for_the_sake_of_your_sanity/?mt=1456206806399) We start off a bit light-hearted this week, with the important, breaking news that finally a long-standing OpenBSD bug has been addressed for the HTTP daemon. Specifically? It changes the default 404 page fonts away from Comic Sans, to a bit more crowd-pleasing alternative: “For some reason the httpd status pages (e.g. 404) use the Comic Sans typeface. This patch removes comic sans and sets the typeface to the default sans-serif typeface of the client. “This lowers the number of people contacting website maintainers with typeface complaints bordering on harassment”. Operators running HTTPD are highly encouraged to update their systems to the latest code, right now……... No seriously, we are waiting for you. Get it done now and then we'll continue with the show. Registration for AsiaBSDCon 2016 is now open + Talk Schedule (https://2016.asiabsdcon.org/registration/?lang=en) After a few delays, the registration for AsiaBSDCon has now opened! The conference starts in less than two weeks! now, so be sure to get signed up ASAP. In addition the schedule has been posted, and here's some of the highlights of this year's conference. In addition to FreeBSD and NetBSD dev summits on the first two days, we have some excellent tutorials being given this year by Kirk, Gnn, Dru and more! (https://2016.asiabsdcon.org/program.html.en) The regular paper talks also have lots of good ones this year, including this crazy encrypted boot loader one given by our very own Allan Jude! *** OPENBSD ON AWS : AN UNEXPECTED JOURNEY (http://blog.d2-si.fr/2016/02/15/openbsd-on-aws/?hn) We have a blog post from Antoine Jacoutot, talking about the process of getting OpenBSD up and running in AWS It starts with his process of creating an AMI from scratch, which ended up not being that bad: create and loopback-mount a raw image containing a UFS filesystem extract the OpenBSD base sets (which are just regular tarballs) and kernel enable console output (so that one could “aws ec2 get-console-output”) install the boot loader on the image then use the ec2 tools to import the RAW image to S3, convert it into a volume (ec2-import-volume) which we can snapshot (ec2-create-snapshot) and create an AMI from (ec2-register) The blog post also has a link to a script which automates this process, so don't be daunted if you didn't quite follow all of that. Thanks to the recently landed DomU support, the final pieces of the puzzle fell into place, allowing OpenBSD to function as a proper guest (with networking!) Next it details the process of injecting a public SSH key into the instances for instant remote access. An ec2-init.sh script was created (also on github) which does the following: setting the hostname installing the provided SSH public key to /root/.ssh/authorized_keys executing user-data (if it starts with a shebang) displaying the host SSH fingerprints on the console (to match cloud-init) With that done, OpenBSD is pretty much AWS ready! He then gives a brief walkthrough of setting up nginx for new users, but if you've already done this before then the instance is ready for you to hacking on. Start thinking of ideas for things with FreeBSD for Google's 2016 Summer of Code (https://wiki.freebsd.org/SummerOfCodeIdeas) Students and Developers, listen up! It's time to start thinking about GSoC again, and FreeBSD is looking to update its project ideas page. There's some good ones on the list, plus ones that should be pruned (such as GELI boot), but now is the time to start adding new ones before we get too deep into the process. This goes for the other BSD's as well, start thinking about your proposals, or if you are developer, which projects would be a good fit for mentoring. (Improving the Linux Compat layer is one I think should be done!) Guide to getting started with kernel hacking (https://wiki.freebsd.org/Graphics/Getting%20started%20with%20kernel%20projects) One of the things that's been asked frequently is how to contribute towards the efforts to bring updated DRM / X drivers to the FreeBSD kernel. Jean-Sébastien Pédron has started a great guide on the Wiki which details how to get started with the porting effort, and that developers need not be afraid of helping. *** Storage Summit Roundup Earlier this week a number of developers from FreeBSD, as well as various vendors that use FreeBSD, or provide products used with FreeBSD met for a Storage Summit (https://wiki.freebsd.org/201602StorageSummit), to discuss the future of these technologies The summit was co-located with the USENIX FAST (Filesystems And Storage Technologies) conference The summit was sponsored by the FreeBSD Foundation and FlightAware After a short introduction, the event opened with a Networking Synergy panel The focus of this panel was to see if there were techniques and lessons learned in improving the networking stack over the last 10 years that could be applied to improving the storage stack A lot of time was spent discussing issues like multi-queue support, CPU scheduling, and ways to modernize the stack CAM Scheduling & Locking Revamp (https://wiki.freebsd.org/201602StorageSummit/CAM) No notes posted User Space Storage Stack (https://wiki.freebsd.org/201602StorageSummit/UserSpace) One of the user space storage stacks discussed was Diskmap Like netmap, but for disks (diskmap) Kernel bypass for accessing disks Ilias Marinos, who is working on diskmap at Cambridge University, described diskmap to the group A design discussion then followed in which the memory management was covered as that's an issue for any sort of "IO" map system Action Items: Discuss with Luigi the idea of code merges Need a reset path API Kernel buffer mapping for reliability Support for other interfaces (SATA/SCSI) GEOM layer adaptation Adapting to New Storage Technologies (https://wiki.freebsd.org/201602StorageSummit/NewStorageTechnologies) This working group was led by Adrian Palmer, from Seagate SMR Persistent Memory Session 1: Device Identification and the structural requirements Agenda: We'll look over the Identification nuances and what needs to change to support the structure. Support for IO order guarantees, forward-write only requirements, new commands and topology. Dig into CAM and GEOM layers. Solutions should be fast and have as few code paths as possible Results: Small audience. We talked about zoned characteristics, and how it can be used in various workloads, projected to be implemented in years Session 2: Information dissemination and consumption Agenda: Where and how will information from the report_zones command be gathered, stored, combined and used. This will include userspace storage and multi-volume management. Will CAM store this data, or will GEOM? How frequently will this need to be queried/updated/verified from the drive? Results: Merged with ZFS working group to discuss SMR. Came up with idea that could be implemented as circular buffer zone type. Began to discuss solutions among developers ZFS (https://wiki.freebsd.org/201602StorageSummit/ZFS) During the first session we discussed how to improve dedup support + A dedup throttle or cap was discussed. When the size of the DDT grows beyond this size, new entries would not be deduped. An alternative to this was also discussed, where when the DDT reached the cap size, it would remove a random entry with only a single reference from the DDT to make room for the new entry. When a block is going to be freed, if it is not found in the DDT, it is assumed to have only 1 reference, and removed. There was also discussion of replacing the DDT with an in-memory hash table and a “log” of increment/decrement operations, that is periodically compacted. The hash table is recreated from the log at pool import time. This would reduce the in-memory footprint of the DDT, as well as speed up all write operations as adding an entry to the dedup log will be less expensive than updating the DDT. There was also discussion of using dedicated device(s) for the DDT, either using the DDT on SSD work by Nexenta, or the Metadata Classes work by Intel The first session also discussed Secure Delete and related things The desire for an implementation of TRIM that uses the “secure erase” functionality provided by some disks was expressed Overwriting sectors with patterns of garbage may be insufficient because SSDs may internally remap where a specific LBA physically resides The possibility of using something like the “eager zero” feature to periodically write zeros over all free blocks in the pool to erase any lingering data fragments Problems with the FreeBSD TRIM implementation were discussed, as well as looking at ways to implement the new ZFS TRIM implementation on FreeBSD ABD (ARC Buf Data) was discussed, a new design that lessens the requirement for contiguous memory. Only a small area of contiguous blocks is reserved at boot, and compressed ARC blocks are constructed of scatter-gather lists of individual pages The second session combined with the SMR group and talked about SMR support in ZFS Later in the second session ZFS Encryption was also discussed, mostly with a focus on what the use cases are The third session combined all of the groups for an overview of upcoming ZFS features including device removal and channel programs There was also a request for code review, for mostly finished projects like Persistent L2ARC, Writeback cache, and Large dnode support Hallway Track ZFS / VFS Interaction Adrian Palmer has been a FreeBSD hobbyist since FreeBSD 7, and I think I managed to convince him to start contributing *** News Roundup One Week with NetBSD 7.0: Back to Unix basics (http://jamesdeagle.blogspot.com/2016/02/one-week-with-netbsd-70-back-to-unix.html) The author of this blog series is sending a week using NetBSD 7.0, following a previous series on Solaris 10 “This is actually familiar territory, as I've been using BSD variants almost exclusively since 2006. My recent SunOS explorations were triggered last summer by OpenBSD having choked on my current laptop's NVIDIA card, and from what I could see at the time, FreeBSD had the same problem, although I now know NVIDIA drivers exist for that system. The thing that keeps me from going all-in with FreeBSD 10.x, however, is the fact that Firefox crashes and leaves "core dump" messages in its wake, and I'm just not a Chrome kinda guy.” “For those with a catholic taste in Unix, NetBSD is a keg party at the Vatican. If you're an absolute Unix beginner, or have been living on Ubuntu-based Linux distros for too long, then you may feel stranded at first by NetBSD's sparseness. You'll find yourself staring into the abyss and seeing only a blinking cursor staring back. If you have the presence of mind to type startx, you'll be greeted by twm, a window manager offering little more than an xterm window with the same blinking cursor until you learn how to configure the .twmrc file to include whatever applications you want or need in the right-click menu.” “As for NetBSD itself, I can't think of any major productivity applications that can't be installed, and most multimedia stuff works fine.” Issues the author hopes to sort out in later posts: Audio playback (youtube videos in Firefox) Wireless Flash Digital Camera SD Card readability, video playback Audacity A “fancy” desktop like Gnome 2, KDE, or xfce In a follow-up post (http://jamesdeagle.blogspot.com/2016/02/one-week-with-netbsd-70-libreoffice.html), the author got LibreOffice installed and sorted out the audio issues they were having In a later follow-up (http://jamesdeagle.blogspot.com/2016/02/one-week-with-netbsd-70-mixed-review-of.html) XFCE is up and running as well *** ZFS is for Containers in Ubuntu 16.04 (http://blog.dustinkirkland.com/2016/02/zfs-is-fs-for-containers-in-ubuntu-1604.html) As you may have heard, Ubuntu 16.04 will include ZFS -- baked directly into Ubuntu -- supported by Canonical “ZFS one of the most beloved features of Solaris, universally coveted by every Linux sysadmin with a Solaris background. To our delight, we're happy to make to OpenZFS available on every Ubuntu system.” What does “supported by Canonical” mean? “You'll find zfs.ko automatically built and installed on your Ubuntu systems. No more DKMS-built modules” “The user space zfsutils-linux package will be included in Ubuntu Main, with security updates provided by Canonical” The article then provides a quick tutorial for setting up Linux Containers (LXC) backed by ZFS In the example, ZFS is backed by a file on the existing disk, not by a real disk, and with no redundancy However, the setup script seems to support using real block devices The Software Freedom Conservancy (https://sfconservancy.org/) is expected to issue a statement detailing their opinion on the legalities and licensing issues of bundling ZFS with Linux. *** Polling is a Hack: Server Sent Events (EventSource) with gevent, Flask, nginx, and FreeBSD (http://hypatia.software/2016/01/29/polling-is-a-hack-server-sent-events-eventsource-with-gevent-flask-nginx-and-freebsd/) A tutorial on setting up ‘Server-Sent Events', also know as EventSource in javascript, to notify website clients of new data, rather than having the javascript constantly poll for new data. The setup uses FreeBSD, nginx, gevent, Python, and the Flask framework The tutorial walks through setting a basic Python application using the Flask framework Then setting up the client side in Javascript Then for the server side setup, it covers installing and configuring nginx, and py-supervisor on FreeBSD The tutorial also includes links to additional resources and examples, including how to rate limit the Flash application *** Why FreeBSD? (http://www.aikchar.me/blog/why-freebsd.html) An excellent article written by Hamza Sheikh, discussing why FreeBSD is now his clear choice for learning UNIX. The article is pretty well written and lengthy, but has some great parts which we wanted to share with you: There were many rough edges in the Linux world and some of them exist even today. Choosing the right distribution (distro) for the task at hand is always the first and most difficult decision to make. While this is a strength of the Linux community it is also its weakness. This is exacerbated with the toxic infighting within the community in the last few years. A herd of voices believes it is their right to bring down a distro community because it is not like their distro of choice. Forking upstream projects has somehow become taboo. Hurling abuse in mailing lists is acceptable. Helping new users is limited to lambasting their distro of choice. Creating conspiracy theories over software decisions is the way to go. Copyleft zealots roam social media declaring non-copyleft free software heretic abominations. It all boils down to an ecosystem soured by the presence of maniacs who have the loudest voices and they seem to be everywhere you turn. Where is the engineering among all this noise? Btrfs - baking for a long time - is still nowhere near ZFS in stability or feature parity. systemd is an insatiable entity that feeds on every idea in sight and just devours indiscriminately. Wayland was promised years ago and its time has yet to arrive. Containers are represented by Docker that neither securely contains applications nor makes them easy to manage in production. Firewalling is dithering between firewalld, nftables, etc. SystemTap cannot match DTrace. In the same time span what do various BSDs offer? pf, CARP, ZFS, Hammer, OpenSSH, jails, pkgsrc, (software) ports, DTrace, hardware portability; just to name a few. Few would deny that BSDs have delivered great engineering with free software licenses to the entire world. To me they appear to be better flag bearers of free software with engineering to back it. He then goes through some of the various BSD's and the specifics on why FreeBSD was the logical choice for his situation. But at the end has a great summary on the community as a whole: Finally - and maybe repeating myself here - I have nothing but praise for the community. Be it BSD Now, mailing lists, Reddit, Twitter, LFNW, or SeaGL, people have encouraged me, answered my questions, and filed bugs for me. I have been welcomed and made a part of the community with open arms. These reasons are (good) enough for me to use FreeBSD and contribute to it. BeastieBits OPNsense 16.1.3 released (https://opnsense.org/opnsense-16-1-3-released/) Copies of "FreeBSD Mastery: Specialty Filesystems" seen in the wild (https://twitter.com/Savagedlight/status/700001944547491842) pfsense training available in Europe (http://www.netgate.com/training/) LiteBSD now has 50 ports in its ports tree (https://github.com/ibara/LiteBSD-Ports) Ports tree locked for OpenBSD 5.9 (http://marc.info/?l=openbsd-ports&m=145615281431064&w=2) “FreeBSD Filesystem Fun” at March semibug (http://blather.michaelwlucas.com/archives/2556) Event #46 — Embedded Platforms (BSD, OpenWRT, Plan 9 & Inferno) (http://oshug.org/event/46) Feedback/Questions Frank - ZFS RAM? (http://slexy.org/view/s21lcCKrSB) David - ARM Porting (http://slexy.org/view/s204lxjvlq) Johnny - Lumina Default? (http://slexy.org/view/s2xMiSNLYn) Adam - PC-BSD Install and Q's (http://slexy.org/view/s214gJbLwD) Jeremy - Video Card Q (http://slexy.org/view/s20UNyzEeh) ***

Coming up this time on the show, we'll be chatting with Lee Sharp. He's recently revived the m0n0wall codebase, now known as SmallWall, and we'll find out what the future holds for this new addition to the BSD family. Answers to your emails and all this week's news, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines BSDCan and pkgsrcCon videos (https://www.youtube.com/channel/UCAEx6zhR2sD2pAGKezasAjA/videos) Even more BSDCan 2015 videos are slowly but surely making their way to the internet Nigel Williams, Multipath TCP for FreeBSD (https://www.youtube.com/watch?v=P3vB_FWtyIs) Stephen Bourne, Early days of Unix and design of sh (https://www.youtube.com/watch?v=2kEJoWfobpA) John Criswell, Protecting FreeBSD with Secure Virtual Architecture (https://www.youtube.com/watch?v=hRIC_aF_u24) Shany Michaely, Expanding RDMA capability over Ethernet in FreeBSD (https://www.youtube.com/watch?v=stsaeKvF3no) John-Mark Gurney, Adding AES-ICM and AES-GCM to OpenCrypto (https://www.youtube.com/watch?v=JaufZ7yCrLU) Sevan Janiyan, Adventures in building (https://www.youtube.com/watch?v=-HMXyzybgdM) open source software (https://www.youtube.com/watch?v=Xof-uKnQ6cY) And finally, the BSDCan 2015 closing (https://www.youtube.com/watch?v=Ynm0bGnYdfY) Some videos (https://vimeo.com/channels/pkgsrccon/videos) from this year's pkgsrcCon (http://pkgsrc.org/pkgsrcCon/2015/) are also starting to appear online Sevan Janiyan, A year of pkgsrc 2014 - 2015 (https://vimeo.com/channels/pkgsrccon/132767946) Pierre Pronchery, pkgsrc meets pkg-ng (https://vimeo.com/channels/pkgsrccon/132766052) Jonathan Perkin, pkgsrc at Joyent (https://vimeo.com/channels/pkgsrccon/132760863) Jörg Sonnenberger, pkg_install script framework (https://vimeo.com/channels/pkgsrccon/132757658) Benny Siegert, New Features in BulkTracker (https://vimeo.com/channels/pkgsrccon/132751897) This is the first time we've ever seen recordings from the conference - hopefully they continue this trend *** OPNsense 15.7 released (https://forum.opnsense.org/index.php?topic=839.0) The OPNsense team has released version 15.7, almost exactly six months after their initial debut (http://www.bsdnow.tv/episodes/2015_01_14-common_sense_approach) In addition to pulling in the latest security fixes from upstream FreeBSD, 15.7 also includes new integration of an intrusion detection system (and new GUI for it) as well as new blacklisting options for the proxy server Taking a note from upstream PF's playbook, ALTQ traffic shaping support has finally been retired as of this release (it was deprecated from OpenBSD a few years ago, and the code was completely removed (http://undeadly.org/cgi?action=article&sid=20140419151959) just over a year ago) The LibreSSL flavor has been promoted to production-ready, and users can easily migrate over from OpenSSL via the GUI - switching between the two is simple; no commitment needed Various third party ports have also been bumped up to their latest versions to keep things fresh, and there's the usual round of bug fixes included Shortly afterwards, 15.7.1 (https://forum.opnsense.org/index.php?topic=915.0) was released with a few more small fixes *** NetBSD at Open Source Conference 2015 Okinawa (https://mail-index.netbsd.org/netbsd-advocacy/2015/07/04/msg000688.html) If you liked last week's episode (http://www.bsdnow.tv/episodes/2015_07_01-lost_technology) then you'll probably know what to expect with this one The NetBSD users group of Japan hit another open source conference, this time in Okinawa This time, they had a few interesting NetBSD machines on display that we didn't get to see in the interview last week We'd love to see something like this in North America or Europe too - anyone up for installing BSD on some interesting devices and showing them off at a Linux con? *** OpenBSD BGP and VRFs (http://firstyear.id.au/entry/21) "VRFs (https://en.wikipedia.org/wiki/Virtual_routing_and_forwarding), or in OpenBSD rdomains, are a simple, yet powerful (and sometimes confusing) topic" This article aims to explain both BGP and rdomains, using network diagrams, for some network isolation goodness With multiple rdomains, it's also possible to have two upstream internet connections, but lock different groups of your internal network to just one of them The idea of a "guest network" can greatly benefit from this separation as well, even allowing for the same IP ranges to be used without issues Combining rdomains with the BGP protocol allows for some very selective and precise blocking/passing of traffic between networks, which is also covered in detail here The BSDCan talk on rdomains (https://www.youtube.com/watch?v=BizrC8Zr-YY) expands on the subject a bit more if you haven't seen it, as well as a few related (https://www.packetmischief.ca/2011/09/20/virtualizing-the-openbsd-routing-table/) posts (http://cybermashup.com/2013/05/21/complex-routing-with-openbsd/) *** Interview - Lee Sharp - lee@smallwall.org (mailto:lee@smallwall.org) SmallWall (http://smallwall.org), a continuation of m0n0wall News Roundup Solaris adopts more BSD goodies (https://blogs.oracle.com/solarisfw/entry/pf_for_solaris) We mentioned a while back that Oracle developers have begun porting a current version of OpenBSD's PF firewall to their next version, even contributing back patches for SMP and other bug fixes They recently published an article about PF, talking about what's different about it on their platform compared to others - not especially useful for BSD users, but interesting to read if you like firewalls Darren Moffat, who was part of originally getting an SSH implementation into Solaris, has a second blog post (https://blogs.oracle.com/darren/entry/openssh_in_solaris_11_3) up about their "SunSSH" fork Going forward, their next version is going to offer a completely vanilla OpenSSH option as well, with the plan being to phase out SunSSH after that The article talks a bit about the history of getting SSH into the OS, forking the code and also lists some of the differences between the two In a third blog post (https://blogs.oracle.com/darren/entry/solaris_new_system_calls_getentropy), they talk about a new system call they're borrowing from OpenBSD, getentropy(2) (http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/getentropy.2), as well as the addition of arc4random (http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man3/arc4random.3) to their libc With an up-to-date and SMP-capable PF, ZFS with native encryption, jail-like Zones, unaltered OpenSSH and secure entropy calls… is Solaris becoming better than us? Look forward to the upcoming "Solaris Now" podcast (not really) *** EuroBSDCon 2015 talks and tutorials (https://2015.eurobsdcon.org/talks/) This year's EuroBSDCon is set to be held in Sweden at the beginning of October, and the preliminary list of accepted presentations has been published The list looks pretty well-balanced between the different BSDs, something Paul would be happy to see if he was still with us It even includes an interesting DragonFly talk and a couple talks from NetBSD developers, in addition to plenty of FreeBSD and OpenBSD of course There are also a few tutorials (https://2015.eurobsdcon.org/tutorials/) planned for the event, some you've probably seen already and some you haven't Registration for the event will be opening very soon (likely this week or next) *** Using ZFS replication to improve offsite backups (https://www.iceflatline.com/2015/07/using-zfs-replication-features-in-freebsd-to-improve-my-offsite-backups/) If you take backups seriously, you're probably using ZFS and probably keeping an offsite copy of the data This article covers doing just that, but with a focus on making use of the replication capability It'll walk you through taking a snapshot of your pool and then replicating it to another remote system, using "zfs send" and SSH - this has the benefit of only transferring the files that have changed since the last time you did it Steps are also taken to allow a regular user to take and manage snapshots, so you don't need to be root for the SSH transfer Data integrity is a long process - filesystem-level checksums, resistance to hardware failure, ECC memory, multiple copies in different locations... they all play a role in keeping your files secure; don't skip out on any of them One thing the author didn't mention in his post: having an offline copy of the data, ideally sealed in a safe place, is also important *** Block encryption in OpenBSD (http://anadoxin.org/blog/blog/20150705/block-encryption-in-openbsd/) We've covered (http://www.bsdnow.tv/tutorials/fde) ways to do fully-encrypted installations of OpenBSD (and FreeBSD) before, but that requires dedicating a whole drive or partition to the sensitive data This blog post takes you through the process of creating encrypted containers in OpenBSD, à la TrueCrypt - that is, a file-backed virtual device with an encrypted filesystem It goes through creating a file that looks like random data, pointing vnconfig at it, setting up the crypto and finally using it as a fake storage device The encrypted container method offers the advantage of being a bit more portable across installations than other ways *** Docker hits FreeBSD ports (https://svnweb.freebsd.org/ports?view=revision&revision=391421) The inevitable has happened, and an early FreeBSD port of docker is finally here Some details and directions (https://github.com/kvasdopil/docker/blob/freebsd-compat/FREEBSD-PORTING.md) are available to read if you'd like to give it a try, as well as a list of which features work and which don't There was also some Hacker News discussion (https://news.ycombinator.com/item?id=9840025) on the topic *** Microsoft donates to OpenSSH (http://undeadly.org/cgi?action=article&sid=20150708134520&mode=flat) We've talked about big businesses using BSD and contributing back before, even mentioning a few other large public donations - now it's Microsoft's turn With their recent decision to integrate OpenSSH into an upcoming Windows release, Microsoft has donated a large sum of money to the OpenBSD foundation, making them a gold-level sponsor They've also posted some contract work offers on the OpenSSH mailing list, and say that their changes will be upstreamed if appropriate - we're always glad to see this *** Feedback/Questions Joe writes in (http://slexy.org/view/s2NqbhwOoH) Mike writes in (http://slexy.org/view/s2T3NEia98) Randy writes in (http://slexy.org/view/s20RlTK6Ha) Tony writes in (http://slexy.org/view/s2rjCd0bGX) Kevin writes in (http://slexy.org/view/s21PfSIyG5) ***

Coming up this week, we'll be talking with Jun Ebihara about some lesser-known CPU architectures in NetBSD. He'll tell us what makes these old (and often forgotten) machines so interesting. As usual, we've also got answers to your emails and all this week's news on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Out with the old, in with the less (http://www.tedunangst.com/flak/post/out-with-the-old-in-with-the-less) Our friend Ted Unangst has a new article up, talking about "various OpenBSD replacements and reductions" "Instead of trying to fix known bugs, we're trying to fix unknown bugs. It's not based on the current buggy state of the code, but the anticipated future buggy state of the code. Past bugs are a bigger factor than current bugs." In the post, he goes through some of the bigger (and smaller) examples of OpenBSD rewriting tools to be simpler and more secure It starts off with a lesser-known SCSI driver that "tried to do too much" being replaced with three separate drivers "Each driver can now be modified in isolation without unintentional side effects on other hardware, or the need to consider if and where further special cases need to be added. Despite the fact that these three drivers duplicate all the common boilerplate code, combined they only amount to about half as much code as the old driver." In contrast to that example, he goes on to cite mandoc as taking a very non "unixy" direction, but at the same time being smaller and simpler than all the tools it replaced The next case is the new http daemon, and he talks a bit about the recently-added rewrite support being done in a simple and secure way (as opposed to regex and its craziness) He also talks about the rewritten "file" utility: "Almost by definition, its sole input will be untrusted input. Perversely, people will then trust what file tells them and then go about using that input, as if file somehow sanitized it." Finally, sudo in OpenBSD's base system is moving to ports soon, and the article briefly describes a new tool that may or may not replace it (https://marc.info/?l=openbsd-ports&m=143481227122523&w=2), called "doas" There's also a nice wrap-up of all the examples at the end, and the "Pruning and Polishing (http://www.openbsd.org/papers/pruning.html)" talk is good complementary reading material *** More OpenZFS and BSDCan videos (https://www.youtube.com/channel/UC0IK6Y4Go2KtRueHDiQcxow/videos) We mentioned last week (http://www.bsdnow.tv/episodes/2015_06_24-bitrot_group_therapy) that some of the videos from the second OpenZFS conference in Europe were being uploaded - here's some more Matt Ahrens did a Q&A session (https://www.youtube.com/watch?v=I6fXZ_6OT5c) and talked about ZFS send and receive (https://www.youtube.com/watch?v=iY44jPMvxog), as well as giving an overview of OpenZFS (https://www.youtube.com/watch?v=RQlMDmnty80) George Wilson talked about a performance retrospective (https://www.youtube.com/watch?v=KBI6rRGUv4E) Toshiba (https://www.youtube.com/watch?v=sSi47-k78IM), Syneto (https://www.youtube.com/watch?v=Hhje5KEF5cE) and HGST (https://www.youtube.com/watch?v=aKgxXipss8k) also gave some talks about their companies and how they're using ZFS As for BSDCan, more of their BSD presentations have been uploaded too... Ryan Stone, PCI SR-IOV on FreeBSD (https://www.youtube.com/watch?v=INeMd-i5jzM) George Neville-Neil, Measure Twice, Code Once (https://www.youtube.com/watch?v=LE4wMsP7zeA) Kris Moore, Unifying jail and package management for PC-BSD, FreeNAS and FreeBSD (https://www.youtube.com/watch?v=qNYXqpJiFN0) Warner Losh, I/O Scheduling in CAM (https://www.youtube.com/watch?v=3WqOLolj5EU) Kirk McKusick, An Introduction to the Implementation of ZFS (https://www.youtube.com/watch?v=l-RCLgLxuSc) Midori Kato, Extensions to FreeBSD Datacenter TCP for Incremental Deployment Support (https://www.youtube.com/watch?v=zZXvjhWcg_4) Baptiste Daroussin, Packaging FreeBSD's (https://www.youtube.com/watch?v=Br6izhH5P1I) base system (https://www.youtube.com/watch?v=v7px6ktoDAI) Matt Ahrens, New OpenZFS features supporting remote replication (https://www.youtube.com/watch?v=UOX7WDAjqso) Ed Schouten, CloudABI Cloud computing meets fine-grained capabilities (https://www.youtube.com/watch?v=SVdF84x1EdA) The audio of Ingo Schwarze's talk "mandoc: becoming the main BSD manual toolbox" got messed up, but there's an alternate recording here (http://www.bsdcan.org/2015/audio/mandoc.mp3), and the slides are here (http://www.openbsd.org/papers/bsdcan15-mandoc.pdf) *** SMP steroids for PF (https://www.marc.info/?l=openbsd-tech&m=143526329006942&w=2) An Oracle employee that's been porting OpenBSD's PF to an upcoming Solaris release has sent in an interesting patch for review Attached to the mail was what may be the beginnings of making native PF SMP-aware Before you start partying, the road to SMP (specifically, giant lock removal) is a long and very complicated one, requiring every relevant bit of the stack to be written with it in mind - this is just one piece of the puzzle The initial response (https://www.marc.info/?l=openbsd-tech&m=143532243322281&w=2) has been quite positive though, with some back and forth (https://www.marc.info/?l=openbsd-tech&m=143532963824548&w=2) between developers and the submitter For now, let's be patient and see what happens *** DragonFly 4.2.0 released (http://www.dragonflybsd.org/release42/) DragonFlyBSD has released the next big update of their 4.x branch, complete with a decent amount of new features and fixes i915 and Radeon graphics have been updated, and DragonFly can claim the title of first BSD with Broadwell support in a release Sendmail in the base system has been replaced with their homegrown DragonFly Mail Agent, and there's a wiki page (http://www.dragonflybsd.com/docs/docs/newhandbook/mta/) about configuring it They've also switched the default compiler to GCC 5, though why they've gone in that direction instead of embracing Clang is a mystery The announcement page also contains a list of kernel changes, details on the audio and graphics updates, removal of the SCTP protocol, improvements to the temperature sensors, various userland utility fixes and a list of updates to third party tools Work is continuing on the second generation HAMMER filesystem, and Matt Dillon provides a status update in the release announcement There was also some hacker news discussion (https://news.ycombinator.com/item?id=9797932) you can check out, as well as upgrade instructions (http://lists.dragonflybsd.org/pipermail/users/2015-June/207801.html) *** OpenSMTPD 5.7.1 released (https://opensmtpd.org/announces/release-5.7.1.txt) The OpenSMTPD guys have just released version 5.7.1, a major milestone version that we mentioned recently Crypto-related bits have been vastly improved: the RSA engine is now privilege-separated, TLS errors are handled more gracefully, ciphers and curve preferences can now be specified, the PKI interface has been reworked to allow custom CAs, SNI and certificate verification have been simplified and the DH parameters are now 2048 bit by default The long-awaited filter API is now enabled by default, though still considered slightly experimental Documentation has been improved quite a bit, with more examples and common use cases (as well as exotic ones) Many more small additions and bugfixes were made, so check the changelog for the full list Starting with 5.7.1, releases are now cryptographically (https://twitter.com/OpenSMTPD/status/613257722574839808) signed (https://www.opensmtpd.org/archives/opensmtpd-5.7.1.sum.sig) to ensure integrity This release has gone through some major stress testing to ensure stability - Gilles regularly asks their Twitter followers to flood a test server (https://twitter.com/OpenSMTPD/status/608399272447471616) with thousands of emails per second, even offering prizes (https://twitter.com/OpenSMTPD/status/608235180839567360) to whoever can DDoS them the hardest OpenSMTPD runs on all the BSDs of course, and seems to be getting pretty popular lately Let's all encourage (mailto:feedback@bsdnow.tv) Kris to stop procrastinating on switching from Postfix *** Interview - Jun Ebihara (蛯原純) - jun@netbsd.org (mailto:jun@netbsd.org) / @ebijun (https://twitter.com/ebijun) Lesser-known CPU architectures, embedded NetBSD devices News Roundup FreeBSD foundation at BSDCan (http://freebsdfoundation.blogspot.com/2015/06/bsdcan-2015-trip-report-steven-douglas.html) The FreeBSD foundation has posted a few BSDCan summaries on their blog The first, from Steven Douglas, begins with a sentiment a lot of us can probably identify with: "Where I live, there are only a handful of people that even know what BSD is, let alone can talk at a high level about it. That was one of my favorite things, being around like minded people." He got to meet a lot of the people working on big-name projects, and enjoyed being able to ask them questions so easily Their second (http://freebsdfoundation.blogspot.com/2015/06/bsdcan-2015-trip-report-ahmed-kamal.html) trip report is from Ahmed Kamal, who flew in all the way from Egypt A bit starstruck, he seems to have enjoyed all the talks, particularly Andrew Tanenbaum's about MINIX and NetBSD There are also two more wrap-ups from Zbigniew Bodek (http://freebsdfoundation.blogspot.com/2015/06/bsdcan-2015-trip-report-zbigniew-bodek.html) and Vsevolod Stakhov (http://freebsdfoundation.blogspot.com/2015/06/bsdcan-2015-trip-report-vsevolod-stakhov.html), so you've got plenty to read *** OpenBSD from a veteran Linux user perspective (http://cfenollosa.com/blog/openbsd-from-a-veteran-linux-user-perspective.html) In a new series of blog posts, a self-proclaimed veteran Linux user is giving OpenBSD a try for the first time "For the first time I installed a BSD box on a machine I control. The experience has been eye-opening, especially since I consider myself an 'old-school' Linux admin, and I've felt out of place with the latest changes on the system administration." The post is a collection of his thoughts about what's different between Linux and BSD, what surprised him as a beginner - admittedly, a lot of his knowledge carried over, and there were just minor differences in command flags One of the things that surprised him (in a positive way) was the documentation: "OpenBSD's man pages are so nice that RTFMing somebody on the internet is not condescending but selfless." He also goes through some of the basics, installing and updating software, following different branches It concludes with "If you like UNIX, it will open your eyes to the fact that there is more than one way to do things, and that system administration can still be simple while modern." *** FreeBSD on the desktop, am I crazy (http://sysconfig.org.uk/freebsd-on-the-desktop-am-i-crazy.html) Similar to the previous article, the guy that wrote the SSH two factor authentication post we covered last week has another new article up - this time about FreeBSD on the desktop He begins with a bit of forewarning for potential Linux switchers: "It certainly wasn't an easy journey, and I'm tempted to say do not try this at home to anybody who isn't going to leverage any of FreeBSD's strong points. Definitely don't try FreeBSD on the desktop if you haven't used it on servers or virtual machines before. It's got less in common with Linux than you might think." With that out of the way, the list of positives is pretty large: a tidy base system, separation between base and ports, having the option to choose binary packages or ports, ZFS, jails, licensing and of course the lack of systemd The rest of the post talks about some of the hurdles he had to overcome, namely with graphics and the infamous Adobe Flash Also worth noting is that he found jails to be not only good for isolating daemons on a server, but pretty useful for desktop applications as well In the end, he says it was worth all the trouble, and is even planning on converting his laptop to FreeBSD soon too *** OpenIKED and Cisco CSR 1000v IPSEC (https://www.netflask.net/ipsec-ikev2-cisco-csr1000v-openiked/) This article covers setting up a site-to-site IPSEC tunnel between a Cisco CSR 1000v router and an OpenBSD gateway running OpenIKED What kind of networking blog post would be complete without a diagram where the internet is represented by a big cloud There are lots of details (and example configuration files) for using IKEv2 and OpenBSD's built-in IKE daemon It also goes to show that the BSDs generally play well with existing network infrastructure, so if you were a business that's afraid to try them… don't be *** HardenedBSD improves stack randomization (https://github.com/HardenedBSD/hardenedBSD/commit/bd5cecb4dc7947a5e214fc100834399b4bffdee8) The HardenedBSD guys have improved their FreeBSD ASLR patchset, specifically in the stack randomization area In their initial implementation, the stack randomization was a random gap - this update makes the base address randomized as well They're now stacking the new on top of the old as well, with the goal being even more entropy This change triggered an ABI and API incompatibility, so their major version has been bumped *** OpenSSH 6.9 released (https://lists.mindrot.org/pipermail/openssh-unix-announce/2015-July/000121.html) The OpenSSH team has announced the release of a new version which, following their tick/tock major/minor release cycle, is focused mainly on bug fixes There are a couple new things though - the "AuthorizedKeysCommand" config option now takes custom arguments One very notable change is that the default cipher has changed as of this release The traditional pairing of AES128 in counter mode with MD5 HMAC has been replaced by the ever-trendy ChaCha20-Poly1305 combo Their next release, 7.0, is set to get rid a number of legacy items: PermitRootLogin will be switched to "no" by default, SSHv1 support will be totally disabled, the 1024bit diffie-hellman-group1-sha1 KEX will be disabled, old ssh-dss and v00 certs will be removed, a number of weak ciphers will be disabled by default (including all CBC ones) and RSA keys will be refused if they're under 1024 bits Many small bugs fixes and improvements were also made, so check the announcement for everything else The native version is in OpenBSD -current, and an update to the portable version should be hitting a ports or pkgsrc tree near you soon *** Feedback/Questions Brad writes in (http://slexy.org/view/s2Ws6Y2rZy) Mason writes in (http://slexy.org/view/s21GvZ5xbs) Jochen writes in (http://slexy.org/view/s209TrPK4e) Simon writes in (http://slexy.org/view/s21TQjUjxv) ***

This time on the show, we'll be talking some ZFS with Sean Chittenden. He's been using it on FreeBSD at Groupon, and has some interesting stories about how it's saved his data. Answers to your emails and all of this week's headlines, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines More BSDCan 2015 videos (https://www.bsdcan.org/2015/schedule/) Almost as if we said it would happen last week, more BSD-related presentation videos have been uploaded Alexander Motin, Feature-rich and fast SCSI target with CTL and ZFS (https://www.youtube.com/watch?v=lBE4BfxVDQc) Daichi Goto, FreeBSD for High Density Servers (https://www.youtube.com/watch?v=r2BoQ70bwK4) Ken Moore, Lumina-DE (https://www.youtube.com/watch?v=Qh_YK9y4_Os) Kevin Bowling, FreeBSD Operations at (https://www.youtube.com/watch?v=4l2rlRjkGhk) Limelight Networks (https://www.youtube.com/watch?v=K1-ZyiY5z48) Maciej Pasternacki, Jetpack, a container (https://www.youtube.com/watch?v=8phbsAhJ-9w) runtime for FreeBSD (https://www.youtube.com/watch?v=kJ74mgkzLxc) Ray Percival, Networking with OpenBSD in a virtualized environment (https://www.youtube.com/watch?v=gx5FILdSp2w) Reyk Floeter, Introducing OpenBSD's (https://www.youtube.com/watch?v=DV1-EfdIp8I) new httpd (https://www.youtube.com/watch?v=_v0lI6qDWFs) Still more to come, hopefully *** OpenBSD httpd rewrite support (https://www.marc.info/?l=openbsd-tech&m=143480475721221&w=2) One of the most-requested features of OpenBSD's new HTTP daemon (in fact, you can hear someone asking about it in the video just above) is rewrite support There were concerns about regex code being too complicated and potentially allowing another attack surface, so that was out Instead, Reyk ported over an implementation of lua pattern matching while on the flight back from BSDCan, turning it into a C API without the lua bindings In the mailing list post, he shows an example of how to use it for redirects and provides the diff (https://www.marc.info/?l=openbsd-tech&m=143489473103114&w=2) if you'd like to give it a try now It's since been committed (https://www.marc.info/?l=openbsd-cvs&m=143507301715409&w=2) to -current, so you can try it out with a snapshot too *** SSH 2FA on FreeBSD (http://sysconfig.org.uk/two-factor-authentication-with-ssh.html) We've discussed different ways to lock down SSH access to your BSD boxes before - use keys instead of passwords, whitelist IPs, or even use two-factor authentication This article serves as a sort of "roundup" on different methods to set up two-factor authentication on FreeBSD It touches on key pairs with a server-side password, google authenticator and a few other variations While the article is focused on FreeBSD, a lot of it can be easily applied to the others too OpenSSH has a great security record, but two-factor authentication is always a good thing to have for the most important systems *** NetBSD 7.0-RC1 released (https://blog.netbsd.org/tnf/entry/netbsd_7_0_rc1_binaries) NetBSD has just announced the first release candidate for the 7.0 branch, after a long delay since the initial beta (11 months ago (http://www.bsdnow.tv/episodes/2014_07_23-des_challenge_iv)) Some of the standout features include: improved KMS/DRM with support for modern GPUs, SMP support on ARM, lots of new ARM boards officially supported, GPT support in the installer, Lua kernel scripting, a multiprocessor USB stack, improvements to NPF (their firewall) and, optionally, Clang 3.6.1 They're looking for as much testing as possible, so give it a try and report your findings to the release engineering team *** Interview - Sean Chittenden - seanc@freebsd.org (mailto:seanc@freebsd.org) / @seanchittenden (https://twitter.com/seanchittenden) FreeBSD at Groupon, ZFS News Roundup OpenSMTPD and Dovecot (http://www.tumfatig.net/20150620/opensmtpd-and-dovecot-on-openbsd-5-7/) We've covered a number of OpenSMTPD mail server guides on the show, each with just a little something different to offer than the last This blog post about it has something not mentioned before: virtual domains and virtual users This means you can easily have "user1@domain.com" and "user2@otherdomain.com" both go to a local user on the box (or a different third address) It also covers SSL certificates, blocking spam and setting up IMAP access, the usual Now might also be a good time to test out OpenSMTPD 5.7.1-rc1 (https://www.mail-archive.com/misc@opensmtpd.org/msg02177.html), which we'll cover in more detail when it's released... *** OctoPkg, a QT frontend to pkgng (https://github.com/aarnt/octopkg) A PC-BSD user has begun porting over a graphical package management utility from Arch linux called Octopi (https://octopiproject.wordpress.com/about/) Obviously, it needed to be rewritten to use FreeBSD's pkg system instead of pacman There are some basic instructions on how to get it built and running on the github page After some testing, it'll likely make its way to the FreeBSD ports tree Tools like this might make it easier for desktop users (who are used to similar things in Ubuntu or related distros) to switch over *** AFL vs. mandoc, a quantitative analysis (http://undeadly.org/cgi?action=article&sid=20150619071929) Ingo Schwarze has written a pretty detailed article about how he and other OpenBSD developers have been fuzzing mandoc with AFL It's meant to be accompanying material to his BSDCan talk, which already covered nine topics mandoc is an interesting example to stress test with fuzzing, since its main job is to take and parse some highly varying input The article breaks down the 45 different bugs that were found, based on their root cause If you're interested in secure coding practices, this'll be a great one to read *** OpenZFS conference videos (https://www.youtube.com/playlist?list=PLaUVvul17xScvtic0SPoks2MlQleyejks) Videos from the second OpenZFS conference have just started to show up The first talk is by, you guessed it, Matt Ahrens In it, he covers some ZFS history, the Oracle takeover, the birth of illumos and OpenZFS, some administration basics and also some upcoming features that are being worked on There are also videos from Nexenta (https://www.youtube.com/watch?v=5ciV4z7WWmo) and HGST (https://www.youtube.com/watch?v=a2lnMxMUxyc), talking about how they use and contribute to OpenZFS *** Feedback/Questions Bryson writes in (http://slexy.org/view/s2FqJfmeK3) Kevin writes in (http://slexy.org/view/s20erRHahQ) ***

This week on the show, we'll be chatting with Marc Espie. He's recently added some additional security measures to dpb, OpenBSD's package building tool, and we'll find out why they're so important. We've also got all this week's news, answers to your emails and even a BSDCan wrap-up, coming up on BSD Now - the place to B.. SD. This episode was brought to you by Headlines BSDCan 2015 videos (https://www.bsdcan.org/2015/schedule/) BSDCan just ended last week, but some of the BSD-related presentation videos are already online Allan Jude, UCL for FreeBSD (https://www.youtube.com/watch?v=8l6bhKIDecg) Andrew Cagney, What happens when a dwarf and a daemon start dancing by the light of the silvery moon? (https://www.youtube.com/watch?v=XDIcD4LR5HE) Andy Tanenbaum, A reimplementation of NetBSD (https://www.youtube.com/watch?v=0pebP891V0c) using a MicroKernel (https://www.youtube.com/watch?v=Bu1JuwVfYTc) Brooks Davis, CheriBSD: A research fork of FreeBSD (https://www.youtube.com/watch?v=DwCg-51vFAs) Giuseppe Lettieri, Even faster VM networking with virtual passthrough (https://www.youtube.com/watch?v=Lo6wDCapo4k) Joseph Mingrone, Molecular Evolution, Genomic Analysis and FreeBSD (https://www.youtube.com/watch?v=K2pnf1YcMTY) Olivier Cochard-Labbe, Large-scale plug&play x86 network appliance deployment over Internet (https://www.youtube.com/watch?v=6jhSvdnu4k0) Peter Hessler, Using routing domains / routing tables in a production network (https://www.youtube.com/watch?v=BizrC8Zr-YY) Ryan Lortie, a stitch in time: jhbuild (https://www.youtube.com/watch?v=YSVFnM3_2Ik) Ted Unangst, signify: Securing OpenBSD From Us To You (https://www.youtube.com/watch?v=9R5s3l-0wh0) Many more still to come... *** Documenting my BSD experience (http://pid1.com/posts/post1.html) Increasingly common scenario: a long-time Linux user (since the mid-90s) decides it's finally time to give BSD a try "That night I came home, I had been trying to find out everything I could about BSD and I watched many videos, read forums, etc. One of the shows I found was BSD Now. I saw that they helped people and answered questions, so I decided to write in." In this ongoing series of blog posts, a user named Michael writes about his initial experiences with trying different BSDs for some different tasks The first post covers ZFS on FreeBSD, used to build a file server for his house (and of course he lists the hardware, if you're into that) You get a glimpse of a brand new user trying things out, learning how great ZFS-based RAID arrays are and even some of the initial hurdles someone could run into He's also looking to venture into the realm of replacing some of his VMs with jails and bhyve soon His second post (http://pid1.com/posts/post2.html) explores replacing the firewall on his self-described "over complicated home network" with an OpenBSD box After going from ipfwadmin to ipchains to iptables, not even making it to nftables, he found the simple PF syntax to be really refreshing All the tools for his networking needs, the majority of which are in the base system, worked quickly and were easy to understand Getting to hear experiences like this are very important - they show areas where all the BSD developers' hard work has paid off, but can also let us know where we need to improve *** PC-BSD tries HardenedBSD builds (https://github.com/pcbsd/hardenedBSD-stable) The PC-BSD team has created a new branch of their git repo with the HardenedBSD ASLR patches integrated They're not the first major FreeBSD-based project to offer an alternate build - OPNsense did that (https://hardenedbsd.org/article/shawn-webb/2015-05-08/hardenedbsd-teams-opnsense) a few weeks ago - but this might open the door for more projects to give it a try as well With Personacrypt, OpenNTPD, LibreSSL and recent Tor integration through the tools, these additional memory protections will offer PC-BSD users even more security that a default FreeBSD install won't have Time will tell if more projects and products like FreeNAS might be interested too *** C-states in OpenBSD (https://www.marc.info/?l=openbsd-cvs&m=143423172522625&w=2) People who run BSD on their notebooks, you'll want to pay attention to this one OpenBSD has recently committed some ACPI improvements for deep C-states (http://www.hardwaresecrets.com/article/Everything-You-Need-to-Know-About-the-CPU-C-States-Power-Saving-Modes/611), enabling the processor to enter a low-power mode According (https://twitter.com/StevenUniq/status/610586711358316545) to a (https://www.marc.info/?l=openbsd-misc&m=143430996602802&w=2) few users (https://www.marc.info/?l=openbsd-misc&m=143429914700826&w=2) so far (https://www.marc.info/?l=openbsd-misc&m=143425943026225&w=2), the change has resulted in dramatically lower CPU temperatures on their laptops, as well as much better battery life If you're running OpenBSD -current on a laptop, try out the latest snapshot and report back (https://www.marc.info/?l=openbsd-misc&m=143423391222952&w=2) with your findings *** NetBSD at Open Source Conference 2015 Hokkaido (https://mail-index.netbsd.org/netbsd-advocacy/2015/06/13/msg000687.html) The Japanese NetBSD users group never sleeps, and they've hit yet another open source conference As is usually the case, lots of strange machines on display were running none other than NetBSD (though it was mostly ARM this time) We'll be having one of these guys on the show next week to discuss some of the lesser-known NetBSD platforms *** Interview - Marc Espie - espie@openbsd.org (mailto:espie@openbsd.org) / @espie_openbsd (https://twitter.com/espie_openbsd) Recent (https://www.marc.info/?l=openbsd-ports&m=143051151521627&w=2) improvements (https://www.marc.info/?l=openbsd-ports&m=143151777209226&w=2) to OpenBSD's dpb (http://www.bsdnow.tv/tutorials/dpb) tool News Roundup Introducing xhyve, bhyve on OS X (https://github.com/mist64/xhyve/blob/master/README.md) We've talked about FreeBSD's "bhyve" hypervisor a lot on the show, and now it's been ported to another OS As the name "xhyve" might imply, it's a port of bhyve to Mac OS X Currently it only has support for virtualizing a few Linux distributions, but more guest systems can be added in the future It runs entirely in userspace, and has no extra requirements beyond OS X 10.10 or newer There are also a few examples (http://www.pagetable.com/?p=831) on how to use it *** 4K displays on DragonFlyBSD (http://www.dragonflybsd.org/docs/newhandbook/docs/newhandbook/4KDisplays/) If you've been using DragonFly as a desktop, maybe with those nice Broadwell graphics, you'll be pleased to know that 4K displays work just fine Matthew Dillon wrote up a wiki page about some of the specifics, including a couple gotchas Some GUI applications might look weird on such a huge resolution, HDMI ports are mostly limited to a 30Hz refresh rate, and there are slightly steeper hardware requirements for a smooth experience *** Sandboxing port daemons on OpenBSD (http://coderinaworldofcode.blogspot.com/2015/06/chrooting-mumble-server-on-openbsd.html) We talked about different containment methods last week, and mentioned that a lot of the daemons in OpenBSD's base as chrooted by default - things from ports or packages don't always get the same treatment This blog post uses a mumble server as an example, but you can apply it to any service from ports that doesn't chroot by default It goes through the process of manually building a sandbox with all the libraries you'll need to run the daemon, and this setup will even wipe and refresh the chroot every time you restart it With a few small changes, similar tricks could be done on the other BSDs as well - everybody has chroots *** SmallWall 1.8.2 released (http://smallwall.freeforums.net/thread/44/version-1-8-2-released) SmallWall is a relatively new BSD-based project that we've never covered before It's an attempt to keep the old m0n0wall codebase going, and appears to have started around the time m0n0wall called it quits They've just released the first official version (http://www.smallwall.org/download.html), so you can give it a try now If you're interested in learning more about SmallWall, the lead developer just might be on the show in a few weeks... *** Feedback/Questions David writes in (http://slexy.org/view/s21gRTNnk7) Brian writes in (http://slexy.org/view/s2DdiMvELg) Dan writes in (http://slexy.org/view/s2h4ZS6SMd) Joel writes in (http://slexy.org/view/s20kA1jeXY) Steve writes in (http://slexy.org/view/s2wJ9HP1bs) ***

We're at BSDCan this week, but fear not! We've got a great interview with Sepherosa Ziehau, a DragonFly developer, about their network stack. After that, we'll be discussing different methods of containment and privilege separation. Assuming no polar bears eat us, we'll be back next week with more BSD Now - the place to B.. SD. This episode was brought to you by Interview - Sepherosa Ziehau - sephe@dragonflybsd.org (mailto:sephe@dragonflybsd.org) Features of DragonFlyBSD's network stack Discussion Comparing containment methods and privilege separation chroot, jails, systrace, capsicum, filesystem permissions, separating users *** Feedback/Questions Brad writes in (http://slexy.org/view/s2GjCsGPef) Anonymous writes in (http://slexy.org/view/s21jj3QgTj) Benjamin writes in (http://slexy.org/view/s2irrhYfPT) Jeroen writes in (http://slexy.org/view/s21gtuqXAe) ***

Coming up this week, we'll be chatting with Lucas Holt, founder of MidnightBSD. It's a slightly lesser-known fork of FreeBSD, with a focus on easy desktop use. We'll find out what's different about it and why it was created. Answers to your emails and all this week's news, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Zocker, it's like docker on FreeBSD (http://toni.yweb.fi/2015/05/zocker-diy-docker-on-freebsd.html) Containment is always a hot topic, and docker has gotten a lot of hype in Linux land in the last couple years - they're working on native FreeBSD support at the moment This blog post is about a docker-like script, mainly for ease-of-use, that uses only jails and ZFS in the base system In total, it's 1,500 lines of shell script (https://github.com/toddnni/zocker) The post goes through the process of using the tool, showing off all the subcommands and explaining the configuration In contrast to something like ezjail, Zocker utilizes the jail.conf system in the 10.x branch *** Patrol Read in OpenBSD (https://www.marc.info/?l=openbsd-cvs&m=143285964216970&w=4) OpenBSD has recently imported some new code to support the Patrol Read (http://www.intel.com/support/motherboards/server/sb/CS-028742.htm) function of some RAID controllers In a nutshell, Patrol Read is a function that lets you check the health of your drives in the background, similar to a zpool "scrub" operation The goal is to protect file integrity by detecting drive failures before they can damage your data It detects bad blocks and prevents silent data corruption, while marking any bad sectors it finds *** HAMMER 2 improvements (http://lists.dragonflybsd.org/pipermail/commits/2015-May/418653.html) DragonFly BSD has been working on the second generation HAMMER FS It now uses LZ4 compression by default, which we've been big fans of in ZFS They've also switched to a faster CRC (http://lists.dragonflybsd.org/pipermail/commits/2015-May/418652.html) algorithm, further improving HAMMER's performance, especially (http://lists.dragonflybsd.org/pipermail/commits/2015-May/418651.html) when using iSCSI *** FreeBSD foundation May update (https://www.freebsdfoundation.org/press/2015mayupdate.pdf) The FreeBSD foundation has published another update newsletter, detailing some of the things they've been up to lately In it, you'll find some development status updates: notably more ARM64 work and the addition of 64 bit Linux emulation Some improvements were also made to FreeBSD's release building process for non-X86 architectures There's also an AsiaBSDCon recap that covers some of the presentations and the dev events They also have an accompanying blog post (http://freebsdfoundation.blogspot.com/2015/05/another-data-center-site-visit-nyi.html) where Glen Barber talks about more sysadmin and clusteradm work at NYI *** Interview - Lucas Holt - questions@midnightbsd.org (mailto:questions@midnightbsd.org) / @midnightbsd (https://twitter.com/midnightbsd) MidnightBSD News Roundup The launchd on train is never coming (http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/launchd-on-bsd.html) Replacement of init systems has been quite controversial in the last few years Fortunately, the BSDs have avoided most of that conflict thus far, but there have been a few efforts made to port launchd from OS X (https://en.wikipedia.org/wiki/Launchd) This blog post details the author's opinion on why he thinks we're never going to have launchd in any of the BSDs Email us your thoughts on the matter *** Native SSH comes to… Windows (http://blogs.msdn.com/b/looking_forward_microsoft__support_for_secure_shell_ssh1/archive/2015/06/02/managing-looking-forward-microsoft-support-for-secure-shell-ssh.aspx) In what may be the first (and last) mention of Microsoft on BSD Now... They've just recently announced that PowerShell will get native SSH support in the near future It's not based on the commercial SSH either, it's the same one from OpenBSD that we already use everywhere Up until now, interacting between BSD and Windows has required something like PuTTY, WinSCP, FileZilla or Cygwin - most of which are based on really outdated versions The announcement also promises that they'll be working with the OpenSSH community, so we'll see how many Microsoft-submitted patches make it upstream (or how many donations (http://www.openbsdfoundation.org/index.html) they make) *** Moving to FreeBSD (http://www.textplain.net/blog/2015/moving-to-freebsd/) This blog post describes a long-time Linux user's first BSD switching experience The author first talks about his Linux journey, eventually coming to love the more customization-friendly systems, but the journey ended with systemd After doing a bit of research, he gave FreeBSD a try and ended up liking it - the rest of the post mostly covers why that is He also plans to write about his experience with other BSDs, and is writing some tutorials too - we'll check in with him again later on *** Feedback/Questions Adam writes in (http://slexy.org/view/s29hS2cI05) Dan writes in (http://slexy.org/view/s20VRZYBsw) Ivan writes in (http://slexy.org/view/s20bumJ5u9) Josh writes in (http://slexy.org/view/s21BU6Pnka) ***

This week on the show, we've got something pretty different. We went to a Linux convention and asked various people if they've ever tried BSD and what they know about it. Stay tuned for that, all this week's news and, of course, answers to your emails, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines LUKS in OpenBSD (https://www.marc.info/?l=openbsd-tech&m=143247114716771&w=2) Last week, we were surprised to find out that DragonFlyBSD has support (http://leaf.dragonflybsd.org/cgi/web-man?command=cryptsetup§ion=8) for dm-crypt (https://en.wikipedia.org/wiki/Dm-crypt), sometimes referred to as LUKS (Linux Unified Key Setup (https://en.wikipedia.org/wiki/Linux_Unified_Key_Setup)) It looks like they might not be the only BSD with support for it for much longer, as OpenBSD is currently reviewing a patch for it as well LUKS would presumably be an additional option in OpenBSD's softraid (http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/softraid.4) system, which already provides native disk encryption Support hasn't been officially committed yet, it's still going through testing, but the code is there if you want to try it out and report your findings If enabled, this might pave the way for the first (semi-)cross platform encryption scheme since the demise of TrueCrypt (and maybe other BSDs will get it too in time) *** FreeBSD gets 64bit Linux emulation (https://lists.freebsd.org/pipermail/svn-src-head/2015-May/072255.html) For those who might be unfamiliar, FreeBSD has an emulation layer (https://www.freebsd.org/doc/handbook/linuxemu.html) to run Linux-only binaries (as rare as they may be) The most common use case is for desktop users, enabling them to run proprietary applications like Adobe Flash or Skype Similar systems can also be found in NetBSD (https://www.netbsd.org/docs/guide/en/chap-linux.html) and OpenBSD (http://www.openbsd.org/faq/faq9.html#Interact) (though disabled by default on the latter) However, until now, it's only supported binaries compiled for the i386 architecture This new update, already committed to -CURRENT, will open some new possibilities that weren't previously possible Meanwhile, HardenedBSD considers removing the emulation layer (https://hardenedbsd.org/content/poll-linuxulator-removal) entirely *** BSD at Open Source Conference 2015 Nagoya (https://mail-index.netbsd.org/netbsd-advocacy/2015/05/23/msg000686.html) We've covered the Japanese NetBSD users group setting up lots of machines at various conferences in the past, but now they're expanding Their latest report includes many of the NetBSD things you'd expect, but also a couple OpenBSD machines Some of the NetBSD ones included a Power Mac G4, SHARP NetWalker, Cubieboard2 and the not-so-foreign Raspberry Pi One new addition of interest is the OMRON LUNA88k, running the luna88k (http://www.openbsd.org/luna88k.html) port of OpenBSD There was even an old cell phone running Windows games (https://twitter.com/tsutsuii/status/601458973338775553) on NetBSD Check the mailing list post for some (https://pbs.twimg.com/media/CFrSmztWEAAS2uE.jpg) links (http://image.movapic.com/pic/m_201505230541335560130d49213.jpeg) to (http://image.movapic.com/pic/m_2015052305145455600ccea723a.jpeg) all (https://pbs.twimg.com/media/CFjPv9_UEAA8iEx.jpg:large) of (https://pbs.twimg.com/media/CD4k6ZUUMAA0tEM.jpg) the (https://pbs.twimg.com/media/CFqn1GXUsAAFuro.jpg) nice (https://pbs.twimg.com/media/CFdIS2IUkAAZvjc.jpg) pictures (https://pbs.twimg.com/media/CFf5mToUIAAFrRU.jpg) *** LLVM introduces OpenMP support (http://blog.llvm.org/2015/05/openmp-support_22.html) One of the things that has kept some people in the GCC camp is the lack of OpenMP (https://en.wikipedia.org/wiki/OpenMP) support in LLVM According to the blog post, it "enables Clang users to harness full power of modern multi-core processors with vector units" With Clang being the default in FreeBSD, Bitrig and OS X, and with some other BSDs exploring the option of switching, the need for this potential speed boost was definitely there This could also open some doors for more BSD in the area of high performance computing, putting an end to the current Linux monopoly *** Interview - Eric, FSF, John, Jose, Kris and Stewart Various "man on the street" style mini-interviews News Roundup BSD-licensed gettext replacement (https://gitlab.com/worr/libintl/blob/master/src/usr.bin/gettext/gettext.c) If you've ever installed ports on any of the BSDs, you've probably had GNU's gettext pulled in as a dependency Wikipedia says "gettext is an internationalization and localization (i18n) system commonly used for writing multilingual programs on Unix-like computer operating systems" A new BSD-licensed rewrite has begun, with the initial version being for NetBSD (but it's likely to be portable) If you've got some coding skills, get involved with the project - the more freely-licensed replacements, the better *** Unix history git repo (https://github.com/dspinellis/unix-history-repo) A git repository was recently created to show off some Unix source code history The repository contains 659 thousand commits and 2306 merges You can see early 386BSD commits all the way up to some of the more modern FreeBSD code If you want to browse through the giant codebase, it can be a great history lesson *** PCBSD 10.1.2 and Lumina updates (http://blog.pcbsd.org/2015/05/hotfix-release-to-10-1-2-now-available/) We mentioned 10.1.1 being released last week (and all the cool features a couple weeks before) but now 10.1.2 is out This minor update contained a few hotfixes: RAID-Z installation, cache and log devices and the text-only installer in UEFI mode There's also a new post (http://blog.pcbsd.org/2015/05/lumina-desktop-status-updatefaq/) on the PCBSD blog about Lumina, answering some frequently asked questions and giving a general status update *** Feedback/Questions Jake writes in (http://slexy.org/view/s25h4Biwzq) Van writes in (http://slexy.org/view/s2AF0bGmL6) Anonymous writes in (http://slexy.org/view/s20Ie1USFD) Dominik writes in (http://slexy.org/view/s20vBtoKqL) (text answer (http://slexy.org/view/s20RjbIT5v)) Chris writes in (http://slexy.org/view/s20USR3WzT) *** Mailing List Gold Death by chocolate (https://lists.mindrot.org/pipermail/openssh-unix-dev/2015-May/033945.html) ***

This time on the show, we'll be chatting with Jed Reynolds about ZFS. He's been using it extensively on a certain other OS, and we can both learn a bit about the other side's implementation. Answers to your questions and all this week's news, coming up on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Playing with sandboxing (http://blog.conviso.com.br/2015/05/playing-with-sandbox-analysis-of_13.html) Sandboxing and privilege separation are popular topics these days - they're the goal of the new "shill" scripting language, they're used heavily throughout OpenBSD, and they're gaining traction with the capsicum framework This blog post explores capsicum in FreeBSD, some of its history and where it's used in the base system They also include some code samples so you can verify that capsicum is actually denying the program access to certain system calls Check our interview about capsicum (http://www.bsdnow.tv/episodes/2014_05_28-the_friendly_sandbox) from a while back if you haven't seen it already *** OpenNTPD on by default (https://www.marc.info/?l=openbsd-cvs&m=143195693612629&w=4) OpenBSD has enabled ntpd (http://www.bsdnow.tv/episodes/2015_02_11-time_for_a_change) by default in the installer, rather than prompting the user if they want to turn it on In nearly every case, you're going to want to have your clock synced via NTP With the HTTPS constraints feature also enabled by default, this should keep the time checked and accurate, even against spoofing attacks Lots of problems can be traced back to the time on one system or another being wrong, so this will also eliminate some of those cases For those who might be curious (http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/etc/ntpd.conf), they're using the "pool.ntp.org (http://www.pool.ntp.org/en/)" cluster of addresses and google for HTTPS constraints (but these can be easily changed (http://www.bsdnow.tv/tutorials/ntpd)) *** FreeBSD workshop in Landshut (https://www.banym.de/freebsd/review-first-freebsd-workshop-in-landshut-on-15-may-2015) We mentioned a BSD installfest happening in Germany a few weeks back, and the organizer wrote in with a review of the event The installfest instead became a "FreeBSD workshop" session, introducing curious new users to some of the flagship features of the OS They covered when to use UFS or ZFS, firewall options, the release/stable/current branches and finally how to automate installations with Ansible If you're in south Germany and want to give similar introduction talks or Q&A sessions about the other BSDs, get in touch We'll hear more from him about how it went in the feedback section today *** Swap encryption in DragonFly (http://lists.dragonflybsd.org/pipermail/users/2015-May/207690.html) Doing full disk encryption (http://www.bsdnow.tv/tutorials/fde) is very important, but something that people sometimes overlook is encrypting their swap This can actually be more important than the contents of your disks, especially if an unencrypted password or key hits your swap (as it can be recovered quite easily) DragonFlyBSD has added a new experimental option to automatically encrypt your swap partition in fstab There was another way (http://lists.dragonflybsd.org/pipermail/users/2015-May/207691.html) to do it previously, but this is a lot easier You can achieve similar results in FreeBSD by adding ".eli" to the end of the swap device in fstab, there are a few steps (https://www.netbsd.org/docs/misc/#cgd-swap) to do it in NetBSD and swap in OpenBSD is encrypted by default A one-time key will be created and then destroyed in each case, making recovery of the plaintext nearly impossible *** Interview - Jed Reynolds - jed@bitratchet.com (mailto:jed@bitratchet.com) / @jed_reynolds (https://twitter.com/jed_reynolds) Comparing ZFS on Linux and FreeBSD News Roundup USB thermometer on OpenBSD (http://www.cambus.net/rding-temper-gold-usb-thermometer-on-openbsd/) So maybe you've got BSD on your server or router, maybe NetBSD on a toaster, but have you ever used a thermometer with one? This blog post introduces the RDing TEMPer Gold USB thermometer, a small device that can tell the room temperature, and how to get it working on OpenBSD Wouldn't you know it, OpenBSD has a native "ugold (http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/ugold.4)" driver to support it with the sensors framework How useful such a device would be is another story though *** NAS4Free now on ARM (http://sourceforge.net/projects/nas4free/files/NAS4Free-ARM/10.1.0.2.1511/) We talk a lot about hardware for network-attached storage devices on the show, but ARM doesn't come up a lot That might be changing soon, as NAS4Free has just released some ARM builds These new (somewhat experimental) images are based on FreeBSD 11-CURRENT Included in the announcement is a list of fully-supported and partially-supported hardware that they've tested it with If anyone has experience with running a NAS on slightly exotic hardware, write in to us *** pkgsrcCon 2015 CFP and info (http://pkgsrc.pub/pkgsrcCon/2015/) This year's pkgsrcCon will be in Berlin, Germany on July 4th and 5th (https://mail-index.netbsd.org/pkgsrc-users/2015/05/16/msg021560.html) They're looking for talk proposals and ideas for things you'd like to see If you or your company uses pkgsrc, or if you're just interested in NetBSD in general, it would be a good event to check out *** BSDTalk episode 253 (http://bsdtalk.blogspot.com/2015/05/bsdtalk253-george-neville-neil.html) BSDTalk has released another new episode In it, he interviews George Neville-Neil about the 2nd edition of "The Design and Implementation of the FreeBSD Operating System" They discuss what's new since the last edition, who the book's target audience is and a lot more We're up to 90 episodes now, slowly catching up to Will... *** Feedback/Questions Dominik writes in (http://slexy.org/view/s2SWlyuOeb) Brad writes in (http://slexy.org/view/s216z44lDU) Corvin writes in (http://slexy.org/view/s2djtX0dSE) James writes in (http://slexy.org/view/s21XM4hPRh) ***

This week on the show, we'll be talking to Mike Larkin about various memory protections in OpenBSD. We'll cover recent W^X improvements, SSP, ASLR, PIE and all kinds of acronyms! We've also got a bunch of news and answers to your questions, coming up on BSD Now - the place to B.. SD. This episode was brought to you by Headlines OpenSMTPD for the whole family (http://homing-on-code.blogspot.com/2015/05/accept-from-any-for-any-relay-via.html) Setting up a BSD mail server is something a lot of us are probably familiar with doing, at least for our own accounts This article talks about configuring a home mail server too, but even for the other people you live with After convincing his wife to use their BSD-based Owncloud server for backups, the author talks about moving her over to his brand new OpenSMTPD server too If you've ever run a mail server and had to deal with greylisting, you'll appreciate the struggle he went through In the end, BGP-based list distribution saved the day, and his family is being served well by a BSD box *** NetBSD on the Edgerouter Lite (https://blog.netbsd.org/tnf/entry/hands_on_experience_with_edgerouter) We've talked a lot about building your own BSD-based router on the show, but not many of the devices we mention are in the same price range as consumer devices The EdgeRouter Lite, a small MIPS-powered machine, is starting to become popular (and is a bit cheaper) A NetBSD developer has been hacking on it, and documents the steps to get a working install in this blog post The process is fairly simple, and you can cross-compile (http://www.bsdnow.tv/tutorials/current-nbsd) your own installation image on any CPU architecture (even from another BSD!) OpenBSD and FreeBSD also have some (http://www.openbsd.org/octeon.html) support (http://rtfm.net/FreeBSD/ERL/) for these devices *** Bitrig at NYC*BUG (https://www.youtube.com/watch?v=h4FhgBdYSUU) The New York City BSD users group has semi-regular meetings with presentations, and this time the speaker was John Vernaleo John discussed Bitrig (http://www.bsdnow.tv/episodes/2014_12_10-must_be_rigged), an OpenBSD fork that we've talked about a couple times on the show He talks about what they've been up to lately, why they're doing what they're doing, difference in supported platforms Ports and packages between the two projects are almost exactly the same, but he covers the differences in the base systems, how (some) patches get shared between the two and finally some development model differences *** OPNsense, meet HardenedBSD (https://hardenedbsd.org/article/shawn-webb/2015-05-08/hardenedbsd-teams-opnsense) Speaking of forks, two FreeBSD-based forked projects we've mentioned on the show, HardenedBSD (http://www.bsdnow.tv/episodes/2014_08_27-reverse_takeover) and OPNsense (http://www.bsdnow.tv/episodes/2015_01_14-common_sense_approach), have decided to join forces Backporting their changes to the 10-STABLE branch, HardenedBSD hopes to introduce some of their security additions to the OPNsense codebase Paired up with LibreSSL, this combination should offer a good solution for anyone wanting a BSD-based firewall with an easy web interface We'll cover more news on the collaboration as it comes out *** Interview - Mike Larkin - mlarkin@openbsd.org (mailto:mlarkin@openbsd.org) / @mlarkin2012 (https://twitter.com/mlarkin2012) Memory protections in OpenBSD: W^X (https://en.wikipedia.org/wiki/W%5EX), ASLR (https://en.wikipedia.org/wiki/Address_space_layout_randomization), PIE (https://en.wikipedia.org/wiki/Position-independent_code), SSP (https://en.wikipedia.org/wiki/Buffer_overflow_protection) News Roundup A closer look at FreeBSD (http://www.techopedia.com/2/31035/software/a-closer-look-at-freebsd) The week wouldn't be complete without at least one BSD article making it to a mainstream tech site This time, it's a high-level overview of FreeBSD, some of its features and where it's used Being that it's an overview article on a more mainstream site, you won't find anything too technical - it covers some BSD history, stability, ZFS, LLVM and Clang, ports and packages, jails and the licensing If you have any BSD-curious Linux friends, this might be a good one to send to them *** Linksys NSLU2 and NetBSD (http://ramblingfoo.blogspot.com/2015/05/linksys-nslu2-adventures-into-netbsd.html) The Linksys NSLU2 is a proprietary network-attached storage device introduced back in 2004 "About 2 months ago I set a goal to run some kind of BSD on the spare Linksys NSLU2 I had. This was driven mostly by curiosity, after listening to a few BSDNow episodes and becoming a regular listener [...]" After doing some research, the author of this post discovered that he could cross-compile NetBSD for the device straight from his Linux box If you've got one of these old devices kicking around, check out this write-up and get some BSD action on there *** OpenBSD disklabel templates (http://blog.jeffreyforman.net/2015/05/09/from-0-to-an-openbsd-install-with-no-hands-and-a-custom-disk-layou) We've covered OpenBSD's "autoinstall" feature for unattended installations in the past, but one area where it didn't offer a lot of customization was with the disk layout With a few recent changes (http://undeadly.org/cgi?action=article&sid=20150505123418), there are now a series of templates you can use for a completely customized partition scheme This article takes you through the process of configuring an autoinstall answer file and adding the new section for disklabel Combine this new feature with our -stable iso tutorial (http://www.bsdnow.tv/tutorials/stable-iso), and you could deploy completely patched and customized images en masse pretty easily *** FreeBSD native ARM builds (https://svnweb.freebsd.org/base?view=revision&revision=282693) FreeBSD -CURRENT builds for the ARM CPU architecture can now be built natively, without utilities that aren't part of base Some of the older board-specific kernel configuration files have been replaced, and now the "IMC6" target is used This goes along with what we read in the most recent quarterly status report - ARM is starting to get treated as a first class citizen *** Feedback/Questions Sean writes in (http://slexy.org/view/s2088U2OjO) Ron writes in (http://slexy.org/view/s29ZKhQKOz) Charles writes in (http://slexy.org/view/s2NCVHEKt1) Bostjan writes in (http://slexy.org/view/s2mGRoKo5G) ***

This time on the show, we'll be talking with Ed Schouten about CloudABI. It's a new application binary interface with a strong focus on isolation and restricted capabilities. As always, all this week's BSD news and answers to your emails, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines FreeBSD quarterly status report (https://www.freebsd.org/news/status/report-2015-01-2015-03.html) The FreeBSD team has posted a report of the activities that went on between January and March of this year As usual, it's broken down into separate reports from the various teams in the project (ports, kernel, virtualization, etc) The ports team continuing battling the flood of PRs, closing quite a lot of them and boasting nearly 7,000 commits this quarter The core team and cluster admins dealt with the accidental deletion of the Bugzilla database, and are making plans for an improved backup strategy within the project going forward FreeBSD's future release support model was also finalized and published in February, which should be a big improvement for both users and the release team Some topics are still being discussed internally, mainly MFCing ZFS ARC responsiveness patches to the 10 branch and deciding whether to maintain or abandon C89 support in the kernel code Lots of activity is happening in bhyve, some of which we've covered recently (http://www.bsdnow.tv/episodes/2015_04_29-on_the_list), and a number of improvements were made this quarter Clang, LLVM and LLDB have been updated to the 3.6.0 branch in -CURRENT Work to get FreeBSD booting natively on the POWER8 CPU architecture is also still in progress, but it does boot in KVM for the time being The project to replace forth in the bootloader with lua is in its final stages, and can be used on x86 already ASLR work (http://www.bsdnow.tv/episodes/2014_08_27-reverse_takeover) is still being done by the HardenedBSD guys, and their next aim is position-independent executable The report also touches on multipath TCP support, the new automounter, opaque ifnet, pkgng updates, secureboot (which should be in 10.2-RELEASE), GNOME and KDE on FreeBSD, PCIe hotplugging, nested kernel support and more Also of note: work is going on to make ARM a Tier 1 platform in the upcoming 11.0-RELEASE (and support for more ARM boards is still being added, including ARM64) *** OpenBSD 5.7 released (http://www.openbsd.org/57.html) OpenBSD has formally released another new version, complete with the giant changelog we've come to expect In the hardware department, 5.7 features many driver improvements and fixes, as well as support for some new things: USB 3.0 controllers, newer Intel and Atheros wireless cards and some additional 10gbit NICs If you're using one of the Soekris boards, there's even a new driver (http://bodgitandscarper.co.uk/openbsd/further-soekris-net6501-improvements-for-openbsd/) to manipulate the GPIO and LEDs on them - this has some fun possibilities Some new security improvements include: SipHash (https://en.wikipedia.org/wiki/SipHash) being sprinkled in some areas to protect hashing functions, big W^X improvements (https://www.marc.info/?l=openbsd-tech&m=142120787308107&w=2) in the kernel space, static PIE (http://www.bsdnow.tv/episodes/2015_04_15-pie_in_the_sky) on all architectures, deterministic "random" functions being replaced (https://www.marc.info/?l=openbsd-tech&m=141807224826859&w=2) with strong randomness, and support for remote logging over TLS The entire source tree has also been audited to use reallocarray (http://lteo.net/blog/2014/10/28/reallocarray-in-openbsd-integer-overflow-detection-for-free/), which unintentionally saved (https://splone.com/blog/2015/3/11/integer-overflow-prevention-in-c) OpenBSD's libc from being vulnerable to earlier attacks (https://guidovranken.wordpress.com/2015/02/04/full-disclosure-heap-overflow-in-h-spencers-regex-library-on-32-bit-systems/) affecting other BSDs' implementations Being that it's OpenBSD, a number of things have also been removed from the base system: procfs, sendmail, SSLv3 support and loadable kernel modules are all gone now (not to mention the continuing massacre of dead code in LibreSSL) Some people seem to be surprised about the removal of loadable modules, but almost nothing utilized them in OpenBSD, so it was really just removing old code that no one used anymore - very different from FreeBSD or Linux in this regard, where kernel modules are used pretty heavily BIND and nginx have been taken out, so you'll need to either use the versions in ports or switch to Unbound and the in-base HTTP daemon Speaking of httpd, it's gotten a number of new (http://www.openbsd.org/papers/httpd-slides-asiabsdcon2015.pdf) features (http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man5/httpd.conf.5), and has had time to grow and mature since its initial debut - if you've been considering trying it out, now would be a great time to do so This release also includes the latest OpenSSH (with stronger fingerprint types and host key rotation), OpenNTPD (with the HTTPS constraints feature), OpenSMTPD, LibreSSL and mandoc (http://www.bsdnow.tv/episodes/2014_11_12-a_mans_man) Check the errata page (http://www.openbsd.org/errata57.html) for any post-release fixes, and the upgrade guide (http://www.openbsd.org/faq/upgrade57.html) for specific instructions on updating from 5.6 Groundwork has also been laid for some major SMP scalability improvements - look forward to those in future releases There's a song and artwork (http://www.openbsd.org/lyrics.html#57) to go along with the release as always, and CDs should be arriving within a few days - we'll show some pictures next week Consider picking one up (https://www.openbsdstore.com) to support the project (and it's the only way to get puffy stickers) For those of you paying close attention, the banner image (http://www.openbsd.org/images/puffy57.gif) for this release just might remind you of a certain special episode (http://www.bsdnow.tv/episodes/2014_09_03-its_hammer_time) of BSD Now... *** Tor-BSD diversity project (https://torbsd.github.io/) We've talked about Tor on the show a few times, and specifically about getting more of the network on BSD (Linux has an overwhelming majority right now) A new initiative has started to do just that, called the Tor-BSD diversity project "Monocultures in nature are dangerous, as vulnerabilities are held in common across a broad spectrum. Diversity means single vulnerabilities are less likely to harm the entire ecosystem. [...] A single kernel vulnerability in GNU/Linux that impacting Tor relays could be devastating. We want to see a stronger Tor network, and we believe one critical ingredient for that is operating system diversity." In addition to encouraging people to put up more relays, they're also continuing work on porting the Tor Browser Bundle to BSD, so more desktop users can have easy access to online privacy There's an additional progress report (http://trac.haqistan.net/blog/tor-browser-ports-progress) for that part specifically, and it looks like most of the work is done now Engaging the broader BSD community about Tor and fixing up the official documentation are also both on their todo list If you've been considering running a node to help out, there's always our handy tutorial (http://www.bsdnow.tv/tutorials/tor) on getting set up *** PC-BSD 10.1.2-RC1 released (http://blog.pcbsd.org/2015/05/pc-bsd-10-1-2-rc1-now-available/) If you want a sneak peek at the upcoming PC-BSD 10.1.2, the first release candidate is now available to grab This quarterly update includes a number of new features, improvements and even some additional utilities PersonaCrypt is one of them - it's a new tool for easily migrating encrypted home directories between systems A new "stealth mode" option allows for a one-time login, using a blank home directory that gets wiped after use Similarly, a new "Tor mode" allows for easy tunneling of all your traffic through the Tor network IPFW is now the default firewall, offering improved VIMAGE capabilities The life preserver backup tool now allows for bare-metal restores via the install CD ISC's NTP daemon has been replaced with OpenNTPD (http://www.bsdnow.tv/episodes/2015_02_11-time_for_a_change), and OpenSSL has been replaced with LibreSSL (http://www.bsdnow.tv/episodes/2015_03_25-ssl_in_the_wild) It also includes the latest Lumina (http://www.bsdnow.tv/episodes/2014_09_10-luminary_environment) desktop, and there's another post dedicated to that (http://blog.pcbsd.org/2015/05/pc-bsd-10-1-2-rc1-lumina-desktop-0-8-4-released/) Binary packages have also been updated to fresh versions from the ports tree More details, including upgrade instructions, can be found in the linked blog post *** Interview - Ed Schouten - ed@freebsd.org (mailto:ed@freebsd.org) / @edschouten (https://twitter.com/edschouten) CloudABI (https://www.bsdcan.org/2015/schedule/track/Security/524.en.html) News Roundup Open Household Router Contraption (http://code.saghul.net/index.php/2015/05/01/announcing-the-open-household-router-contraption/) This article introduces OpenHRC, the "Open Household Router Contraption" In short, it's a set of bootstrapping scripts to turn a vanilla OpenBSD install into a feature-rich gateway device It also makes use of Ansible playbooks for configuration, allowing for a more "mass deployment" type of setup Everything is configured via a simple text file, and you end up with a local NTP server, DHCP server, firewall (obviously) and local caching DNS resolver - it even does DNSSEC validation All the code is open source and on Github (https://github.com/ioc32/openhrc), so you can read through what's actually being changed and put in place There's also a video guide (https://www.youtube.com/watch?v=LZeKDM5jc90) to the entire process, if you're more of a visual person *** OPNsense 15.1.10 released (https://forum.opnsense.org/index.php?topic=365.0) Speaking of BSD routers, if you're looking for a "prebuilt and ready to go" option, OPNsense has just released a new version 15.1.10 drops some of the legacy patches they inherited from pfSense, aiming to stay closer to the mainline FreeBSD source code Going along with this theme, they've redone how they do ports, and are now kept totally in sync with the regular ports tree Their binary packages are now signed using the fingerprint-style method, various GUI menus have been rewritten and a number of other bugs were fixed NanoBSD-based images are also available now, so you can try it out on hardware with constrained resources as well Version 15.1.10.1 (https://twitter.com/opnsense/status/596009164746432512) was released shortly thereafter, including a hotfix for VLANs *** IBM Workpad Z50 and NetBSD (https://www.ibm.com/developerworks/community/blogs/hpcgoulash/entry/ibm_workpad_z50_netbsd_an_interesting_combination1?lang=en) Before the infamous netbook fad came and went, IBM had a handheld PDA device that looked pretty much the same Back in 1999, they released the Workpad Z50 (http://www.hpcfactor.com/reviews/hardware/ibm/workpad-z50/) with Windows CE, sporting a 131MHz MIPS CPU, 16MB of RAM and a 640x480 display You can probably tell where this is going... the article is about installing NetBSD it "What prevents me from taking my pristine Workpad z50 to the local electronics recycling facility is NetBSD. With a little effort it is possible to install recent versions of NetBSD on the Workpad z50 and even have XWindows running" The author got pkgsrc up and running on it too, and cleverly used distcc to offload the compiling jobs to something a bit more modern He's also got a couple (https://www.youtube.com/watch?v=hSLVnSZKB9I) videos (https://www.youtube.com/watch?v=mIA-NWEHLM4) of the bootup process and running Xorg (neither of which we'd call "speedy" by any stretch of the imagination) *** FreeBSD from the trenches (http://freebsdfoundation.blogspot.com/2015/04/from-trenches-tips-tricks-edition.html) The FreeBSD foundation has a new blog post up in their "from the trenches" series, detailing FreeBSD in some real-world use cases In this installment, Glen Barber talks about how he sets up all his laptops with ZFS and GELI While the installer allows for an automatic ZFS layout, Glen notes that it's not a one-size-fits-all thing, and goes through doing everything manually Each command is explained, and he walks you through the process of doing an encrypted installation (http://www.bsdnow.tv/tutorials/fde) on your root zpool *** Broadwell in DragonFly (http://lists.dragonflybsd.org/pipermail/users/2015-May/207671.html) DragonFlyBSD has officially won the race to get an Intel Broadwell graphics driver Their i915 driver has been brought up to speed with Linux 3.14's, adding not only Broadwell support, but many other bugfixes for other cards too It's planned for commit to the main tree very soon, but you can test it out with a git branch for the time being *** Feedback/Questions Bostjan writes in (http://slexy.org/view/s216QQcHyX) Hunter writes in (http://slexy.org/view/s21hGSk3c0) Hrishi writes in (http://slexy.org/view/s20JwPw9Je) Clint writes in (http://slexy.org/view/s2x1GYr7y6) Sergei writes in (http://slexy.org/view/s2swXxr2PX) *** Mailing List Gold How did you guess (https://lists.freebsd.org/pipermail/freebsd-advocacy/2015-May/004541.html) ***

Coming up this time on the show, we'll be speaking with Christos Zoulas, a NetBSD security officer. He's got a new project called blacklistd, with some interesting possibilities for stopping bruteforce attacks. We've also got answers to your emails and all this week's news, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines New PAE support in OpenBSD (https://www.marc.info/?l=openbsd-cvs&m=142990524317070&w=2) OpenBSD has just added Physical Address Extention (https://en.wikipedia.org/wiki/Physical_Address_Extension) support to the i386 architecture, but it's probably not what you'd think of when you hear the term In most operating systems, PAE's main advantage is to partially circumvent the 4GB memory limit on 32 bit platforms - this version isn't for that Instead, this change specifically allows the system to use the No-eXecute Bit (https://en.wikipedia.org/wiki/NX_bit#OpenBSD) of the processor for the userland, further hardening the in-place memory protections Other operating systems enable the CPU feature without doing anything to the page table entries (https://en.wikipedia.org/wiki/Page_table#Role_of_the_page_table), so they do get the available memory expansion, but don't get the potential security benefit As we discussed in a previous episode (http://www.bsdnow.tv/episodes/2015_01_14-common_sense_approach), the AMD64 platform already saw some major W^X kernel and userland improvements - the i386 kernel reworking will begin shortly Not all CPUs support this feature, but, if yours supports NX, this will improve upon the previous version of W^X that was already there The AMD64 improvements will be in 5.7, due out in just a couple days as of when we're recording this, but the i386 improvements will likely be in 5.8 *** Booting Windows in bhyve (https://twitter.com/nahannisys/status/591733319357730816) Work on FreeBSD's bhyve (http://www.bsdnow.tv/episodes/2014_01_15-bhyve_mind) continues, and a big addition is on the way Thus far, bhyve has only been able to boot operating systems with a serial console - no VGA, no graphics, no Windows This is finally changing, and a teasing screenshot of Windows Server was recently posted on Twitter Graphics emulation is still in the works; this image was taken by booting headless and using RDP A lot of the needed code is being committed to -CURRENT now, but the UEFI portion of it requires a bit more development (and the aim for that is around the time of BSDCan) Not a lot of details on the matter currently, but we'll be sure to bring you more info as it comes out Are you more interested in bhyve or Xen on FreeBSD? Email us your thoughts *** MidnightBSD 0.6 released (http://www.midnightbsd.org/notes/) MidnightBSD is a smaller project we've not covered a lot on the show before It's an operating system that was forked from FreeBSD back in the 6.1 days, and their focus seems to be on ease-of-use They also have their own, smaller version of FreeBSD ports, called "mports" If you're already using it, this new version is mainly a security and bugfix release It syncs up with the most recent FreeBSD security patches and gets a lot of their ports closer to the latest versions You can check their site (http://www.midnightbsd.org/about/) for more information about the project We're trying to get the lead developer to come on for an interview, but haven't heard anything back yet *** OpenBSD rewrites the file utility (https://www.marc.info/?l=openbsd-cvs&m=142989267412968&w=4) We're all probably familiar with the traditional file (https://en.wikipedia.org/wiki/File_%28command%29) command - it's been around since the 1970s (http://darwinsys.com/file/) For anyone who doesn't know, it's used to determine what type of file something actually is This tool doesn't see a lot of development these days, and it's had its share of security issues as well Some of those security issues remain (https://www.marc.info/?l=openbsd-tech&m=141857001403570&w=2) unfixed (https://www.marc.info/?l=freebsd-security&m=142980545021888&w=2) in various BSDs even today, despite being publicly known for a while It's not uncommon for people to run file on random things they download from the internet, maybe even as root, and some of the previous bugs have allowed file to overwrite other files or execute code as the user running it When you think about it, file was technically designed to be used on untrusted files OpenBSD developer Nicholas Marriott, who also happens to be the author of tmux, decided it was time to do a complete rewrite - this time with modern coding practices and the usual OpenBSD scrutiny This new version will, by default, run as an unprivileged user (https://www.marc.info/?l=openbsd-cvs&m=143014212727213&w=2) with no shell, and in a systrace sandbox (https://www.marc.info/?l=openbsd-cvs&m=143014276127454&w=2), strictly limiting what system calls can be made With these two things combined, it should drastically reduce the damage a malicious file could potentially do Ian Darwin, the original author of the utility, saw the commit and replied (https://www.marc.info/?l=openbsd-cvs&m=142989483913635&w=4), in what may be a moment in BSD history to remember It'll be interesting to see if the other BSDs, OS X, Linux or other UNIXes consider adopting this implementation in the future - someone's already thrown together an unofficial portable version Coincidentally, the lead developer and current maintainer of file just happens to be our guest today… *** Interview - Christos Zoulas - christos@netbsd.org (mailto:christos@netbsd.org) blacklistd (https://www.youtube.com/watch?v=0UKCAsezF3Q) and NetBSD advocacy News Roundup GSoC-accepted BSD projects (https://www.google-melange.com/gsoc/projects/list/google/gsoc2015) The Google Summer of Code people have published a list of all the projects that got accepted this year, and both FreeBSD and OpenBSD are on that list FreeBSD's list (https://wiki.freebsd.org/SummerOfCode2015Projects) includes: NE2000 device model in userspace for bhyve, updating Ficl in the bootloader, type-aware kernel virtual memory access for utilities, JIT compilation for firewalls, test cluster automation, Linux packages for pkgng, an mtree parsing and manipulation library, porting bhyve to ARM-based platforms, CD-ROM emulation in CTL, libc security extensions, gptzfsboot support for dynamically discovering BEs during startup, CubieBoard support, a bhyve version of the netmap virtual passthrough for VMs, PXE support for FreeBSD guests in bhyve and finally.. memory compression and deduplication OpenBSD's list (http://www.openbsdfoundation.org/gsoc2015.html) includes: asynchronous USB transfer submission from userland, ARM SD/MMC & controller driver in libsa, improving USB userland tools and ioctl, automating module porting, implementing a KMS driver to the kernel and, wait for it... porting HAMMER FS to OpenBSD We'll be sure to keep you up to date on developments from both projects Hopefully the other BSDs will make the cut too next year *** FreeBSD on the Gumstix Duovero (http://www.jumpnowtek.com/gumstix-freebsd/FreeBSD-Duovero-build-workstation-setup.html) If you're not familiar with the Gumstix Duovero, it's an dual core ARM-based computer-on-module (https://store.gumstix.com/index.php/coms/duovero-coms.html) They actually look more like a stick of RAM than a mini-computer This article shows you how to build a FreeBSD -CURRENT image to run on them, using crochet-freebsd (https://github.com/freebsd/crochet) If anyone has any interesting devices like this that they use BSD on, write up something about it and send it to us *** EU study recommends OpenBSD (https://joinup.ec.europa.eu/community/osor/news/ep-study-%E2%80%9Ceu-should-finance-key-open-source-tools%E2%80%9D) A recent study by the European Parliament was published, explaining that more funding should go into critical open source projects and tools This is especially important, in all countries, after the mass surveillance documents came out "[...] the use of open source computer operating systems and applications reduces the risk of privacy intrusion by mass surveillance. Open source software is not error free, or less prone to errors than proprietary software, the experts write. But proprietary software does not allow constant inspection and scrutiny by a large community of experts." The report goes on to mention users becoming more and more security and privacy-aware, installing additional software to help protect themselves and their traffic from being spied on Alongside Qubes, a Linux distro focused on containment and isolation, OpenBSD got a special mention: "Proactive security and cryptography are two of the features highlighted in the product together with portability, standardisation and correctness. Its built-in cryptography and packet filter make OpenBSD suitable for use in the security industry, for example on firewalls, intrusion-detection systems and VPN gateways" Reddit, Undeadly and Hacker News also had (https://www.reddit.com/r/programming/comments/340xh3/eu_study_recommends_use_of_openbsd_for_its/) some (http://undeadly.org/cgi?action=article&sid=20150427093546) discussion (https://news.ycombinator.com/item?id=9445831), particularly about corporations giving back to the BSDs that they make use of in their infrastructure - something we've discussed with Voxer (http://www.bsdnow.tv/episodes/2014_10_08-behind_the_masq) and M:Tier (http://www.bsdnow.tv/episodes/2015_04_22-business_as_usual) before *** FreeBSD workflow with Git (https://lists.freebsd.org/pipermail/freebsd-current/2015-April/055551.html) If you're interested in contributing to FreeBSD, but aren't a big fan of SVN, they have a Github mirror too This mailing list post talks about interacting between (https://wiki.freebsd.org/GitWorkflow/GitSvn) the official source repository and the Git mirror This makes it easy to get pull requests merged into the official tree, and encourages more developers to get involved *** Feedback/Questions Sean writes in (http://slexy.org/view/s2vjh3ogvG) Bryan writes in (http://slexy.org/view/s20GMcWvKE) Sean writes in (http://slexy.org/view/s21M1imT3d) Charles writes in (http://slexy.org/view/s25ScxQSwb) ***

Coming up this time on the show, we'll be chatting with Antoine Jacoutot about how M:Tier uses BSD in their business. After that, we'll be discussing the different release models across the BSDs, and which style we like the most. As always, answers to your emails and all the latest news, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Optimizing TLS for high bandwidth applications (https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf) Netflix has released a report on some of their recent activities, pushing lots of traffic through TLS on FreeBSD TLS has traditionally had too much overhead for the levels of bandwidth they're using, so this pdf outlines some of their strategy in optimizing it The sendfile() syscall (which nginx uses) isn't available when data is encrypted in userland To get around this, Netflix is proposing to add TLS support to the FreeBSD kernel Having encrypted movie streams would be pretty neat *** Crypto in unexpected places (https://www.marc.info/?l=openbsd-cvs&m=142944822223482&w=2) OpenBSD is somewhat known for its integrated cryptography, right down to strong randomness in every place you could imagine (process IDs, TCP initial sequence numbers, etc) One place you might not expect crypto to be used (or even needed) is in the "ping" utility, right? Well, think again David Gwynne recently committed (https://www.marc.info/?l=openbsd-cvs&m=142944754923359&w=2) a change that adds MAC (https://en.wikipedia.org/wiki/Message_authentication_code) to the ping timestamp payload By default, it'll be filled with a ChaCha stream instead of an unvarying payload, and David says "this lets us have some confidence that the timestamp hasn't been damaged or tampered with in transit" Not only is this a security feature, but it should also help detect dodgy or malfunctioning network equipment going forward Maybe we can look forward to a cryptographically secure "echo" command next... *** Broadwell in DragonFly (http://www.dragonflybsd.org/docs/newhandbook/docs/newhandbook/BroadwellBoxes/) The DragonFlyBSD guys have started a new page on their wiki to discuss Broadwell hardware and its current status Matt Dillon, the project lead, recently bought some hardware with this chipset, and lays out what works and what doesn't work The two main show-stoppers right now are the graphics and wireless, but they have someone who's already making progress with the GPU support Wireless support will likely have to wait until FreeBSD gets it, then they'll port it back over None of the BSDs currently have full Broadwell support, so stay tuned for further updates *** DIY NAS software roundup (http://blog.brianmoses.net/2015/04/diy-nas-software-roundup.html) In this blog post, the author compares a few different software solutions for a network attached storage device He puts FreeNAS, one of our favorites, up against a number of opponents - both BSD and Linux-based NAS4Free gets an honorable mention as well, particularly for its lower hardware requirements and sleek interface If you've been thinking about putting together a NAS, but aren't quite comfortable enough to set it up by yourself yet, this article should give you a good view of the current big names Some competition is always good, gotta keep those guys on their toes *** Interview - Antoine Jacoutot - ajacoutot@openbsd.org (mailto:ajacoutot@openbsd.org) / @ajacoutot (https://twitter.com/ajacoutot) OpenBSD at M:Tier (http://www.mtier.org/about-us/), business adoption of BSD, various topics News Roundup OpenBSD on DigitalOcean (http://www.tubsta.com/2015/04/openbsd-on-digital-ocean/) When DigitalOcean rolled out initial support for FreeBSD, it was a great step in the right direction - we hoped that all the other BSDs would soon follow This is not yet the case, but a blog article here has details on how you can install OpenBSD (and likely the others too) on your VPS Using a -current snapshot and some swapfile trickery, it's possible to image an OpenBSD ramdisk installer onto an unmounted portion of the virtual disk After doing so, you just boot from their web UI-based console and can perform a standard installation You will have to pay special attention to some details of the disk layout, but this article takes you through the entire process step by step *** Initial ARM64 support lands in FreeBSD (https://svnweb.freebsd.org/base?view=revision&revision=281494) The ARM64 architecture, sometimes called ARMv8 or AArch64 (https://wiki.freebsd.org/arm64), is a new generation of CPUs that will mostly be in embedded devices FreeBSD has just gotten support for this platform in the -CURRENT branch Previously, it was only the beginnings of the kernel and enough bits to boot in QEMU - now a full build (https://lists.freebsd.org/pipermail/freebsd-testing/2015-April/000918.html) is possible Work should now start happening in the main source code tree, and hopefully they'll have full support in a branch soon *** Scripting with least privilege (http://shill.seas.harvard.edu/) A new scripting language with a focus on privilege separation and running with only what's absolutely needed has been popular in the headlines lately Shell scripts are used everywhere today: startup scripts, orchestration scripts for mass deployment, configuring and compiling software, etc. Shill aims to answer the questions "how do we limit the authority of scripts" and "how do we determine what authority is necessary" by including a declarative security policy that's checked and enforced by the language runtime If used on FreeBSD, Shill will use Capsicum for sandboxing You can find some more of the technical information in their documentation pdf (http://shill.seas.harvard.edu/shill-osdi-2014.pdf) or watch their USENIX presentation (https://2459d6dc103cb5933875-c0245c5c937c5dedcca3f1764ecc9b2f.ssl.cf2.rackcdn.com/osdi14/moore.mp4) video Hacker News also had some discussion (https://news.ycombinator.com/item?id=9328277) on the topic *** OpenBSD first impressions (http://blog.greduan.com/2015-04-19-mstobfi.html) A brand new BSD user has started documenting his experience through a series of blog posts Formerly a Linux guy, he's tried out FreeBSD and OpenBSD so far, and is currently working on an OpenBSD desktop The first post goes into why he chose BSD at all, why he's switching away from Linux, how the initial transition has been, what you'll need to relearn and what he's got planned going forward He's only been using OpenBSD for a few days as of the time this was written - we don't usually get to hear from people this early in on their BSD journey, so it offers a unique perspective *** PCBSD and 4K oh my! (http://blog.pcbsd.org/2015/04/pc-bsd-and-4k-oh-my/) Yesterday, Kris got ahold of some 4K monitor hardware to test PC-BSD out The short of it - It works great! Minor tweaks being made to some of the PC-BSD defaults to better accommodate 4K out of box This particular model monitor ships with DisplayPort set to 1.1 mode only, switching it to 1.2 mode enables 60Hz properly *** Feedback/Questions Darin writes in (http://slexy.org/view/s21kFuvAFs) Mitch writes in (http://slexy.org/view/s2nf4o9p4E) *** Discussion Comparison of BSD release cycles FreeBSD (https://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/introduction.html#idp55486416), OpenBSD (http://www.openbsd.org/faq/faq5.html#Flavors), NetBSD (https://www.netbsd.org/releases/release-map.html) and DragonFlyBSD (https://www.dragonflybsd.org/releases/) ***

This time on the show, we'll be talking with Pascal Stumpf about static PIE in the upcoming OpenBSD release. He'll tell us what types of attacks it prevents, and why it's such a big deal. We've also got answers to questions from you in the audience and all this week's news, on BSD Now - the place to B.. SD. This episode was brought to you by Headlines Solaris' networking future is with OpenBSD (http://bsdly.blogspot.com/2015/04/solaris-admins-for-glimpse-of-your.html) A curious patch from someone with an Oracle email address was recently sent in (https://www.marc.info/?l=openbsd-tech&m=142822852613581&w=2) to one of the OpenBSD mailing lists It was revealed that future releases of Solaris are going to drop their IPFilter firewall entirely, in favor of a port of the current version of PF For anyone unfamiliar with the history of PF, it was actually made as a replacement for IPFilter in OpenBSD, due to some licensing issues What's more, Solaris was the original development platform for IPFilter, so the fact that it would be replaced in its own home is pretty interesting This blog post goes through some of the backstory of the two firewalls PF is in a lot of places - other BSDs, Mac OS X and iOS - but there are plenty of other OpenBSD-developed technologies end up ported to other projects too "Many of the world's largest corporations and government agencies are heavy Solaris users, meaning that even if you're neither an OpenBSD user or a Solaris user, your kit is likely interacting intensely with both kinds, and with Solaris moving to OpenBSD's PF for their filtering needs, we will all be benefiting even more from the OpenBSD project's emphasis on correctness, quality and security" You're welcome, Oracle *** BAFUG discussion videos (https://www.youtube.com/watch?v=Cb--h-iOQEM#t=15) The Bay Area FreeBSD users group has been uploading some videos from their recent meetings Sean Bruno gave a recap of his experiences at EuroBSDCon last year, including the devsummit and some proposed ideas from it (as well as their current status) Craig Rodrigues also gave a talk (https://www.youtube.com/watch?v=kPs8Dni_g3M#t=15) about Kyua and the FreeBSD testing framework Lastly, Kip Macy gave a talk (https://www.youtube.com/watch?v=Q13WtuqbZ7E#t=15) titled "network stack changes, user-level FreeBSD" The main two subjects there are some network stack changes, and how to get more people contributing, but there's also open discussion about a variety of FreeBSD topics If you're close to the Bay Area in California, be sure to check out their group and attend a meeting sometime *** More than just a makefile (http://homing-on-code.blogspot.com/2015/04/ports-are-more-than-just-makefile.html) If you're not a BSD user just yet, you might be wondering how the various ports and pkgsrc systems compare to the binary way of doing things on Linux This blog entry talks about the ports system in OpenBSD, but a lot of the concepts apply to all the ports systems across the BSDs As it turns out, the ports system really isn't that different from a binary package manager - they are what's used to create binary packages, after all The author goes through what makefiles do, customizing which options software is compiled with, patching source code to build and getting those patches back upstream After that, he shows you how to get your new port tested, if you're interesting in doing some porting yourself, and getting involved with the rest of the community This post is very long and there's a lot more to it, so check it out (and more discussion on Hacker News (https://news.ycombinator.com/item?id=9360827)) *** Securing your home fences (http://www.scip.ch/en/?labs.20150409) Hopefully all our listeners have realized that trusting your network(s) to a consumer router is a bad (http://www.devttys0.com/2015/04/hacking-the-d-link-dir-890l/) idea (https://threatpost.com/12-million-home-routers-vulnerable-to-takeover/109970) by now We hear from a lot of users who want to set up some kind of BSD-based firewall, but don't hear back from them after they've done it.. until now In this post, someone goes through the process of setting up a home firewall using OPNsense on a PCEngines APU board (http://www.pcengines.ch/apu1d4.htm) He notes that you have a lot of options software-wise, including vanilla FreeBSD (http://blog.pcbsd.org/2015/01/using-trueos-as-a-ipfw-based-home-router/), OpenBSD (http://www.bsdnow.tv/tutorials/openbsd-router) or even Linux, but decided to go with OPNsense because of the easy interface and configuration The post covers all the hardware you'll need, getting the OS installed to a flash drive or SD card and going through the whole process Finally, he goes through setting up the firewall with the graphical interface, applying updates and finishing everything up If you don't have any experience using a serial console, this guide also has some good info for beginners about those (which also applies to regular FreeBSD) We love super-detailed guides like this, so everyone should write more and send them to us immediately *** Interview - Pascal Stumpf - pascal@openbsd.org (mailto:pascal@openbsd.org) Static PIE in OpenBSD News Roundup LLVM's new libFuzzer (http://blog.llvm.org/2015/04/fuzz-all-clangs.html) We've discussed fuzzing on the show a number of times, albeit mostly with the American Fuzzy Lop utility It looks like LLVM is going to have their own fuzzing tool too now The Clang and LLVM guys are no strangers to this type of code testing, but decided to "close the loop" and start fuzzing parts of LLVM (including Clang) using LLVM itself With Clang being the default in both FreeBSD and Bitrig, and with the other BSDs considering the switch, this could make for some good bug hunting across all the projects in the future *** HardenedBSD upgrades secadm (http://hardenedbsd.org/article/shawn-webb/2015-04-14/introducing-secadm-02) The HardenedBSD guys have released a new version of their secadm tool, with the showcase feature being integriforce support We covered both the secadm tool and integriforce in previous episodes, but the short version is that it's a way to prevent files from being altered (even as root) Their integriforce feature itself has also gotten a couple improvements: shared objects are now checked too, instead of just binaries, and it uses more caching to speed up the whole process now *** RAID5 returns to OpenBSD (https://www.marc.info/?l=openbsd-tech&m=142877132517229&w=2) OpenBSD's softraid (http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/softraid.4) subsystem, somewhat similar to FreeBSD's GEOM, has had experimental RAID5 support for a while However, it was exactly that - experimental - and required a recompile to enable With some work from recent hackathons, the final piece (https://www.marc.info/?l=openbsd-cvs&m=142876943116907&w=2) was added to enable resuming partial array rebuilds Now it's on by default (https://www.marc.info/?l=openbsd-cvs&m=142877026917030&w=2), and there's a call for testing being put out, so grab a snapshot and put the code through its paces The bioctl softraid command also now supports (https://www.marc.info/?l=openbsd-cvs&m=142877223817406&w=2) DUIDs during pseudo-device detachment, possibly paving the way for the installer to drop (https://www.marc.info/?l=openbsd-tech&m=142643313416298&w=2) the "do you want to enable DUIDs?" question entirely *** pkgng 1.5.0 released (https://lists.freebsd.org/pipermail/freebsd-current/2015-April/055463.html) Going back to what we talked about last week (http://www.bsdnow.tv/episodes/2015_04_08-pkg_remove_freebsd-update), the final version of pkgng 1.5.0 is out The "provides" and "requires" support is finally in a regular release A new "-r" switch will allow for direct installation to a chroot or alternate root directory Memory usage should be much better now, and some general code speed-ups were added This version also introduces support for Mac OS X, NetBSD and EdgeBSD - it'll be interesting to see if anything comes of that Many more bugs were fixed, so check the mailing list announcement for the rest (and plenty new bugs were added, according to bapt) *** p2k15 hackathon reports (http://undeadly.org/cgi?action=article&sid=20150411160247) There was another OpenBSD hackathon that just finished up in the UK - this time it was mainly for ports work As usual, the developers sent in reports of some of the things they got done at the event Landry Breuil, both an upstream Mozilla developer and an OpenBSD developer, wrote in about the work he did on the Firefox port (specifically WebRTC) and some others, as well as reviewing lots of patches that were ready to commit Stefan Sperling wrote in (http://undeadly.org/cgi?action=article&sid=20150414064710), detailing his work with wireless chipsets, specifically when the vendor doesn't provide any hardware documentation, as well as updating some of the games in ports Ken Westerback also sent in a report (http://undeadly.org/cgi?action=article&sid=20150413163333), but decided to be a rebel and not work on ports at all - he got a lot of GPT-related work done, and also reviewed the RAID5 support we talked about earlier *** Feedback/Questions Shaun writes in (http://slexy.org/view/s2iNBo2swq) Hrishi writes in (http://slexy.org/view/s202BRLwrd) Randy writes in (http://slexy.org/view/s2KT7M35uY) Zach writes in (http://slexy.org/view/s2Q5lOoxzl) Ben writes in (http://slexy.org/view/s2ynDjuzVi) *** Mailing List Gold Gstreamer hates us (https://www.marc.info/?l=openbsd-ports&m=142884995931428&w=2) At least he's honest (https://lists.torproject.org/pipermail/tor-relays/2015-April/006765.html) I find myself in a situation (https://lists.freebsd.org/pipermail/freebsd-current/2015-April/055390.html) ***