Podcasts about vbsdcon

Unix is 50, Hunting down Ken's PDP-7, OpenBSD and OPNSense have new releases, Clarification on what GhostBSD is, sshuttle - VPN over SSH, and more. Headlines Unix is 50 (https://www.bell-labs.com/unix50/) In the summer of 1969 computer scientists Ken Thompson and Dennis Ritchie created the first implementation of Unix with the goal of designing an elegant and economical operating system for a little-used PDP-7 minicomputer at Bell Labs. That modest project, however, would have a far-reaching legacy. Unix made large-scale networking of diverse computing systems — and the Internet — practical. The Unix team went on to develop the C language, which brought an unprecedented combination of efficiency and expressiveness to programming. Both made computing more "portable". Today, Linux, the most popular descendent of Unix, powers the vast majority of servers, and elements of Unix and Linux are found in most mobile devices. Meanwhile C++ remains one of the most widely used programming languages today. Unix may be a half-century old but its influence is only growing. Hunting down Ken's PDP-7: video footage found (https://bsdimp.blogspot.com/2019/10/video-footage-of-first-pdp-7-to-run-unix.html) In my prior blog post, I traced Ken's scrounged PDP-7 to SN 34. In this post I'll show that we have actual video footage of that PDP-7 due to an old film from Bell Labs. this gives us almost a minute of footage of the PDP-7 Ken later used to create Unix. News Roundup OpenBSD 6.6 Released (https://openbsd.org/66.html) Announce: https://marc.info/?l=openbsd-tech&m=157132024225971&w=2 Upgrade Guide: https://openbsd.org/faq/upgrade66.html Changelog: https://openbsd.org/plus66.html OPNsense 19.7.5 released (https://opnsense.org/opnsense-19-7-5-released/) Hello friends and followers, Lots of plugin and ports updates this time with a few minor improvements in all core areas. Behind the scenes we are starting to migrate the base system to version 12.1 which is supposed to hit the next 20.1 release. Stay tuned for more infos in the next month or so. Here are the full patch notes: + system: show all swap partitions in system information widget + system: flatten services_get() in preparation for removal + system: pin Syslog-ng version to specific package name + system: fix LDAP/StartTLS with user import page + system: fix a PHP warning on authentication server page + system: replace most subprocess.call use + interfaces: fix devd handling of carp devices (contributed by stumbaumr) + firewall: improve firewall rules inline toggles + firewall: only allow TCP flags on TCP protocol + firewall: simplify help text for direction setting + firewall: make protocol log summary case insensitive + reporting: ignore malformed flow records + captive portal: fix type mismatch for timeout read + dhcp: add note for static lease limitation with lease registration (contributed by Northguy) + ipsec: add margintime and rekeyfuzz options + ipsec: clear $dpdline correctly if not set + ui: fix tokenizer reorder on multiple saves + plugins: os-acme-client 1.26[1] + plugins: os-bind will reload bind on record change (contributed by blablup) + plugins: os-etpro-telemetry minor subprocess.call replacement + plugins: os-freeradius 1.9.4[2] + plugins: os-frr 1.12[3] + plugins: os-haproxy 2.19[4] + plugins: os-mailtrail 1.2[5] + plugins: os-postfix 1.11[6] + plugins: os-rspamd 1.8[7] + plugins: os-sunnyvalley LibreSSL support (contributed by Sunny Valley Networks) + plugins: os-telegraf 1.7.6[8] + plugins: os-theme-cicada 1.21 (contributed by Team Rebellion) + plugins: os-theme-tukan 1.21 (contributed by Team Rebellion) + plugins: os-tinc minor subprocess.call replacement + plugins: os-tor 1.8 adds dormant mode disable option (contributed by Fabian Franz) + plugins: os-virtualbox 1.0 (contributed by andrewhotlab) Dealing with the misunderstandings of what is GhostBSD (http://ghostbsd.org/node/194) Since the release of 19.09, I have seen a lot of misunderstandings on what is GhostBSD and the future of GhostBSD. GhostBSD is based on TrueOS with FreeBSD 12 STABLE with our twist to it. We are still continuing to use TrueOS for OpenRC, and the new package's system for the base system that is built from ports. GhostBSD is becoming a slow-moving rolling release base on the latest TrueOS with FreeBSD 12 STABLE. When FreeBSD 13 STABLE gets released, GhostBSD will be upgraded to TrueOS with FreeBSD 13 STABLE. Our official desktop is MATE, which means that the leading developer of GhostBSD does not officially support XFCE. Community releases are maintained by the community and for the community. GhostBSD project will provide help to build and to host the community release. If anyone wants to have a particular desktop supported, it is up to the community. Sure I will help where I can, answer questions and guide new community members that contribute to community release. There is some effort going on for Plasma5 desktop. If anyone is interested in helping with XFCE and Plasma5 or in creating another community release, you are well come to contribute. Also, Contribution to the GhostBSD base system, to ports and new ports, and in house software are welcome. We are mostly active on Telegram https://t.me/ghostbsd, but you can also reach us on the forum. SHUTTLE – VPN over SSH | VPN Alternative (https://www.terminalbytes.com/sshuttle-vpn-over-ssh-vpn-alternative/) Looking for a lightweight VPN client, but are not ready to spend a monthly recurring amount on a VPN? VPNs can be expensive depending upon the quality of service and amount of privacy you want. A good VPN plan can easily set you back by 10$ a month and even that doesn’t guarantee your privacy. There is no way to be sure whether the VPN is storing your confidential information and traffic logs or not. sshuttle is the answer to your problem it provides VPN over ssh and in this article we’re going to explore this cheap yet powerful alternative to the expensive VPNs. By using open source tools you can control your own privacy. VPN over SSH – sshuttle sshuttle is an awesome program that allows you to create a VPN connection from your local machine to any remote server that you have ssh access on. The tunnel established over the ssh connection can then be used to route all your traffic from client machine through the remote machine including all the dns traffic. In the bare bones sshuttle is just a proxy server which runs on the client machine and forwards all the traffic to a ssh tunnel. Since its open source it holds quite a lot of major advantages over traditional VPN. OpenSSH 8.1 Released (http://www.openssh.com/txt/release-8.1) Security ssh(1), sshd(8), ssh-add(1), ssh-keygen(1): an exploitable integer overflow bug was found in the private key parsing code for the XMSS key type. This key type is still experimental and support for it is not compiled by default. No user-facing autoconf option exists in portable OpenSSH to enable it. This bug was found by Adam Zabrocki and reported via SecuriTeam's SSD program. ssh(1), sshd(8), ssh-agent(1): add protection for private keys at rest in RAM against speculation and memory side-channel attacks like Spectre, Meltdown and Rambleed. This release encrypts private keys when they are not in use with a symmetric key that is derived from a relatively large "prekey" consisting of random data (currently 16KB). This release includes a number of changes that may affect existing configurations: ssh-keygen(1): when acting as a CA and signing certificates with an RSA key, default to using the rsa-sha2-512 signature algorithm. Certificates signed by RSA keys will therefore be incompatible with OpenSSH versions prior to 7.2 unless the default is overridden (using "ssh-keygen -t ssh-rsa -s ..."). New Features ssh(1): Allow %n to be expanded in ProxyCommand strings ssh(1), sshd(8): Allow prepending a list of algorithms to the default set by starting the list with the '^' character, E.g. "HostKeyAlgorithms ^ssh-ed25519" ssh-keygen(1): add an experimental lightweight signature and verification ability. Signatures may be made using regular ssh keys held on disk or stored in a ssh-agent and verified against an authorized_keys-like list of allowed keys. Signatures embed a namespace that prevents confusion and attacks between different usage domains (e.g. files vs email). ssh-keygen(1): print key comment when extracting public key from a private key. ssh-keygen(1): accept the verbose flag when searching for host keys in known hosts (i.e. "ssh-keygen -vF host") to print the matching host's random-art signature too. All: support PKCS8 as an optional format for storage of private keys to disk. The OpenSSH native key format remains the default, but PKCS8 is a superior format to PEM if interoperability with non-OpenSSH software is required, as it may use a less insecure key derivation function than PEM's. Beastie Bits Say goodbye to the 32 CPU limit in NetBSD/aarch64 (https://twitter.com/jmcwhatever/status/1185584719183962112) vBSDcon 2019 videos (https://www.youtube.com/channel/UCvcdrOSlYOSzOzLjv_n1_GQ/videos) Browse the web in the terminal - W3M (https://www.youtube.com/watch?v=3Hfda0Tjqsg&feature=youtu.be) NetBSD 9 and GSoC (http://netbsd.org/~kamil/GSoC2019.html#slide1) BSDCan 2019 Videos (https://www.youtube.com/playlist?list=PLeF8ZihVdpFegPoAKppaDSoYmsBvpnSZv) NYC*BUG Install Fest: Nov 6th 18:45 @ Suspenders (https://www.nycbug.org/index?action=view&id=10673) FreeBSD Miniconf at linux.conf.au 2020 Call for Sessions Now Open (https://www.freebsdfoundation.org/blog/freebsd-miniconf-at-linux-conf-au-2020-call-for-sessions-now-open/) FOSDEM 2020 - BSD Devroom Call for Participation (https://people.freebsd.org/~rodrigo/fosdem20/) University of Cambridge looking for Research Assistants/Associates (https://twitter.com/ed_maste/status/1184865668317007874) Feedback/Questions Trenton - Beeping Thinkpad (http://dpaste.com/0ZEXNM6#wrap) Alex - Per user ZFS Datasets (http://dpaste.com/1K31A65#wrap) Allan’s old patch from 2015 (https://reviews.freebsd.org/D2272) Javier - FBSD 12.0 + ZFS + encryption (http://dpaste.com/1XX4NNA#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.

DragonFlyBSD vs. FreeBSD vs. Linux benchmark on Ryzen 7, JFK Presidential Library chooses TrueNAS for digital archives, FreeBSD 12.1-beta is available, cool but obscure X11 tools, vBSDcon trip report, Project Trident 12-U7 is available, a couple new Unix artifacts, and more. Headlines DragonFlyBSD 5.6 vs. FreeBSD 12 vs. Linux - Ryzen 7 3700X (https://www.phoronix.com/scan.php?page=article&item=bsd-linux-3700x) For those wondering how well FreeBSD and DragonFlyBSD are handling AMD's new Ryzen 3000 series desktop processors, here are some benchmarks on a Ryzen 7 3700X with MSI MEG X570 GODLIKE where both of these popular BSD operating systems were working out-of-the-box. For some fun mid-week benchmarking, here are those results of FreeBSD 12.0 and DragonFlyBSD 5.6.2 up against openSUSE Tumbleweed and Ubuntu 19.04. Back in July I looked at FreeBSD 12 on the Ryzen 9 3900X but at that time at least DragonFlyBSD had troubles booting on that system. When trying out the Ryzen 7 3700X + MSI GODLIKE X570 motherboard on the latest BIOS, everything "just worked" without any compatibility issues for either of these BSDs. We've been eager to see how well DragonFlyBSD is performing on these new AMD Zen 2 CPUs with DragonFlyBSD lead developer Matthew Dillon having publicly expressed being impressed by the new AMD Ryzen 3000 series CPUs. For comparison to those BSDs, Ubuntu 19.04 and openSUSE Tumbleweed were tested on the same hardware in their out-of-the-box configurations. While Clear Linux is normally the fastest, on this system Clear's power management defaults had caused issues in being unable to detect the Samsung 970 EVO Plus NVMe SSD used for testing and so we left it out this round. All of the hardware was the same throughout testing as were the BIOS settings and running the Ryzen 7 3700X at stock speeds. (Any differences in the reported hardware for the system table just come down to differences in what is exposed by each OS for reporting.) All of the BSD/Linux benchmarks on this eight core / sixteen thread processor were run via the Phoronix Test Suite. In the case of FreeBSD 12.0, we benchmarked both with its default LLVM Clang 6.0 compiler as well as with GCC 9.1 so that it would match the GCC compiler being the default on the other operating systems under test. JFK Presidential Library Chooses iXsystems TrueNAS to Preserve Precious Digital Archives (https://www.ixsystems.com/blog/jfk-presidential-library-pr/) iXsystems is honored to have the TrueNAS® M-Series unified storage selected to store, serve, and protect the entire digital archive for the John F. Kennedy Library Foundation. This is in support of the collection at the John F. Kennedy Presidential Library and Museum (JFK Library). Over the next several years, the Foundation hopes to grow the digital collection from hundreds of terabytes today to cover much more of the Archives at the Kennedy Library. Overall there is a total of 25 million documents, audio recordings, photos, and videos once the project is complete. Having first deployed the TrueNAS M50-HA earlier in 2019, the JFK Library has now completed the migration of its existing digital collection and is now in the process of digitizing much of the rest of its vast collection. Not only is the catalog of material vast, it is also diverse, with files being copied to the storage system from a variety of sources in numerous file types. To achieve this ambitious goal, the library required a high-end NAS system capable of sharing with a variety of systems throughout the digitization process. The digital archive will be served from the TrueNAS M50 and made available to both in-person and online visitors. With precious material and information comes robust demands. The highly-available TrueNAS M-Series has multiple layers of protection to help keep data safe, including data scrubs, checksums, unlimited snapshots, replication, and more. TrueNAS is also inherently scalable with data shares only limited by the number of drives connected to the pool. Perfect for archival storage, the deployed TrueNAS M50 will grow with the library’s content, easily expanding its storage capacity over time as needed. Supporting a variety of protocols, multi-petabyte scalability in a single share, and anytime, uninterrupted capacity expansion, the TrueNAS M-Series ticked all the right boxes. Youtube Video (https://www.youtube.com/watch?v=8rFjH5-0Fiw) News Roundup FreeBSD 12.1-beta available (https://www.phoronix.com/scan.php?page=news_item&px=FreeBSD-12.1-Beta-Released) FreeBSD 12.0 is already approaching one year old while FreeBSD 12.1 is now on the way as the next installment with various bug/security fixes and other alterations to this BSD operating system. FreeBSD 12.1 has many security/bug fixes throughout, no longer enables "-Werror" by default as a compiler flag (Update: This change is just for the GCC 4.2 compiler), has imported BearSSL into the FreeBSD base system as a lightweight TLS/SSL implementation, bzip2recover has been added, and a variety of mostly lower-level changes. More details can be found via the in-progress release notes. For those with time to test this weekend, FreeBSD 12.1 Beta 1 is available for all prominent architectures. The FreeBSD release team is planning for at least another beta or two and around three release candidates. If all goes well, FreeBSD 12.1 will be out in early November. Announcement Link (https://lists.freebsd.org/pipermail/freebsd-stable/2019-September/091533.html) Cool, but obscure X11 tools. More suggestions in the source link (https://cyber.dabamos.de/unix/x11/) ASClock Free42 FSV2 GLXGears GMixer GVIM Micropolis Sunclock Ted TiEmu X026 X48 XAbacus XAntfarm XArchiver XASCII XBiff XBill XBoard XCalc XCalendar XCHM XChomp XClipboard XClock XClock/Cat Clock XColorSel XConsole XDiary XEarth XEdit Xev XEyes XFontSel XGalaga XInvaders 3D XKill XLennart XLoad XLock XLogo XMahjongg XMan XMessage XmGrace XMixer XmMix XMore XMosaic XMOTD XMountains XNeko XOdometer XOSView Xplore XPostIt XRoach XScreenSaver XSnow XSpread XTerm XTide Xv Xvkbd XWPE XZoom vBSDCon 2019 trip report from iXSystems (https://www.ixsystems.com/blog/vbsdcon-2019/) The fourth biennial vBSDCon was held in Reston, VA on September 5th through 7th and attracted attendees and presenters from not only the Washington, DC area, but also Canada, Germany, Kenya, and beyond. While MeetBSD caters to Silicon Valley BSD enthusiasts on even years, vBSDcon caters to East Coast and DC area enthusiasts on odd years. Verisign was again the key sponsor of vBSDcon 2019 but this year made a conscious effort to entrust the organization of the event to a team of community members led by Dan Langille, who you probably know as the lead BSDCan organizer. The result of this shift was a low key but professional event that fostered great conversation and brainstorming at every turn. Project Trident 12-U7 now available (https://project-trident.org/post/2019-09-21_stable12-u7_available/) Package Summary New Packages: 130 Deleted Packages: 72 Updated Packages: 865 Stable ISO - https://pkg.project-trident.org/iso/stable/Trident-x64-TOS-12-U7-20190920.iso A Couple new Unix Artifacts (https://minnie.tuhs.org//pipermail/tuhs/2019-September/018685.html) I fear we're drifting a bit here and the S/N ratio is dropping a bit w.r.t the actual history of Unix. Please no more on the relative merits of version control systems or alternative text processing systems. So I'll try to distract you by saying this. I'm sitting on two artifacts that have recently been given to me: by two large organisations of great significance to Unix history who want me to keep "mum" about them as they are going to make announcements about them soon* and I am going slowly crazy as I wait for them to be offically released. Now you have a new topic to talk about :-) Cheers, Warren * for some definition of "soon" Beastie Bits NetBSD machines at Open Source Conference 2019 Hiroshima (https://mail-index.netbsd.org/netbsd-advocacy/2019/09/16/msg000813.html) Hyperbola a GNU/Linux OS is using OpenBSD's Xenocara (https://www.hyperbola.info/news/end-of-xorg-support/) Talos is looking for a FreeBSD Engineer (https://www.talosintelligence.com/careers/freebsd_engineer) GitHub - dylanaraps/pure-sh-bible: A collection of pure POSIX sh alternatives to external processes. (https://github.com/dylanaraps/pure-sh-bible) dsynth: you’re building it (https://www.dragonflydigest.com/2019/09/23/23523.html) Percy Ludgate, the missing link between Babbage’s machine and everything else (http://lists.sigcis.org/pipermail/members-sigcis.org/2019-September/001606.html) Feedback/Questions Bruce - Down the expect rabbithole (http://dpaste.com/147HGP3#wrap) Bruce - Expect (update) (http://dpaste.com/37MNVSW#wrap) David - Netgraph answer (http://dpaste.com/2SE1YSE) Mason - Beeps? (http://dpaste.com/00KKXJM) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.

DragonFlyBSD vs. FreeBSD vs. Linux benchmark on Ryzen 7, JFK Presidential Library chooses TrueNAS for digital archives, FreeBSD 12.1-beta is available, cool but obscure X11 tools, vBSDcon trip report, Project Trident 12-U7 is available, a couple new Unix artifacts, and more.

DragonFlyBSD vs. FreeBSD vs. Linux benchmark on Ryzen 7, JFK Presidential Library chooses TrueNAS for digital archives, FreeBSD 12.1-beta is available, cool but obscure X11 tools, vBSDcon trip report, Project Trident 12-U7 is available, a couple new Unix artifacts, and more.

vBSDcon 2019 recap, Unix at 50, OpenBSD on fan-less Tuxedo InfinityBook, humungus - an hg server, how to configure a network dump in FreeBSD, and more. Headlines vBSDcon Recap Allan and Benedict attended vBSDcon 2019, which ended last week. It was held again at the Hyatt Regency Reston and the main conference was organized by Dan Langille of BSDCan fame.The two day conference was preceded by a one day FreeBSD hackathon, where FreeBSD developers had the chance to work on patches and PRs. In the evening, a reception was held to welcome attendees and give them a chance to chat and get to know each other over food and drinks. The first day of the conference was opened with a Keynote by Paul Vixie about DNS over HTTPS (DoH). He explained how we got to the current state and what challenges (technical and social) this entails. If you missed this talk and are dying to see it, it will also be presented at EuroBSDCon next week John Baldwin followed up by giving an overview of the work on “In-Kernel TLS Framing and Encryption for FreeBSD” abstract (https://www.vbsdcon.com/schedule/2019-09-06.html#talk:132615) and the recent commit we covered in episode 313. Meanwhile, Brian Callahan was giving a separate session in another room about “Learning to (Open)BSD through its porting system: an attendee-driven educational session” where people had the chance to learn about how to create ports for the BSDs. David Fullard’s talk about “Transitioning from FreeNAS to FreeBSD” was his first talk at a BSD conference and described how he built his own home NAS setup trying to replicate FreeNAS’ functionality on FreeBSD, and why he transitioned from using an appliance to using vanilla FreeBSD. Shawn Webb followed with his overview talk about the “State of the Hardened Union”. Benedict’s talk about “Replacing an Oracle Server with FreeBSD, OpenZFS, and PostgreSQL” was well received as people are interested in how we liberated ourselves from the clutches of Oracle without compromising functionality. Entertaining and educational at the same time, Michael W. Lucas talk about “Twenty Years in Jail: FreeBSD Jails, Then and Now” closed the first day. Lucas also had a table in the hallway with his various tech and non-tech books for sale. People formed small groups and went into town for dinner. Some returned later that night to some work in the hacker lounge or talk amongst fellow BSD enthusiasts. Colin Percival was the keynote speaker for the second day and had an in-depth look at “23 years of software side channel attacks”. Allan reprised his “ELI5: ZFS Caching” talk explaining how the ZFS adaptive replacement cache (ARC) work and how it can be tuned for various workloads. “By the numbers: ZFS Performance Results from Six Operating Systems and Their Derivatives” by Michael Dexter followed with his approach to benchmarking OpenZFS on various platforms. Conor Beh was also a new speaker to vBSDcon. His talk was about “FreeBSD at Work: Building Network and Storage Infrastructure with pfSense and FreeNAS”. Two OpenBSD talks closed the talk session: Kurt Mosiejczuk with “Care and Feeding of OpenBSD Porters” and Aaron Poffenberger with “Road Warrior Disaster Recovery: Secure, Synchronized, and Backed-up”. A dinner and reception was enjoyed by the attendees and gave more time to discuss the talks given and other things until late at night. We want to thank the vBSDcon organizers and especially Dan Langille for running such a great conference. We are grateful to Verisign as the main sponsor and The FreeBSD Foundation for sponsoring the tote bags. Thanks to all the speakers and attendees! humungus - an hg server (https://humungus.tedunangst.com/r/humungus) Features View changes, files, changesets, etc. Some syntax highlighting. Read only. Serves multiple repositories. Allows cloning via the obvious URL. Supports go get. Serves files for downloads. Online documentation via mandoc. Terminal based admin interface. News Roundup OpenBSD on fan-less Tuxedo InfinityBook 14″ v2. (https://hazardous.org/archive/blog/openbsd/2019/09/02/OpenBSD-on-Infinitybook14) The InfinityBook 14” v2 is a fanless 14” notebook. It is an excellent choice for running OpenBSD - but order it with the supported wireless card (see below.). I’ve set it up in a dual-boot configuration so that I can switch between Linux and OpenBSD - mainly to spot differences in the drivers. TUXEDO allows a variety of configurations through their webshop. The dual boot setup with grub2 and EFI boot will be covered in a separate blogpost. My tests were done with OpenBSD-current - which is as of writing flagged as 6.6-beta. See Article for breakdown of CPU, Wireless, Video, Webcam, Audio, ACPI, Battery, Touchpad, and MicroSD Card Reader Unix at 50: How the OS that powered smartphones started from failure (https://arstechnica.com/gadgets/2019/08/unix-at-50-it-starts-with-a-mainframe-a-gator-and-three-dedicated-researchers/) Maybe its pervasiveness has long obscured its origins. But Unix, the operating system that in one derivative or another powers nearly all smartphones sold worldwide, was born 50 years ago from the failure of an ambitious project that involved titans like Bell Labs, GE, and MIT. Largely the brainchild of a few programmers at Bell Labs, the unlikely story of Unix begins with a meeting on the top floor of an otherwise unremarkable annex at the sprawling Bell Labs complex in Murray Hill, New Jersey. It was a bright, cold Monday, the last day of March 1969, and the computer sciences department was hosting distinguished guests: Bill Baker, a Bell Labs vice president, and Ed David, the director of research. Baker was about to pull the plug on Multics (a condensed form of MULTiplexed Information and Computing Service), a software project that the computer sciences department had been working on for four years. Multics was two years overdue, way over budget, and functional only in the loosest possible understanding of the term. Trying to put the best spin possible on what was clearly an abject failure, Baker gave a speech in which he claimed that Bell Labs had accomplished everything it was trying to accomplish in Multics and that they no longer needed to work on the project. As Berk Tague, a staffer present at the meeting, later told Princeton University, “Like Vietnam, he declared victory and got out of Multics.” Within the department, this announcement was hardly unexpected. The programmers were acutely aware of the various issues with both the scope of the project and the computer they had been asked to build it for. Still, it was something to work on, and as long as Bell Labs was working on Multics, they would also have a $7 million mainframe computer to play around with in their spare time. Dennis Ritchie, one of the programmers working on Multics, later said they all felt some stake in the success of the project, even though they knew the odds of that success were exceedingly remote. Cancellation of Multics meant the end of the only project that the programmers in the Computer science department had to work on—and it also meant the loss of the only computer in the Computer science department. After the GE 645 mainframe was taken apart and hauled off, the computer science department’s resources were reduced to little more than office supplies and a few terminals. Some of Allan’s favourite excerpts: In the early '60s, Bill Ninke, a researcher in acoustics, had demonstrated a rudimentary graphical user interface with a DEC PDP-7 minicomputer. Acoustics still had that computer, but they weren’t using it and had stuck it somewhere out of the way up on the sixth floor. And so Thompson, an indefatigable explorer of the labs’ nooks and crannies, finally found that PDP-7 shortly after Davis and Baker cancelled Multics. With the rest of the team’s help, Thompson bundled up the various pieces of the PDP-7—a machine about the size of a refrigerator, not counting the terminal—moved it into a closet assigned to the acoustics department, and got it up and running. One way or another, they convinced acoustics to provide space for the computer and also to pay for the not infrequent repairs to it out of that department’s budget. McIlroy’s programmers suddenly had a computer, kind of. So during the summer of 1969, Thompson, Ritchie, and Canaday hashed out the basics of a file manager that would run on the PDP-7. This was no simple task. Batch computing—running programs one after the other—rarely required that a computer be able to permanently store information, and many mainframes did not have any permanent storage device (whether a tape or a hard disk) attached to them. But the time-sharing environment that these programmers had fallen in love with required attached storage. And with multiple users connected to the same computer at the same time, the file manager had to be written well enough to keep one user’s files from being written over another user’s. When a file was read, the output from that file had to be sent to the user that was opening it. It was a challenge that McIlroy’s team was willing to accept. They had seen the future of computing and wanted to explore it. They knew that Multics was a dead-end, but they had discovered the possibilities opened up by shared development, shared access, and real-time computing. Twenty years later, Ritchie characterized it for Princeton as such: “What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form.” Eventually when they had the file management system more or less fleshed out conceptually, it came time to actually write the code. The trio—all of whom had terrible handwriting—decided to use the Labs’ dictating service. One of them called up a lab extension and dictated the entire code base into a tape recorder. And thus, some unidentified clerical worker or workers soon had the unenviable task of trying to convert that into a typewritten document. Of course, it was done imperfectly. Among various errors, “inode” came back as “eye node,” but the output was still viewed as a decided improvement over their assorted scribbles. In August 1969, Thompson’s wife and son went on a three-week vacation to see her family out in Berkeley, and Thompson decided to spend that time writing an assembler, a file editor, and a kernel to manage the PDP-7 processor. This would turn the group’s file manager into a full-fledged operating system. He generously allocated himself one week for each task. Thompson finished his tasks more or less on schedule. And by September, the computer science department at Bell Labs had an operating system running on a PDP-7—and it wasn’t Multics. By the summer of 1970, the team had attached a tape drive to the PDP-7, and their blossoming OS also had a growing selection of tools for programmers (several of which persist down to this day). But despite the successes, Thompson, Canaday, and Ritchie were still being rebuffed by labs management in their efforts to get a brand-new computer. It wasn’t until late 1971 that the computer science department got a truly modern computer. The Unix team had developed several tools designed to automatically format text files for printing over the past year or so. They had done so to simplify the production of documentation for their pet project, but their tools had escaped and were being used by several researchers elsewhere on the top floor. At the same time, the legal department was prepared to spend a fortune on a mainframe program called “AstroText.” Catching wind of this, the Unix crew realized that they could, with only a little effort, upgrade the tools they had written for their own use into something that the legal department could use to prepare patent applications. The computer science department pitched lab management on the purchase of a DEC PDP-11 for document production purposes, and Max Mathews offered to pay for the machine out of the acoustics department budget. Finally, management gave in and purchased a computer for the Unix team to play with. Eventually, word leaked out about this operating system, and businesses and institutions with PDP-11s began contacting Bell Labs about their new operating system. The Labs made it available for free—requesting only the cost of postage and media from anyone who wanted a copy. The rest has quite literally made tech history. See the link for the rest of the article How to configure a network dump in FreeBSD? (https://www.oshogbo.vexillium.org/blog/68/) A network dump might be very useful for collecting kernel crash dumps from embedded machines and machines with a larger amount of RAM then available swap partition size. Besides net dumps we can also try to compress the core dump. However, often this may still not be enough swap to keep whole core dump. In such situation using network dump is a convenient and reliable way for collecting kernel dump. So, first, let’s talk a little bit about history. The first implementation of the network dumps was implemented around 2000 for the FreeBSD 4.x as a kernel module. The code was implemented in 2010 with the intention of being part of FreeBSD 9.0. However, the code never landed in FreeBSD. Finally, in 2018 with the commit r333283 by Mark Johnston the netdump client code landed in the FreeBSD. Subsequently, many other commitments were then implemented to add support for the different drivers (for example r333289). The first official release of FreeBSD, which support netdump is FreeBSD 12.0. Now, let’s get back to the main topic. How to configure the network dump? Two machines are needed. One machine is to collect core dump, let’s call it server. We will use the second one to send us the core dump - the client. See the link for the rest of the article Beastie Bits Sudo Mastery 2nd edition is not out (https://mwl.io/archives/4530) Empirical Notes on the Interaction Between Continuous Kernel Fuzzing and Development (http://users.utu.fi/kakrind/publications/19/vulnfuzz_camera.pdf) soso (https://github.com/ozkl/soso) GregKH - OpenBSD was right (https://youtu.be/gUqcMs0svNU?t=254) Game of Trees (https://gameoftrees.org/faq.html) Feedback/Questions BostJan - Another Question (http://dpaste.com/1ZPCCQY#wrap) Tom - PF (http://dpaste.com/3ZSCB8N#wrap) JohnnyK - Changing VT without keys (http://dpaste.com/3QZQ7Q5#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.

vBSDcon 2019 recap, Unix at 50, OpenBSD on fan-less Tuxedo InfinityBook, humungus - an hg server, how to configure a network dump in FreeBSD, and more.

vBSDcon 2019 recap, Unix at 50, OpenBSD on fan-less Tuxedo InfinityBook, humungus - an hg server, how to configure a network dump in FreeBSD, and more.

OpenBSD on 7th gen Thinkpad X1 Carbon, how to install FreeBSD on a MacBook, Kernel portion of in-kernel TLS (KTLS), Boot Environments on DragonflyBSD, Project Trident Updates, vBSDcon schedule, and more.

OpenBSD on 7th gen Thinkpad X1 Carbon, how to install FreeBSD on a MacBook, Kernel portion of in-kernel TLS (KTLS), Boot Environments on DragonflyBSD, Project Trident Updates, vBSDcon schedule, and more. Headlines OpenBSD on the Thinkpad X1 Carbon 7th Gen (https://jcs.org/2019/08/14/x1c7) Another year, another ThinkPad X1 Carbon, this time with a Dolby Atmos sound system and a smaller battery. The seventh generation X1 Carbon isn't much different than the fifth and sixth generations. I opted for the non-vPro Core i5-8265U, 16Gb of RAM, a 512Gb NVMe SSD, and a matte non-touch WQHD display at ~300 nits. A brighter 500-nit 4k display is available, though early reports indicated it severely impacts battery life. Gone are the microSD card slot on the back and 1mm of overall thickness (from 15.95mm to 14.95mm), but also 6Whr of battery (down to 51Whr) and a little bit of travel in the keyboard and TrackPoint buttons. I still very much like the feel of both of them, so kudos to Lenovo for not going too far down the Apple route of sacrificing performance and usability just for a thinner profile. On my fifth generation X1 Carbon, I used a vinyl plotter to cut out stickers to cover the webcam, "X1 Carbon" branding from the bottom of the display, the power button LED, and the "ThinkPad" branding from the lower part of the keyboard deck. See link for the rest of the article How To Install FreeBSD On A MacBook 1,1 or 2,1 (http://lexploit.com/freebsdmacbook1-1-2-1/) FreeBSD Setup For MacBook 1,1 and 2,1 FreeBSD with some additional setup can be installed on a MacBook 1,1 or 2,1. This article covers how to do so with FreeBSD 10-12. Installing FreeBSD can be installed as the only OS on your MacBook if desired. What you should have is: A Mac OS X 10.4.6-10.7.5 installer. Unofficial versions modified for these MacBooks such as 10.8 also work. A blank CD or DVD to burn the FreeBSD image to. Discs simply work best with these older MacBooks. An ISO file of FreeBSD for x86. The AMD64 ISO does not boot due to the 32 bit EFI of these MacBooks. Burn the ISO file to the blank CD or DVD. Once done, make sure it's in your MacBook and then power off the MacBook. Turn it on, and hold down the c key until the FreeBSD disc boots. See link for the rest of the guide News Roundup Patch for review: Kernel portion of in-kernel TLS (KTLS) (https://svnweb.freebsd.org/base?view=revision&revision=351522) One of the projects I have been working on for the past several months in conjunction with several other folks is upstreaming work from Netflix to handle some aspects of Transport Layer Security (TLS) in the kernel. In particular, this lets a web server use sendfile() to send static content on HTTPS connections. There is a lot more detail in the review itself, so I will spare pasting a big wall of text here. However, I have posted the patch to add the kernel-side of KTLS for review at the URL below. KTLS also requires other patches to OpenSSL and nginx, but this review is only for the kernel bits. Patches and reviews for the other bits will follow later. https://reviews.freebsd.org/D21277 DragonFly Boot Enviroments (https://github.com/newnix/dfbeadm) This is a tool inspired by the beadm utility for FreeBSD/Illumos systems that creates and manages ZFS boot environments. This utility in contrast is written from the ground up in C, this should provide better performance, integration, and extensibility than the POSIX sh and awk script it was inspired by. During the time this project has been worked on, beadm has been superseded by bectl on FreeBSD. After hammering out some of the outstanding internal logic issues, I might look at providing a similar interface to the command as bectl. See link for the rest of the details Project Trident Updates 19.08 Available (https://project-trident.org/post/2019-08-15_19.08_available/) This is a general package update to the CURRENT release repository based upon TrueOS 19.08. Legacy boot ISO functional again This update includes the FreeBSD fixes for the “vesa” graphics driver for legacy-boot systems. The system can once again be installed on legacy-boot systems. PACKAGE CHANGES FROM 19.07-U1 New Packages: 154 Deleted Packages: 394 Updated Packages: 4926 12-U3 Available (https://project-trident.org/post/2019-08-22_stable12-u3_available/) This is the third general package update to the STABLE release repository based upon TrueOS 12-Stable. PACKAGE CHANGES FROM STABLE 12-U2 New Packages: 105 Deleted Packages: 386 Updated Packages: 1046 vBSDcon (https://www.vbsdcon.com/schedule/) vBSDcon 2019 will return to the Hyatt Regency in Reston, VA on September 5-7 2019. *** Beastie Bits The next NYCBUG meeting will be Sept 4 @ 18:45 (https://www.nycbug.org/index?action=view&id=10671) Feedback/Questions Tom - Questions (http://dpaste.com/1AXXK7G#wrap) Michael - dfbeadm (http://dpaste.com/0PNEDYT#wrap) Bostjan - Questions (http://dpaste.com/1N7T7BR#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.

OpenBSD on 7th gen Thinkpad X1 Carbon, how to install FreeBSD on a MacBook, Kernel portion of in-kernel TLS (KTLS), Boot Environments on DragonflyBSD, Project Trident Updates, vBSDcon schedule, and more.

The UNIX Philosophy in 2019, why use package managers, touchpad interrupted, Porting wine to amd64 on NetBSD second evaluation report, Enhancing Syzkaller Support for NetBSD, all about the Pinebook Pro, killing a process and all of its descendants, fast software the best software, and more. Headlines The UNIX Philosophy in 2019 (https://triosdevelopers.com/jason.eckert/blog/Entries/2019/6/1_Entry_1.html) Today, Linux and open source rules the world, and the UNIX philosophy is widely considered compulsory. Organizations are striving to build small, focused applications that work collaboratively in a cloud and microservices environment. We rely on the network, as well as HTTP (text) APIs for storing and referencing data. Moreover, nearly all configuration is stored and communicated using text (e.g. YAML, JSON or XML). And while the UNIX philosophy has changed dramatically over the past 5 decades, it hasn’t strayed too far from Ken Thompson’s original definition in 1973: We write programs that do one thing and do it well We write programs to work together And we write programs that handle text streams, because that is a universal interface Why Use Package Managers? (https://uwm.edu/hpc/software-management/) Valuable research is often hindered or outright prevented by the inability to install software. This need not be the case. Since I began supporting research computing in 1999, I’ve frequently seen researchers struggle for days or weeks trying to install a single open source application. In most cases, they ultimately failed. In many cases, they could have easily installed the software in seconds with one simple command, using a package manager such as Debian packages, FreeBSD ports, MacPorts, or Pkgsrc, just to name a few. Developer websites often contain poorly written instructions for doing “caveman installs”; manually downloading, unpacking, patching, and building the software. The same laborious process must often be followed for other software packages on which it depends, which can sometimes number in the dozens. Many researchers are simply unaware that there are easier ways to install the software they need. Caveman installs are a colossal waste of man-hours. If 1000 people around the globe spend an average of 20 hours each trying to install the same program that could have been installed with a package manager (this is not uncommon), then 20,000 man-hours have been lost that could have gone toward science. How many important discoveries are delayed by this? The elite research institutions have ample funding and dozens of IT staff dedicated to research computing. They can churn out publications even if their operation is inefficient. Most institutions, however, have few or no IT staff dedicated to research, and cannot afford to squander precious man-hours on temporary, one-off software installs. The wise approach for those of us in that situation is to collaborate on making software deployment easier for everyone. If we do so, then even the smallest research groups can leverage that work to be more productive and make more frequent contributions to science. Fortunately, the vast majority of open source software installs can be made trivial for anyone to do for themselves. Modern package managers perform all the same steps as a caveman install, but automatically. Package managers also install dependencies for us automatically. News Roundup Touchpad, Interrupted (https://jcs.org/2019/07/28/ihidev) For two years I've been driving myself crazy trying to figure out the source of a driver problem on OpenBSD: interrupts never arrived for certain touchpad devices. A couple weeks ago, I put out a public plea asking for help in case any non-OpenBSD developers recognized the problem, but while debugging an unrelated issue over the weekend, I finally solved it. It's been a long journey and it's a technical tale, but here it is. Porting wine to amd64 on NetBSD, second evaluation report (https://blog.netbsd.org/tnf/entry/porting_wine_to_amd64_on2) Summary Presently, Wine on amd64 is in test phase. It seems to work fine with caveats like LDLIBRARYPATH which has to be set as 32-bit Xorg libs don't have ${PREFIX}/emul/netbsd32/lib in its rpath section. The latter is due to us extracting 32-bit libs from tarballs in lieu of building 32-bit Xorg on amd64. As previously stated, pkgsrc doesn't search for pkgconfig files in ${PREFIX}/emul/netbsd32/lib which might have inadvertent effects that I am unaware of as of now. I shall be working on these issues during the final coding period. I would like to thank @leot, @maya and @christos for saving me from shooting myself in the foot many a time. I, admittedly, have had times when multiple approaches, which all seemed right at that time, perplexed me. I believe those are times when having a mentor counts, and I have been lucky enough to have really good ones. Once again, thanks to Google for this wonderful opportunity. Enhancing Syzkaller Support for NetBSD, Part 2 (https://blog.netbsd.org/tnf/entry/enchancing_syzkaller_support_for_netbsd) As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating systems including NetBSD. This report details the work done during the second coding period. You can also take a look at the first report to learn more about the initial support that we added. : https://blog.netbsd.org/tnf/entry/enhancingsyzkallersupportfornetbsd July Update: All about the Pinebook Pro (https://www.pine64.org/2019/07/05/july-update-all-about-the-pinebook-pro/) "So I said I won’t be talking about the BSDs, but I feel like I should at the very least give you a general overview of the RK3399 *BSD functionality. I’ll make it quick. I’ve spoken to *BSD devs whom worked on the RockPro64 and from what I’ve gathered (despite the different *BSDs having varying degree of support for the RK3399 SOC) many of the core features are already supported, which bodes well for *BSD on the Pro. That said, some of the things you’d require on a functional laptop – such as the LCD (using eDP) for instance – will not work on the Pinebook Pro using *BSD as of today. So clearly a degree of work is yet needed for a BSD to run on the device. However, keep in mind that *BSD developers will be receiving their units soon and by the time you receive yours some basic functionality may be available." Killing a process and all of its descendants (http://morningcoffee.io/killing-a-process-and-all-of-its-descendants.html) Killing processes in a Unix-like system can be trickier than expected. Last week I was debugging an odd issue related to job stopping on Semaphore. More specifically, an issue related to the killing of a running process in a job. Here are the highlights of what I learned: Unix-like operating systems have sophisticated process relationships. Parent-child, process groups, sessions, and session leaders. However, the details are not uniform across operating systems like Linux and macOS. POSIX compliant operating systems support sending signals to process groups with a negative PID number. Sending signals to all processes in a session is not trivial with syscalls. Child processes started with exec inherit their parent signal configuration. If the parent process is ignoring the SIGHUP signal, for example, this configuration is propagated to the children. The answer to the “What happens with orphaned process groups” question is not trivial. Fast Software, the Best Software (https://craigmod.com/essays/fast_software/) I love fast software. That is, software speedy both in function and interface. Software with minimal to no lag between wanting to activate or manipulate something and the thing happening. Lightness. Software that’s speedy usually means it’s focused. Like a good tool, it often means that it’s simple, but that’s not necessarily true. Speed in software is probably the most valuable, least valued asset. To me, speedy software is the difference between an application smoothly integrating into your life, and one called upon with great reluctance. Fastness in software is like great margins in a book — makes you smile without necessarily knowing why. But why is slow bad? Fast software is not always good software, but slow software is rarely able to rise to greatness. Fast software gives the user a chance to “meld” with its toolset. That is, not break flow. When the nerds upon Nerd Hill fight to the death over Vi and Emacs, it’s partly because they have such a strong affinity for the flow of the application and its meldiness. They have invested. The Tool Is Good, so they feel. Not breaking flow is an axiom of great tools. A typewriter is an excellent tool because, even though it’s slow in a relative sense, every aspect of the machine itself operates as quickly as the user can move. It is focused. There are no delays when making a new line or slamming a key into the paper. Yes, you have to put a new sheet of paper into the machine at the end of a page, but that action becomes part of the flow of using the machine, and the accumulation of paper a visual indication of work completed. It is not wasted work. There are no fundamental mechanical delays in using the machine. The best software inches ever closer to the physical directness of something like a typewriter. (The machine may break down, of course, ribbons need to be changed — but this is maintenance and separate from the use of the tool. I’d be delighted to “maintain” Photoshop if it would lighten it up.) Beastie Bits Register for vBSDCon 2019, Sept 5-7 in Reston VA (https://vbsdcon.com/registration) Register for EuroBSDCon 2019, Sept 19-22 in Lillehammer, Norway (https://2019.eurobsdcon.org/registration/) Feedback/Questions Paulo - FreeNAS Question (http://dpaste.com/2GDG7WR#wrap) Marc - Changing VT without function keys? (http://dpaste.com/1AKC7A1#wrap) Caleb - Patch, update, and upgrade management (http://dpaste.com/2D6J482#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.

NetBSD 9.0 release process has started, xargs, a tale of two spellcheckers, Adapting TriforceAFL for NetBSD, Exploiting a no-name freebsd kernel vulnerability, and more. Headlines NetBSD 9.0 release process has started (https://mail-index.netbsd.org/netbsd-announce/2019/07/31/msg000301.html) If you have been following source-changes, you may have noticed the creation of the netbsd-9 branch! It has some really exciting items that we worked on: + New AArch64 architecture support: + Symmetric and asymmetrical multiprocessing support (aka big.LITTLE) + Support for running 32-bit binaries + UEFI and ACPI support + Support for SBSA/SBBR (server-class) hardware. + The FDT-ization of many ARM boards: + the 32-bit GENERIC kernel lists 129 different DTS configurations + the 64-bit GENERIC64 kernel lists 74 different DTS configurations + All supported by a single kernel, without requiring per-board configuration. + Graphics driver update, matching Linux 4.4, adding support for up to Kaby Lake based Intel graphics devices. + ZFS has been updated to a modern version and seen many bugfixes. + New hardware-accelerated virtualization via NVMM. + NPF performance improvements and bug fixes. A new lookup algorithm, thmap, is now the default. + NVMe performance improvements + Optional kernel ASLR support, and partial kernel ASLR for the default configuration. + Kernel sanitizers: + KLEAK, detecting memory leaks + KASAN, detecting memory overruns + KUBSAN, detecting undefined behaviour + These have been used together with continuous fuzzing via the syzkaller project to find many bugs that were fixed. + The removal of outdated networking components such as ISDN and all of its drivers + The installer is now capable of performing GPT UEFI installations. + Dramatically improved support for userland sanitizers, as well as the option to build all of NetBSD's userland using them for bug-finding. + Update to graphics userland: Mesa was updated to 18.3.4, and llvmpipe is now available for several architectures, providing 3D graphics even in the absence of a supported GPU. We try to test NetBSD as best as we can, but your testing can help NetBSD 9.0 a great release. Please test it and let us know of any bugs you find. + Binaries are available at https://nycdn.netbsd.org/pub/NetBSD-daily/netbsd-9/latest/ xargs wtf (https://medium.com/@aarontharris/xargs-wtf-34d2618286b7) xargs is probably one of the more difficult to understand of the unix command arsenal and of course that just means it’s one of the most useful too. I discovered a handy trick that I thought was worth a share. Please note there are probably other (better) ways to do this but I did my stackoverflow research and found nothing better. xargs — at least how I’ve most utilized it — is handy for taking some number of lines as input and doing some work per line. It’s hard to be more specific than that as it does so much else. It literally took me an hour of piecing together random man pages + tips from 11 year olds on stack overflow, but eventually I produced this gem: This is an example of how to find files matching a certain pattern and rename each of them. It sounds so trivial (and it is) but it demonstrates some cool tricks in an easy concept. News Roundup PkgSrc: A Tale of Two Spellcheckers (https://bentsukun.ch/posts/pkgsrccon-2019/) This is a transcript of the talk I gave at pkgsrcCon 2019 in Cambridge, UK. It is about spellcheckers, but there are much more general software engineering lessons that we can learn from this case study. The reason I got into this subject at all was my paternal leave last year, when I finally had some more time to spend working on pkgsrc. It was a tiny item in the enormous TODO file at the top of the source tree (“update enchant to version 2.2”) that made me go into this rabbit hole. Adapting TriforceAFL for NetBSD, Part 2 (https://blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part1) I have been working on adapting TriforceAFL for NetBSD kernel syscall fuzzing. This blog post summarizes the work done until the second evaluation. For work done during the first coding period, check out this post. Summary > So far, the TriforceNetBSDSyscallFuzzer has been made available in the form of a pkgsrc package with the ability to fuzz most of NetBSD syscalls. In the final coding period of GSoC. I plan to analyse the crashes that were found until now. Integrate sanitizers, try and find more bugs and finally wrap up neatly with detailed documentation. > Last but not least, I would like to thank my mentor, Kamil Rytarowski for helping me through the process and guiding me. It has been a wonderful learning experience so far! Exploiting a no-name freebsd kernel vulnerability (https://www.synacktiv.com/posts/exploit/exploiting-a-no-name-freebsd-kernel-vulnerability.html) A new patch has been recently shipped in FreeBSD kernels to fix a vulnerability (cve-2019-5602) present in the cdrom device. In this post, we will introduce the bug and discuss its exploitation on pre/post-SMEP FreeBSD revisions. > A closer look at the commit 6bcf6e3 shows that when invoking the CDIOCREADSUBCHANNEL_SYSSPACE ioctl, data are copied with bcopy instead of the copyout primitive. This endows a local attacker belonging to the operator group with an arbitrary write primitive in the kernel memory. [Allan and Benedicts Conference Gear Breakdown] Benedict’s Gear: GlocalMe G3 Mobile Travel HotSpot and Powerbank (https://www.glocalme.com/CA/en-US/cloudsim/g3) Mogics Power Bagel (http://www.mogics.com/3824-2) Charby Sense Power Cable (https://charbycharge.com/charby-sense-worlds-smartest-auto-cutoff-cable/) Allan’s Gear: Huawei E5770s-320 4G LTE 150 Mbps Mobile WiFi Pro (https://smile.amazon.com/gp/product/B013CEGGKI/) AOW Global Data SIM Card for On-Demand 4G LTE Mobile Data in Over 90 Countries (https://smile.amazon.com/dp/B071HJFX27/) All my devices charge from USB-C, so that is great More USB thumb drives than strictly necessary My Lenovo X270 laptop running FreeBSD 13-current My 2016 Macbook Pro (a prize from the raffle at vBSDCon 2017) that I use for email and video conferencing to preserve battery on my FreeBSD machine for work Beastie Bits Replacing the Unix tradition (Warning may be rage inducing) (https://www.youtube.com/watch?v=L9v4Mg8wi4U&feature=youtu.be) Installing OpenBSD over remote serial on the AtomicPI (https://www.thanassis.space/remoteserial.html#remoteserial) Zen 2 and DragonFly (https://www.dragonflydigest.com/2019/08/05/23294.html) Improve Docking on FreeBSD (https://blog.yukiisbo.red/posts/2019/05/improve-docking-on-freebsd/) Register for vBSDCon 2019, Sept 5-7 in Reston VA. Early bird ends August 15th. (https://vbsdcon.com/registration) Register for EuroBSDCon 2019, Sept 19-22 in Lillehammer, Norway (https://2019.eurobsdcon.org/registration/) Feedback/Questions JT - Congrats (http://dpaste.com/0D7Y31E#wrap) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.

DragonFlyBSD Project Update - colo upgrade, future trends, resuming ZFS send, realtime bandwidth terminal graph visualization, fixing telnet fixes, a chapter from the FBI’s history with OpenBSD and an OpenSSH vuln, and more. Headlines DragonFlyBSD Project Update - colo upgrade, future trends (http://lists.dragonflybsd.org/pipermail/users/2019-July/358226.html) For the last week I've been testing out a replacement for Monster, our 48-core opteron server. The project will be removing Monster from the colo in a week or two and replacing it with three machines which together will use half the power that Monster did alone. The goal is to clear out a little power budget in the colo and to really beef-up our package-building capabilities to reduce the turn-around time needed to test ports syncs and updates to the binary package system. Currently we use two blades to do most of the building, plus monster sometimes. The blades take almost a week (120 hours+) to do a full synth run and monster takes around 27.5 hours. But we need to do three bulk builds more or less at the same time... one for the release branch, one for the development branch, and one for staging updates. It just takes too long and its been gnawing at me for a little while. Well, Zen 2 to the rescue! These new CPUs can take ECC, there's actually an IPMI mobo available, and they are fast as hell and cheap for what we get. The new machines will be two 3900X based servers, plus a dual-xeon system that I already had at home. The 3900X's can each do a full synth run in 24.5 hours and the Xeon can do it in around 31 hours. Monster will be retired. And the crazy thing about this? Monster burns 1000W going full bore. Each of the 3900X servers burns 160W and the Xeon burns 200W. In otherwords, we are replacing 1000W with only 520W and getting roughly 6x the performance efficiency in the upgrade. This tell you just how much more power-efficient machines have become in the last 9 years or so. > This upgrade will allow us to do full builds for both release and dev in roughly one day instead of seven days, and do it without interfering with staging work that might be happening at the same time. Future trends - DragonFlyBSD has reached a bit of a cross-roads. With most of the SMP work now essentially complete across the entire system the main project focus is now on supplying reliable binary ports for release and developer branches, DRM (GPU) support and other UI elements to keep DragonFlyBSD relevant on workstations, and continuing Filesystem work on HAMMER2 to get multi-device and clustering going. Resuming ZFS send (https://www.oshogbo.vexillium.org/blog/66/) One of the amazing functionalities of ZFS is the possibility of sending a whole dataset from one place to another. This mechanism is amazing to create backups of your ZFS based machines. Although, there were some issues with this functionality for a long time when a user sent a big chunk of data. What if you would do that over the network and your connection has disappeared? What if your machine was rebooted as you are sending a snapshot? For a very long time, you didn't have any options - you had to send a snapshot from the beginning. Now, this limitation was already bad enough. However, another downside of this approach was that all the data which you already send was thrown away. Therefore, ZFS had to go over all this data and remove them from the dataset. Imagine the terabytes of data which you sent via the network was thrown away because as you were sending the last few bytes, the network went off. In this short post, I don't want to go over the whole ZFS snapshot infrastructure (if you think that such a post would be useful, please leave a comment). Now, to get back to the point, this infrastructure is used to clone the datasets. Some time ago a new feature called “Resuming ZFS send” was introduced. That means that if there was some problem with transmitting the dataset from one point to another you could resume it or throw them away. But the point is, that yes, you finally have a choice. News Roundup Realtime bandwidth terminal graph visualization (https://dataswamp.org/~solene/2019-07-19-ttyplot-netstat-openbsd.html) If for some reasons you want to visualize your bandwidth traffic on an interface (in or out) in a terminal with a nice graph, here is a small script to do so, involving ttyplot, a nice software making graphics in a terminal. The following will works on OpenBSD. You can install ttyplot by pkg_add ttyplot as root, ttyplot package appeared since OpenBSD 6.5. fixing telnet fixes (https://flak.tedunangst.com/post/fixing-telnet-fixes) There’s a FreeBSD commit to telnet. fix a couple of snprintf() buffer overflows. It’s received a bit of attention for various reasons, telnet in 2019?, etc. I thought I’d take a look. Here’s a few random observations. The first line is indented with spaces while the others use tabs. The correct type for string length is size_t not unsigned int. sizeof(char) is always one. There’s no need to multiply by it. If you do need to multiply by a size, this is an unsafe pattern. Use calloc or something similar. (OpenBSD provides reallocarray to avoid zeroing cost of calloc.) Return value of malloc doesn’t need to be cast. In fact, should not be, lest you disguise a warning. Return value of malloc is not checked for NULL. No reason to cast cp to char * when passing to snprintf. It already is that type. And if it weren’t, what are you doing? The whole operation could be simplified by using asprintf. Although unlikely (probably impossible here, but more generally), adding the two source lengths together can overflow, resulting in truncation with an unchecked snprintf call. asprintf avoids this failure case. A Chapter from the FBI’s History with OpenBSD and an OpenSSH Vuln (https://twitter.com/RooneyMcNibNug/status/1152327783055601664) Earlier this year I FOIAed the FBI for details on allegations of backdoor installed in the IPSEC stack in 2010, originally discussed by OpenBSD devs (https://marc.info/?l=openbsd-tech&m=129236621626462 …) Today, I got an interesting but unexpected responsive record: Freedom of Information Act: FBI: OpenBSD (https://www.muckrock.com/foi/united-states-of-america-10/foia-fbi-openbsd-70084/) GitHub Repo (https://github.com/RooneyMcNibNug/FOIA/blob/master/Responsive%20Docs/OpenBSD/FBI_OpenBSD_response_OCRd.pdf) Beastie Bits “Sudo Mastery, 2nd Edition” open for tech review (https://mwl.io/archives/4378) FreeBSD Journal: FreeBSD for Makers (https://www.freebsdnews.com/2019/07/12/freebsd-journal-freebsd-for-makers/) OpenBSD and NetBSD machines at Open Source Conference 2019 Nagoya (http://mail-index.netbsd.org/netbsd-advocacy/2019/07/19/msg000808.html) FreeBSD 12.0: WINE Gaming (https://www.youtube.com/watch?v=zuj9pRNR2oM) Introduction to the Structure and Interpretation of TNF (The NetBSD Foundation) (https://www.netbsd.org/gallery/presentations/wiz/pkgsrccon2019/index.html#/) vBSDcon speakers announced (https://www.vbsdcon.com/) Feedback/Questions Pat - NYCBug Aug 7th (http://dpaste.com/21Y1PRM) Tyler - SSH keys vs password (http://dpaste.com/3JEVVEF#wrap) Lars - Tor-Talk (http://dpaste.com/0RAFMXZ) Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) Your browser does not support the HTML5 video tag.

FreeBSD 11.2 has been released, setting up an MTA behind Tor, running pfsense on DigitalOcean, one year of C, using OpenBGPD to announce VM networks, the power to serve, and a BSDCan trip report. ##Headlines FreeBSD 11.2-RELEASE Available FreeBSD 11.2 was released today (June 27th) and is ready for download Highlights: OpenSSH has been updated to version 7.5p1. OpenSSL has been updated to version 1.0.2o. The clang, llvm, lldb and compiler-rt utilities have been updated to version 6.0.0. The libarchive(3) library has been updated to version 3.3.2. The libxo(3) library has been updated to version 0.9.0. Major Device driver updates to: cxgbe(4) – Chelsio 10/25/40/50/100 gigabit NICs – version 1.16.63.0 supports T4, T5 and T6 ixl(4) – Intel 10 and 40 gigabit NICs, updated to version 1.9.9-k ng_pppoe(4) – driver has been updated to add support for user-supplied Host-Uniq tags New drivers: + drm-next-kmod driver supporting integrated Intel graphics with the i915 driver. mlx5io(4) – a new IOCTL interface for Mellanox ConnectX-4 and ConnectX-5 10/20/25/40/50/56/100 gigabit NICs ocs_fc(4) – Emulex Fibre Channel 8/16/32 gigabit Host Adapters smartpqi(4) – HP Gen10 Smart Array Controller Family The newsyslog(8) utility has been updated to support RFC5424-compliant messages when rotating system logs The diskinfo(8) utility has been updated to include two new flags, -s which displays the disk identity (usually the serial number), and -p which displays the physical path to the disk in a storage controller. The top(1) utility has been updated to allow filtering on multiple user names when the -U flag is used The umount(8) utility has been updated to include a new flag, -N, which is used to forcefully unmount an NFS mounted filesystem. The ps(1) utility has been updated to display if a process is running with capsicum(4) capability mode, indicated by the flag ‘C’ The service(8) utility has been updated to include a new flag, -j, which is used to interact with services running within a jail(8). The argument to -j can be either the name or numeric jail ID The mlx5tool(8) utility has been added, which is used to manage Connect-X 4 and Connect-X 5 devices supported by mlx5io(4). The ifconfig(8) utility has been updated to include a random option, which when used with the ether option, generates a random MAC address for an interface. The dwatch(1) utility has been introduced The efibootmgr(8) utility has been added, which is used to manipulate the EFI boot manager. The etdump(1) utility has been added, which is used to view El Torito boot catalog information. The linux(4) ABI compatibility layer has been updated to include support for musl consumers. The fdescfs(5) filesystem has been updated to support Linux®-specific fd(4) /dev/fd and /proc/self/fd behavior Support for virtio_console(4) has been added to bhyve(4). The length of GELI passphrases entered when booting a system with encrypted disks is now hidden by default. See the configuration options in geli(8) to restore the previous behavior. In addition to the usual CD/DVD ISO, Memstick, and prebuilt VM images (raw, qcow2, vhd, and vmdk), FreeBSD 11.2 is also available on: Amazon EC2 Google Compute Engine Hashicorp/Atlas Vagrant Microsoft Azure In addition to a generic ARM64 image for devices like the Pine64 and Raspberry Pi 3, specific images are provided for: GUMSTIX BANANAPI BEAGLEBONE CUBIEBOARD CUBIEBOARD2 CUBOX-HUMMINGBOARD RASPBERRY PI 2 PANDABOARD WANDBOARD Full Release Notes ###Setting up an MTA Behind Tor This article will document how to set up OpenSMTPD behind a fully Tor-ified network. Given that Tor’s DNS resolver code does not support MX record lookups, care must be taken for setting up an MTA behind a fully Tor-ified network. OpenSMTPD was chosen because it was easy to modify to force it to fall back to A/AAAA lookups when MX lookups failed with a DNS result code of NOTIMP (4). Note that as of 08 May 2018, the OpenSMTPD project is planning a configuration file language change. The proposed change has not landed. Once it does, this article will be updated to reflect both the old language and new. The reason to use an MTA behing a fully Tor-ified network is to be able to support email behind the .onion TLD. This setup will only allow us to send and receive email to and from the .onion TLD. Requirements: A fully Tor-ified network HardenedBSD as the operating system A server (or VM) running HardenedBSD behind the fully Tor-ified network. /usr/ports is empty Or is already pre-populated with the HardenedBSD Ports tree Why use HardenedBSD? We get all the features of FreeBSD (ZFS, DTrace, bhyve, and jails) with enhanced security through exploit mitigations and system hardening. Tor has a very unique threat landscape and using a hardened ecosystem is crucial to mitigating risks and threats. Also note that this article reflects how I’ve set up my MTA. I’ve included configuration files verbatim. You will need to replace the text that refers to my .onion domain with yours. On 08 May 2018, HardenedBSD’s version of OpenSMTPD just gained support for running an MTA behind Tor. The package repositories do not yet contain the patch, so we will compile OpenSMTPD from ports. Steps Installation Generating Cryptographic Key Material Tor Configuration OpenSMTPD Configuration Dovecot Configuration Testing your configuration Optional: Webmail Access iXsystems https://www.forbes.com/sites/forbestechcouncil/2018/06/21/strings-attached-knowing-when-and-when-not-to-accept-vc-funding/#30f9f18f46ec https://www.ixsystems.com/blog/self-2018-recap/ ###Running pfSense on a Digital Ocean Droplet I love pfSense (and opnSense, no discrimination here). I use it for just about anything, from homelab to large scale deployments and I’ll give out on any fancy for a pfSense setup on a decent hardware. I also love DigitalOcean, if you ever used them, you know why, if you never did, head over and try, you’ll understand why. . Unfortunately, while DO offers tremendous amount of useful distros and applications, pfSense isn’t one of them. But, where there’s a will, there’s a way, and here’s how to get pfSense up and running on DO so you can have it as the gatekeeper to your kingdom. Start by creating a FreeBSD droplet, choose your droplet size (for modest setups, I find the 5$ to be quite awesome): There are many useful things you can do with pfSense on your droplet, from OpenVPN, squid, firewalling, fancy routing, url filtering, dns black listing and much much more. One note though, before we wrap up: You have two ways to initiate the initial setup wizard of the web-configurator: Spin up another droplet, log into it and browse your way to the INTERNAL ip address of the internal NIC you’ve set up. This is the long and tedious way, but it’s also somewhat safer as it eliminates the small window of risk the second method poses. or Once your WAN address is all setup, your pfSense is ready to accept https connection to start the initial web-configurator setup. Thing is, there’s a default, well known set of credential to this initial wizard (admin:pfsense), so, there is a slight window of opportunity that someone can swoop in (assuming they know you’ve installed pfsense + your wan IP address + the exact time window between setting up the WAN interface and completing the wizard) and do . I leave it up to you which of the path you’d like to go, either way, once you’re done with the web-configurator wizard, you’ll have a shiny new pfSense installation at your disposal running on your favorite VPS. Hopefully this was helpful for someone, I hope to get a similar post soon detailing how to get FreeNAS up and running on DO. Many thanks to Tubsta and his blogpost as well as to Allan Jude, Kris Moore and Benedict Reuschling for their AWESOME and inspiring podcast, BSD Now. ##News Roundup One year of C It’s now nearly a year that I started writing non-trivial amounts of C code again (the first sokol_gfx.h commit was on the 14-Jul-2017), so I guess it’s time for a little retrospective. In the beginning it was more of an experiment: I wanted to see how much I would miss some of the more useful C++ features (for instance namespaces, function overloading, ‘simple’ template code for containers, …), and whether it is possible to write non-trivial codebases in C without going mad. Here are all the github projects I wrote in C: sokol: a slowly growing set of platform-abstraction headers sokol-samples - examples for Sokol chips - 8-bit chip emulators chips-test - tests and examples for the chip- emulators, including some complete home computer emulators (minus sound) All in all these are around 32k lines of code (not including 3rd party code like flextGL and HandmadeMath). I think I wrote more C code in the recent 10 months than any other language. So one thing seems to be clear: yes, it’s possible to write a non-trivial amount of C code that does something useful without going mad (and it’s even quite enjoyable I might add). Here’s a few things I learned: Pick the right language for a problem C is a perfect match for WebAssembly C99 is a huge improvement over C89 The dangers of pointers and explicit memory management are overrated Less Boilerplate Code Less Language Feature ‘Anxiety’ Conclusion All in all my “C experiment” is a success. For a lot of problems, picking C over C++ may be the better choice since C is a much simpler language (btw, did you notice how there are hardly any books, conferences or discussions about C despite being a fairly popular language? Apart from the neverending bickering about undefined behaviour from the compiler people of course ;) There simply isn’t much to discuss about a language that can be learned in an afternoon. I don’t like some of the old POSIX or Linux APIs as much as the next guy (e.g. ioctl(), the socket API or some of the CRT library functions), but that’s an API design problem, not a language problem. It’s possible to build friendly C APIs with a bit of care and thinking, especially when C99’s designated initialization can be used (C++ should really make sure that the full C99 language can be used from inside C++ instead of continuing to wander off into an entirely different direction). ###Configuring OpenBGPD to announce VM’s virtual networks We use BGP quite heavily at work, and even though I’m not interacting with that directly, it feels like it’s something very useful to learn at least on some basic level. The most effective and fun way of learning technology is finding some practical application, so I decided to see if it could help to improve networking management for my Virtual Machines. My setup is fairly simple: I have a host that runs bhyve VMs and I have a desktop system from where I ssh to VMs, both hosts run FreeBSD. All VMs are connected to each other through a bridge and have a common network 10.0.1/24. The point of this exercise is to be able to ssh to these VMs from desktop without adding static routes and without adding vmhost’s external interfaces to the VMs bridge. I’ve installed openbgpd on both hosts and configured it like this: vmhost: /usr/local/etc/bgpd.conf AS 65002 router-id 192.168.87.48 fib-update no network 10.0.1.1/24 neighbor 192.168.87.41 { descr "desktop" remote-as 65001 } Here, router-id is set vmhost’s IP address in my home network (192.168.87/24), fib-update no is set to forbid routing table update, which I initially set for testing, but keeping it as vmhost is not supposed to learn new routes from desktop anyway. network announces my VMs network and neighbor describes my desktop box. Now the desktop box: desktop: /usr/local/etc/bgpd.conf AS 65001 router-id 192.168.87.41 fib-update yes neighbor 192.168.87.48 { descr "vmhost" remote-as 65002 } It’s pretty similar to vmhost’s bgpd.conf, but no networks are announced here, and fib-update is set to yes because the whole point is to get VM routes added. Both hosts have to have the openbgpd service enabled: /etc/rc.conf.local openbgpdenable="YES" Conclusion As mentioned already, similar result could be achieved without using BGP by using either static routes or bridging interfaces differently, but the purpose of this exercise is to get some basic hands-on experience with BGP. Right now I’m looking into extending my setup in order to try more complex BGP schema. I’m thinking about adding some software switches in front of my VMs or maybe adding a second VM host (if budget allows). You’re welcome to comment if you have some ideas how to extend this setup for educational purposes in the context of BGP and networking. As a side note, I really like openbgpd so far. Its configuration file format is clean and simple, documentation is good, error and information messages are clear, and CLI has intuitive syntax. Digital Ocean ###The Power to Serve All people within the IT Industry should known where the slogan “The Power To Serve” is exposed every day to millions of people. But maybe too much wishful thinking from me. But without “The Power To Serve” the IT industry today will look totally different. Companies like Apple, Juniper, Cisco and even WatsApp would not exist in their current form. I provide IT architecture services to make your complex IT landscape manageable and I love to solve complex security and privacy challenges. Complex challenges where people, processes and systems are heavily interrelated. For this knowledge intensive work I often run some IT experiments. When you run experiments nowadays you have a choice: Rent some cloud based services or DIY (Do IT Yourself) on premise Running your own developments experiments on your own infrastructure can be time consuming. However smart automation saves time and money. And by creating your own CICD pipeline (Continuous Integration, Continuous Deployment) you stay on top of core infrastructure developments. Even hands-on. Knowing how things work from a technical ‘hands-on’ perspective gives great advantages when it comes to solving complex business IT problems. Making a clear distinguish between a business problem or IT problem is useless. Business and IT problems are related. Sometimes causal related, but more often indirect by one or more non linear feedback loops. Almost every business depends of IT systems. Bad IT means often that your customers will leave your business. One of the things of FeeBSD for me is still FreeBSD Jails. In 2015 I had luck to attend to a presentation of the legendary hacker Poul-Henning Kamp . Check his BSD bio to see what he has done for the FreeBSD community! FreeBSD jails are a light way to visualize your system without enormous overhead. Now that the development on Linux for LXD/LXD is more mature (lxd is the next generation system container manager on linux) there is finally again an alternative for a nice chroot Linux based system again. At least when you do not need the overhead and management complexity that comes with Kubernetes or Docker. FreeBSD means control and quality for me. When there is an open source package I need, I want to install it from source. It gives me more control and always some extra knowledge on how things work. So no precompiled binaries for me on my BSD systems! If a build on FreeBSD fails most of the time this is an alert regarding the quality for me. If a complex OSS package is not available at all in the FreeBSD ports collection there should be a reason for it. Is it really that nobody on the world wants to do this dirty maintenance work? Or is there another cause that running this software on FreeBSD is not possible…There are currently 32644 ports available on FreeBSD. So all the major programming language, databases and middleware libraries are present. The FreeBSD organization is a mature organization and since this is one of the largest OSS projects worldwide learning how this community manages to keep innovation and creates and maintains software is a good entrance for learning how complex IT systems function. FreeBSD is of course BSD licensed. It worked well! There is still a strong community with lots of strong commercial sponsors around the community. Of course: sometimes a GPL license makes more sense. So beside FreeBSD I also love GPL software and the rationale and principles behind it. So my hope is that maybe within the next 25 years the hard battle between BSD vs GPL churches will be more rationalized and normalized. Principles are good, but as all good IT architects know: With good principles alone you never make a good system. So use requirements and not only principles to figure out what OSS license fits your project. There is never one size fits all. June 19, 1993 was the day the official name for FreeBSD was agreed upon. So this blog is written to celebrate 25th anniversary of FreeBSD. ###Dave’s BSDCan trip report So far, only one person has bothered to send in a BSDCan trip report. Our warmest thanks to Dave for doing his part. Hello guys! During the last show, you asked for a trip report regarding BSDCan 2018. This was my first time attending BSDCan. However, BSDCan was my second BSD conference overall, my first being vBSDCon 2017 in Reston, VA. Arriving early Thursday evening and after checking into the hotel, I headed straight to the Red Lion for the registration, picked up my badge and swag and then headed towards the ‘DMS’ building for the newbies talk. The only thing is, I couldn’t find the DMS building! Fortunately I found a BSDCan veteran who was heading there themselves. My only suggestion is to include the full building name and address on the BSDCan web site, or even a link to Google maps to help out with the navigation. The on-campus street maps didn’t have ‘DMS’ written on them anywhere. But I digress. Once I made it to the newbies talk hosted by Dan Langille and Michael W Lucas, it highlighted places to meet, an overview of what is happening, details about the ‘BSDCan widow/widower tours’ and most importantly, the 6-2-1 rule! The following morning, we were present with tea/coffee, muffins and other goodies to help prepare us for the day ahead. The first talk, “The Tragedy of systemd” covered what systemd did wrong and how the BSD community could improve on the ideas behind it. With the exception of Michael W Lucas, SSH Key Management and Kirk McKusick, The Evolution of FreeBSD Governance talk, I pretty much attended all of the ZFS talks including the lunchtime BoF session, hosted by Allan Jude. Coming from FreeNAS and being involved in the community, this is where my main interest and motivation lies. Since then I have been able to share some of that information with the FreeNAS community forums and chatroom. I also attended the “Speculating about Intel” lunchtime BoF session hosted by Theo de Raddt, which proved to be “interesting”. The talks ended with the wrap up session with a few words from Dan, covering the record attendance and made very clear there “was no cabal”. Followed by the the handing over of Groff the BSD goat to a new owner, thank you’s from the FreeBSD Foundation to various community committers and maintainers, finally ending with the charity auction, where a things like a Canadian $20 bill sold for $40, a signed FreeBSD Foundation shirt originally worn by George Neville-Neil, a lost laptop charger, Michael’s used gelato spoon, various books, the last cookie and more importantly, the second to last cookie! After the auction, we all headed to the Red Lion for food and drinks, sponsored by iXsystems. I would like to thank the BSDCan organizers, speakers and sponsors for a great conference. I will certainly hope to attend next year! Regards, Dave (aka m0nkey) Thanks to Dave for sharing his experiences with us and our viewers ##Beastie Bits Robert Watson (from 2008) on how much FreeBSD is in Mac OS X Why Intel Skylake CPUs are sometimes 50% slower than older CPUs Kristaps Dzonsons is looking for somebody to maintain this as mentioned at this link camcontrol(8) saves the day again! Formatting floppy disks in a USB floppy disk drive 32+ great indie games now playable on OpenBSD -current; 7 currently on sale! Warsaw BSD User Group. June 27 2018 18:30-21:00, Wheel Systems Office, Aleje Jerozolimskie 178, Warsaw Tarsnap ##Feedback/Questions Ron - Adding a disk to ZFS Marshall - zfs question Thomas - Allan, the myth perpetuator Ross - ZFS IO stats per dataset Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

TrueOS becoming a downstream fork with Trident, our BSDCan 2018 recap, HardenedBSD Foundation founding efforts, VPN with OpenIKED on OpenBSD, FreeBSD on a System76 Galago Pro, and hardware accelerated crypto on Octeons. ##Headlines## TrueOS to Focus on Core Operating System The TrueOS Project has some big plans in the works, and we want to take a minute and share them with you. Many have come to know TrueOS as the “graphical FreeBSD” that makes things easy for newcomers to the BSDs. Today we’re announcing that TrueOS is shifting our focus a bit to become a cutting-edge operating system that keeps all of the stability that you know and love from ZFS (OpenZFS) and FreeBSD, and adds additional features to create a fresh, innovative operating system. Our goal is to create a core-centric operating system that is modular, functional, and perfect for do-it-yourselfers and advanced users alike. TrueOS will become a downstream fork that will build on FreeBSD by integrating new software technologies like OpenRC and LibreSSL. Work has already begun which allows TrueOS to be used as a base platform for other projects, including JSON-based manifests, integrated Poudriere / pkg tools and much more. We’re planning on a six month release cycle to keep development moving and fresh, allowing us to bring you hot new features to ZFS, bhyve and related tools in a timely manner. This makes TrueOS the perfect fit to serve as the basis for building other distributions. Some of you are probably asking yourselves “But what if I want to have a graphical desktop?” Don’t worry! We’re making sure that everyone who knows and loves the legacy desktop version of TrueOS will be able to continue using a FreeBSD-based, graphical operating system in the future. For instance, if you want to add KDE, just use sudo pkg install kde and voila! You have your new shiny desktop. Easy right? This allows us to get back to our roots of being a desktop agnostic operating system. If you want to add a new desktop environment, you get to pick the one that best suits your use. We know that some of you will still be looking for an out-of-the-box solution similar to legacy PC-BSD and TrueOS. We’re happy to announce that Project Trident will take over graphical FreeBSD development going forward. Not much is going to change in that regard other than a new name! You’ll still have Lumina Desktop as a lightweight and feature-rich desktop environment and tons of utilities from the legacy TrueOS toolchain like sysadm and AppCafe. There will be migration paths available for those that would like to move to other FreeBSD-based distributions like Project Trident or GhostBSD. We look forward to this new chapter for TrueOS and hope you will give the new edition a spin! Tell us what you think about the new changes by leaving us a comment. Don’t forget you can ask us questions on our Twitter and be a part of our community by joining the new TrueOS Forums when they go live in about a week. Thanks for being a loyal fan of TrueOS. ###Project Trident FAQ Q: Why did you pick the name “Project Trident”? A: We were looking for a name that was unique, yet would still relate to the BSD community. Since Beastie (the FreeBSD mascot) is always pictured with a trident, it felt like that would be a great name. Q: Where can users go for technical support? A: At the moment, Project Trident will continue sharing the TrueOS community forums and Telegram channels. We are currently evaluating dedicated options for support channels in the future. Q: Can I help contribute to the project? A: We are always looking for developers who want to join the project. If you’re not a developer you can still help, as a community project we will be more reliant on contributions from the community in the form of how-to guides and other user-centric documentation and support systems. Q: How is the project supported financially? A: Project Trident is sponsored by the community, from both individuals and corporations. iXsystems has stepped up as the first enterprise-level sponsor of the project, and has been instrumental in getting Project Trident up and running. Please visit the Sponsors page to see all the current sponsors. Q: How can I help support the project financially? A: Several methods exist, from one time or recurring donations via Paypal to limited time swag t-shirt campaigns during the year. We are also looking into more alternative methods of support, so please visit the Sponsors page to see all the current methods of sponsorship. Q: Will there be any transparency of the financial donations and expenditures? A: Yes, we will be totally open with how much money comes into the project and what it is spent on. Due to concerns of privacy, we will not identify individuals and their donation amounts unless they specifically request to be identified. We will release a monthly overview in/out ledger, so that community members can see where their money is going. Relationship with TrueOS Project Trident does have very close ties to the TrueOS project, since most of the original Project Trident developers were once part of the TrueOS project before it became a distribution platform. For users of the TrueOS desktop, we have some additional questions and answers below. Q: Do we need to be at a certain TrueOS install level/release to upgrade? A: As long as you have a TrueOS system which has been updated to at least the 18.03 release you should be able to just perform a system update to be automatically upgraded to Project Trident. Q: Which members moved from TrueOS to Project Trident? A: Project Trident is being led by prior members of the TrueOS desktop team. Ken and JT (development), Tim (documentation) and Rod (Community/Support). Since Project Trident is a community-first project, we look forward to working with new members of the team. iXsystems ###BSDCan BSDCan finished Saturday last week It started with the GoatBoF on Tuesday at the Royal Oak Pub, where people had a chance to meet and greet. Benedict could not attend due to an all-day FreeBSD Foundation meeting and and even FreeBSD Journal Editorial Board meeting. The FreeBSD devsummit was held the next two days in parallel to the tutorials. Gordon Tetlow, who organized the devsummit, opened the devsummit. Deb Goodkin from the FreeBSD Foundation gave the first talk with a Foundation update, highlighting current and future efforts. Li-Wen Hsu is now employed by the Foundation to assist in QA work (Jenkins, CI/CD) and Gordon Tetlow has a part-time contract to help secteam as their secretary. Next, the FreeBSD core team (among them Allan and Benedict) gave a talk about what has happened this last term. With a core election currently running, some of these items will carry over to the next core team, but there were also some finished ones like the FCP process and FreeBSD members initiative. People in the audience asked questions on various topics of interest. After the coffee break, the release engineering team gave a talk about their efforts in terms of making releases happen in time and good quality. Benedict had to give his Ansible tutorial in the afternoon, which had roughly 15 people attending. Most of them beginners, we could get some good discussions going and I also learned a few new tricks. The overall feedback was positive and one even asked what I’m going to teach next year. The second day of the FreeBSD devsummit began with Gordon Tetlow giving an insight into the FreeBSD Security team (aka secteam). He gave a overview of secteam members and responsibilities, explaining the process based on a long past advisory. Developers were encouraged to help out secteam. NDAs and proper disclosure of vulnerabilities were also discussed, and the audience had some feedback and questions. When the coffee break was over, the FreeBSD 12.0 planning session happened. A Google doc served as a collaborative way of gathering features and things left to do. People signed up for it or were volunteered. Some features won’t make it into 12.0 as they are not 100% ready for prime time and need a few more rounds of testing and bugfixing. Still, 12.0 will have some compelling features. A 360° group picture was taken after lunch, and then people split up into the working groups for the afternoon or started hacking in the UofO Henderson residence. Benedict and Allan both attended the OpenZFS working group, lead by Matt Ahrens. He presented the completed and outstanding work in FreeBSD, without spoiling too much of the ZFS presentations of various people that happened later at the conference. Benedict joined the boot code session a bit late (hallway track is the reason) when most things seem to have already been discussed. BSDCan 2018 — Ottawa (In Pictures) iXsystems Photos from BSDCan 2018 ##News Roundup June HardenedBSD Foundation Update We at HardenedBSD are working towards starting up a 501©(3) not-for-profit organization in the USA. Setting up this organization will allow future donations to be tax deductible. We’ve made progress and would like to share with you the current state of affairs. We have identified, sent invitations out, and received acceptance letters from six people who will serve on the HardenedBSD Foundation Board of Directors. You can find their bios below. In the latter half of June 2018 or the beginning half of July 2018, we will meet for the first time as a board and formally begin the process of creating the documentation needed to submit to the local, state, and federal tax services. Here’s a brief introduction to those who will serve on the board: W. Dean Freeman (Advisor): Dean has ten years of professional experience with deploying and security Unix and networking systems, including assessing systems security for government certification and assessing the efficacy of security products. He was introduced to Unix via FreeBSD 2.2.8 on an ISP shell account as a teenager. Formerly, he was the Snort port maintainer for FreeBSD while working in the Sourcefire VRT, and has contributed entropy-related patches to the FreeBSD and HardenedBSD projects – a topic on which he presented at vBSDCon 2017. Ben La Monica (Advisor): Ben is a Senior Technology Manager of Software Engineering at Morningstar, Inc and has been developing software for over 15 years in a variety of languages. He advocates open source software and enjoys tinkering with electronics and home automation. George Saylor (Advisor): George is a Technical Directory at G2, Inc. Mr. Saylor has over 28 years of information systems and security experience in a broad range of disciplines. His core focus areas are automation and standards in the event correlation space as well as penetration and exploitation of computer systems. Mr Saylor was also a co-founder of the OpenSCAP project. Virginia Suydan (Accountant and general administrator): Accountant and general administrator for the HardenedBSD Foundation. She has worked with Shawn Webb for tax and accounting purposes for over six years. Shawn Webb (Director): Co-founder of HardenedBSD and all-around infosec wonk. He has worked and played in the infosec industry, doing both offensive and defensive research, for around fifteen years. He loves open source technologies and likes to frustrate the bad guys. Ben Welch (Advisor): Ben is currently a Security Engineer at G2, Inc. He graduated from Pennsylvania College of Technology with a Bachelors in Information Assurance and Security. Ben likes long walks, beaches, candlelight dinners, and attending various conferences like BSides and ShmooCon. ###Your own VPN with OpenIKED & OpenBSD Remote connectivity to your home network is something I think a lot of people find desirable. Over the years, I’ve just established an SSH tunnel and use it as a SOCKS proxy, sending my traffic through that. It’s a nice solution for a “poor man’s VPN”, but it can be a bit clunky, and it’s not great having to expose SSH to the world, even if you make sure to lock everything down I set out the other day to finally do it properly. I’d come across this great post by Gordon Turner: OpenBSD 6.2 VPN Endpoint for iOS and macOS Whilst it was exactly what I was looking for, it outlined how to set up an L2TP VPN. Really, I wanted IKEv2 for performance and security reasons (I won’t elaborate on this here, if you’re curious about the differences, there’s a lot of content out on the web explaining this). The client systems I’d be using have native support for IKEv2 (iOS, macOS, other BSD systems). But, I couldn’t find any tutorials in the same vein. So, let’s get stuck in! A quick note ✍️ This guide will walk through the set up of an IKEv2 VPN using OpenIKED on OpenBSD. It will detail a “road warrior” configuration, and use a PSK (pre-shared-key) for authentication. I’m sure it can be easily adapted to work on any other platforms that OpenIKED is available on, but keep in mind my steps are specifically for OpenBSD. Server Configuration As with all my home infrastructure, I crafted this set-up declaratively. So, I had the deployment of the VM setup in Terraform (deployed on my private Triton cluster), and wrote the configuration in Ansible, then tied them together using radekg/terraform-provisioner-ansible. One of the reasons I love Ansible is that its syntax is very simplistic, yet expressive. As such, I feel it fits very well into explaining these steps with snippets of the playbook I wrote. I’ll link the full playbook a bit further down for those interested. See the full article for the information on: sysctl parameters The naughty list (optional) Configure the VPN network interface Configure the firewall Configure the iked service Gateway configuration Client configuration Troubleshooting DigitalOcean ###FreeBSD on a System76 Galago Pro Hey all, It’s been a while since I last posted but I thought I would hammer something out here. My most recent purchase was a System76 Galago Pro. I thought, afer playing with POP! OS a bit, is there any reason I couldn’t get BSD on this thing. Turns out the answer is no, no there isnt and it works pretty decently. To get some accounting stuff out of the way I tested this all on FreeBSD Head and 11.1, and all of it is valid as of May 10, 2018. Head is a fast moving target so some of this is only bound to improve. The hardware Intel Core i5 Gen 8 UHD Graphics 620 16 GB DDR4 Ram RTL8411B PCI Express Card Reader RTL8111 Gigabit ethernet controller Intel HD Audio Samsung SSD 960 PRO 512GB NVMe The caveats There are a few things that I cant seem to make work straight out of the box, and that is the SD Card reader, the backlight, and the audio is a bit finicky. Also the trackpad doesn’t respond to two finger scrolling. The wiki is mostly up to date, there are a few edits that need to be made still but there is a bug where I cant register an account yet so I haven’t made all the changes. Processor It works like any other Intel processor. Pstates and throttling work. Graphics The boot menu sets itself to what looks like 1024x768, but works as you expect in a tiny window. The text console does the full 3200x1800 resolution, but the text is ultra tiny. There isnt a font for the console that covers hidpi screens yet. As for X Windows it requres the drm-kmod-next package. Once installed follow the directions from the package and it works with almost no fuss. I have it running on X with full intel acceleration, but it is running at it’s full 3200x1800 resolution, to scale that down just do xrandr --output eDP-1 --scale 0.5x0.5 it will blow it up to roughly 200%. Due to limitations with X windows and hidpi it is harder to get more granular. Intel Wireless 8265 The wireless uses the iwm module, as of right now it does not seem to automagically load right now. Adding iwm_load=“YES” will cause the module to load on boot and kldload iwm Battery I seem to be getting about 5 hours out of the battery, but everything reports out of the box as expected. I could get more by throttling the CPU down speed wise. Overall impression It is a pretty decent experience. While not as polished as a Thinkpad there is a lot of potential with a bit of work and polishing. The laptop itself is not bad, the keyboard is responsive. The build quality is pretty solid. My only real complaint is the trackpad is stiff to click and sort of tiny. They seem to be a bit indifferent to non linux OSes running on the gear but that isnt anything new. I wont have any problems using it and is enough that when I work through this laptop, but I’m not sure at this stage if my next machine will be a System76 laptop, but they have impressed me enough to put them in the running when I go to look for my next portable machine but it hasn’t yet replaced the hole left in my heart by lenovo messing with the thinkpad. ###Hardware accelerated AES/HMAC-SHA on octeons In this commit, visa@ submitted code (disabled for now) to use built-in acceleration on octeon CPUs, much like AESNI for x86s. I decided to test tcpbench(1) and IPsec, before and after updating and enabling the octcrypto(4) driver. I didn't capture detailed perf stats from before the update, I had heard someone say that Edgerouter Lite boxes would only do some 6MBit/s over ipsec, so I set up a really simple ipsec.conf with ike esp from A to B leading to a policy of esp tunnel from A to B spi 0xdeadbeef auth hmac-sha2-256 enc aes going from one ERL to another (I collect octeons, so I have a bunch to test with) and let tcpbench run for a while on it. My numbers hovered around 7Mbit/s, which coincided with what I've heard, and also that most of the CPU gets used while doing it. Then I edited /sys/arch/octeon/conf/GENERIC, removed the # from octcrypto0 at mainbus0 and recompiled. Booted into the new kernel and got a octcrypto0 line in dmesg, and it was time to rock the ipsec tunnel again. The crypto algorithm and HMAC used by default on ipsec coincides nicely with the list of accelerated functions provided by the driver. Before we get to tunnel traffic numbers, just one quick look at what systat pigs says while the ipsec is running at full steam: PID USER NAME CPU 20 40 60 80 100 58917 root crypto 52.25 ################# 42636 root softnet 42.48 ############## (idle) 29.74 ######### 1059 root tcpbench 24.22 ####### 67777 root crynlk 19.58 ###### So this indicates that the load from doing ipsec and generating the traffic is somewhat nicely evened out over the two cores in the Edgerouter, and there's even some CPU left unused, which means I can actually ssh into it and have it usable. I have had it running for almost 2 days now, moving some 2.1TB over the tunnel. Now for the new and improved performance numbers: 204452123 4740752 37.402 100.00% Conn: 1 Mbps: 37.402 Peak Mbps: 58.870 Avg Mbps: 37.402 204453149 4692968 36.628 100.00% Conn: 1 Mbps: 36.628 Peak Mbps: 58.870 Avg Mbps: 36.628 204454167 5405552 42.480 100.00% Conn: 1 Mbps: 42.480 Peak Mbps: 58.870 Avg Mbps: 42.480 204455188 5202496 40.804 100.00% Conn: 1 Mbps: 40.804 Peak Mbps: 58.870 Avg Mbps: 40.804 204456194 5062208 40.256 100.00% Conn: 1 Mbps: 40.256 Peak Mbps: 58.870 Avg Mbps: 40.256 The tcpbench numbers fluctuate up and down a bit, but the output is nice enough to actually keep tabs on the peak values. Peaking to 58.8MBit/s! Of course, as you can see, the average is lower but nice anyhow. A manyfold increase in performance, which is good enough in itself, but also moves the throughput from a speed that would make a poor but cheap gateway to something actually useful and decent for many home network speeds. Biggest problem after this gets enabled will be that my options to buy cheap used ERLs diminish. ##Beastie Bits Using FreeBSD Text Dumps llvm’s lld now the default linker for amd64 on FreeBSD Author Discoverability Pledge and Unveil in OpenBSD {pdf} EuroBSDCon 2018 CFP Closes June 17, hurry up and get your submissions in Just want to attend, but need help getting to the conference? Applications for the Paul Schenkeveld travel grant accepted until June 15th Tarsnap ##Feedback/Questions Casey - ZFS on Digital Ocean Jürgen - A Question Kevin - Failover best practice Dennis - SQL Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv

Looking at Lumina Desktop 2.0, 2 months of KPTI development in SmartOS, OpenBSD email service, an interview with Ryan Zezeski, NomadBSD released, and John Carmack's programming retreat with OpenBSD. This episode was brought to you by Headlines Looking at Lumina Desktop 2.0 (https://www.trueos.org/blog/looking-lumina-desktop-2-0/) A few weeks ago I sat down with Lead Developer Ken Moore of the TrueOS Project to get answers to some of the most frequently asked questions about Lumina Desktop from the open source community. Here is what he said on Lumina Desktop 2.0. Do you have a question for Ken and the rest of the team over at the TrueOS Project? Make sure to read the interview and comment below. We are glad to answer your questions! Ken: Lumina Desktop 2.0 is a significant overhaul compared to Lumina 1.x. Almost every single subsystem of the desktop has been streamlined, resulting in a nearly-total conversion in many important areas. With Lumina Desktop 2.0 we will finally achieve our long-term goal of turning Lumina into a complete, end-to-end management system for the graphical session and removing all the current runtime dependencies from Lumina 1.x (Fluxbox, xscreensaver, compton/xcompmgr). The functionality from those utilities is now provided by Lumina Desktop itself. Going along with the session management changes, we have compressed the entire desktop into a single, multi-threaded binary. This means that if any rogue script or tool starts trying to muck about with the memory used by the desktop (probably even more relevant now than when we started working on this), the entire desktop session will close/crash rather than allowing targeted application crashes to bypass the session security mechanisms. By the same token, this also prevents “man-in-the-middle” type of attacks because the desktop does not use any sort of external messaging system to communicate (looking at you dbus). This also gives a large performance boost to Lumina Desktop The entire system for how a user's settings get saved and loaded has been completely redone, making it a “layered” settings system which allows the default settings (Lumina) to get transparently replaced by system settings (OS/Distributor/SysAdmin) which can get replaced by individual user settings. This results in the actual changes in the user setting files to be kept to a minimum and allows for a smooth transition between updates to the OS or Desktop. This also provides the ability to “restrict” a user's desktop session (based on a system config file) to the default system settings and read-only user sessions for certain business applications. The entire graphical interface has been written in QML in order to fully-utilize hardware-based GPU acceleration with OpenGL while the backend logic and management systems are still written entirely in C++. This results in blazing fast performance on the backend systems (myriad multi-threaded C++ objects) as well as a smooth and responsive graphical interface with all the bells and whistles (drag and drop, compositing, shading, etc). Q: Are there future plans to implement something like Lumina in a MAC Jail? While I have never tried out Lumina in a MAC jail, I do not see anything on that page which should stop it from running in one right now. Lumina is already designed to be run as an unpriviledged user and is very smart about probing the system to find out what is/not available before showing anything to the user. The only thing that comes to mind is that you might need to open up some other system devices so that X11 itself can draw to the display (graphical environment setup is a bit different than CLI environment). Q: I look forward to these changes. I know the last time I used it when I would scroll I would get flashes like the refresh rate was not high enough. It will be nice to have a fast system as well as I know with the more changes Linux is becoming slower. Not once it has loaded but in the loading process. I will do another download when these changes come out and install again and maybe stay this time. If I recall correctly, one of the very first versions of Lumina (pre-1.0) would occasionally flicker. If that is still happening, you might want to verify that you are using the proper video driver for your hardware and/or enable the compositor within the Lumina settings. Q: Why was enlightenment project not considered for TrueOS? It is BSD licensed and is written in C. This was a common question about 4(?) years ago with the first release of the Lumina desktop and it basically boiled down to long-term support and reliability of the underlying toolkit. Some of the things we had to consider were: cross-platform/cross-architecture support, dependency reliability and support framework (Qt5 > EFL), and runtime requirements and dependency tracking (Qt5 is lighter than the EFL). That plus the fact that the EFL specifically states that it is linux-focused and the BSD's are just an afterthought (especially at the time we were doing the evaluation). Q: I have two questions. 1) The default layout of Unity(menu bar with actual menu entries on top and icon dock on the side) is one of the few things I liked about my first voyage into non-Windows systems, and have been missing since moving on to other distros(and now also other non-Linux systems). However in 1.4.0 screenshots on Lumina's site, the OSX-like layout has the menu attached to the window. Will 2.0 be able to have the menus on the bar? 2) Is there any timeline for a public release, or are you taking a “when it's ready” approach? In Lumina you can already put panels on the left/right side of the screen and give you something like the layout of the Unity desktop. The embedded menu system is not available in Lumina because that is not a specification supported by X11 and the window manager standards at the present time. The way that functionality is currently run on Linux is a hacky-bypass of the display system which only really works with the GTK3 and Qt5 toolkits, resulting in very odd overall desktop behavior in mixed environments where some apps use other graphical toolkits. We are targetting the 18.06 STABLE release of TrueOS for Lumina 2, but that is just a guideline and if necessary we will push back the release date to allow for additional testing/fixing as needed. A long two months (https://blog.cooperi.net/a-long-two-months) IllumOS/SmartOS developer Alex Wilson describes the journey of developing KPTI for IllumOS > On Monday (January 1st) I had the day off work for New Year's day, as is usual in most of the western world, so I slept in late. Lou and her friend decided to go to the wax museum and see several tourist attractions around SF, and I decided to pass the day at home reading. That afternoon, work chat started talking about a Tumblr post by pythonsweetness about an Intel hardware security bug. At the time I definitely did not suspect that this was going to occupy most of my working life for the next (almost) two months. Like many people who work on system security, I had read Anders Fogh's post about a "Negative Result" in speculative execution research in July of 2017. At the time I thought it was an interesting writeup and I remember being glad that researchers were looking into this area. I sent the post to Bryan and asked him about his thoughts on it at the time, to which he replied saying that "it would be shocking if they left a way to directly leak out memory in the speculative execution". None of us seriously thought that there would be low-hanging fruit down that research path, but we also felt it was important that there was someone doing work in the area who was committed to public disclosure. At first, after reading the blog post on Monday, we thought (or hoped) that the bug might "just" be a KASLR bypass and wouldn't require a lot of urgency. We tried to reach out to Intel at work to get more information but were met with silence. (We wouldn't hear back from them until after the disclosure was already made public.) The speculation on Tuesday intensified, until finally on Wednesday morning I arrived at the office to find links to late Tuesday night tweets revealing exploits that allowed arbitrary kernel memory reads. Wednesday was not a happy day. Intel finally responded to our emails -- after they had already initiated public disclosure. We all spent a lot of time reading. An arbitrary kernel memory read (an info leak) is not that uncommon as far as bugs go, but for the most part they tend to be fairly easy to fix. The thing that makes the Meltdown and Spectre bugs particularly notable is that in order to mitigate them, a large amount of change is required in very deep low-level parts of the kernel. The kind of deep parts of the kernel where there are 20-year old errata workarounds that were single-line changes that you have to be very careful to not accidentally undo; the kind of parts where, as they say, mortals fear to tread. On Friday we saw the patches Matthew Dillon put together for DragonFlyBSD for the first time. These were the first patches for KPTI that were very straightforward to read and understand, and applied to a BSD-derived kernel that was similar to those I'm accustomed to working on. To mitigate Meltdown (and partially one of the Spectre variants), you have to make sure that speculative execution cannot reach any sensitive data from a user context. This basically means that the pages the kernel uses for anything potentially sensitive have to be unmapped when we are running user code. Traditionally, CPUs that were built to run a multi-user, UNIX-like OS did this by default (SPARC is an example of such a CPU which has completely separate address spaces for the kernel and userland). However, x86 descends from a single-address-space microcontroller that has grown up avoiding backwards-incompatible changes, and has never really introduced a clean notion of multiple address spaces (segmentation is the closest feature really, and it was thrown out for 64-bit AMD64). Instead, operating systems for x86 have generally wound up (at least in the post-AMD64 era) with flat address space models where the kernel text and data is always present in the page table no matter whether you're in user or kernel mode. The kernel mappings simply have the "supervisor" bit set on them so that user code can't directly access them. The mitigation is basically to stop doing this: to stop mapping the kernel text, data and other memory into the page table while we're running in userland. Unfortunately, the x86 design does not make this easy. In order to be able to take interrupts or traps, the CPU has to have a number of structures mapped in the current page table at all times. There is also no ability to tell an x86 CPU that you want it to switch page tables when an interrupt occurs. So, the code that we jump to when we take an interrupt, as well as space for a stack to push context onto have to be available in both page tables. And finally, of course, we need to be able to figure out somehow what the other page table we should switch to is when we enter the kernel. When we looked at the patches for Linux (and also the DragonFlyBSD patches at the time) on Friday and started asking questions, it became pretty evident that the initial work done by both was done under time constraints. Both had left the full kernel text mapped in both page tables, and the Linux trampoline design seemed over-complex. I started talking over some ideas with Robert Mustacchi about ways to fix these and who we should talk to, and reached out to some of my old workmates from the University of Queensland who were involved with OpenBSD. It seemed to me that the OpenBSD developers would care about these issues even more than we did, and would want to work out how to do the mitigation right. I ended up sending an email to Philip Guenther on Friday afternoon, and on Saturday morning I drove an hour or so to meet up with him for coffee to talk page tables and interrupt trampolines. We wound up spending a good 6 hours at the coffee shop, and I came back with several pages of notes and a half-decent idea of the shape of the work to come. One detail we missed that day was the interaction of per-CPU structures with per-process page tables. Much of the interrupt trampoline work is most easily done by using per-CPU structures in memory (and you definitely want a per-CPU stack!). If you combine that with per-process page tables, however, you have a problem: if you leave all the per-CPU areas mapped in all the processes, you will leak information (via Meltdown) about the state of one process to a different one when taking interrupts. In particular, you will leak things like %rip, which ruins all the work being done with PIE and ASLR pretty quickly. So, there are two options: you can either allocate the per-CPU structures per-process (so you end up with $NCPUS * $NPROCS of them); or you can make the page tables per-CPU. OpenBSD, like Linux and the other implementations so far, decided to go down the road of per-CPU per-process pages to solve this issue. For illumos, we took the other route. In illumos, it turned out that we already had per-CPU page tables. Robert and I re-discovered this on the Sunday of that week. We use them for 32-bit processes due to having full P>V PAE support in our kernel (which is, as it turns out, relatively uncommon amongst open-source OS). The logic to deal with creating and managing them and updating them was all already written, and after reading the code we concluded we could basically make a few small changes and re-use all of it. So we did. By the end of that second week, we had a prototype that could get to userland. But, when working on this kind of kernel change we have a rule of thumb we use: after the first 70% of the patch is done and we can boot again, now it's time for the second 70%. In fact it turned out to be more like the second 200% for us -- a tedious long tail of bugs to solve that ended up necessitating some changes in the design as well. At first we borrowed the method that Matt Dillon used for DragonFlyBSD, by putting the temporary "stack" space and state data for the interrupt trampolines into an extra page tacked onto the end of *%gs (in illumos the structure that lives there is the cpu_t). If you read the existing logic in interrupt handlers for dealing with %gs though, you will quickly notice that the corner cases start to build up. There are a bunch of situations where the kernel temporarily alters %gs, and some of the ways to mess it up have security consequences that end up being worse than the bug we're trying to fix. As it turns out, there are no less than 3 different ways that ISRs use to try to get to having the right cpu_t in %gs on illumos, as it turns out, and they are all subtly different. Trying to tell which you should use when requires a bunch of test logic that in turn requires branches and changes to the CPU state, which is difficult to do in a trampoline where you're trying to avoid altering that state as much as possible until you've got the real stack online to push things into. I kept in touch with Philip Guenther and Mike Larkin from the OpenBSD project throughout the weeks that followed. In one of the discussions we had, we talked about the NMI/MCE handlers and the fact that their handling currently on OpenBSD neglected some nasty corner-cases around interrupting an existing trap handler. A big part of the solution to those issues was to use a feature called IST, which allows you to unconditionally change stacks when you take an interrupt. Traditionally, x86 only changes the stack pointer (%rsp on AMD64) while taking an interrupt when there is a privilege level change. If you take an interrupt while already in the kernel, the CPU does not change the stack pointer, and simply pushes the interrupt stack frame onto the stack you're already using. IST makes the change of stack pointer unconditional. If used unwisely, this is a bad idea: if you stay on that stack and turn interrupts back on, you could take another interrupt and clobber the frame you're already in. However, in it I saw a possible way to simplify the KPTI trampoline logic and avoid having to deal with %gs. A few weeks into the project, John Levon joined us at work. He had previously worked on a bunch of Xen-related stuff as well as other parts of the kernel very close to where we were, so he quickly got up to speed with the KPTI work as well. He and I drafted out a "crazy idea" on the whiteboard one afternoon where we would use IST for all interrupts on the system, and put the "stack" they used in the KPTI page on the end of the cpu_t. Then, they could easily use stack-relative addresses to get the page table to change to, then pivot their stack to the real kernel stack memory, and throw away (almost) all the conditional logic. A few days later, we had convinced each other that this was the way to go. Two of the most annoying x86 issues we had to work around were related to the SYSENTER instruction. This instruction is used to make "fast" system calls in 32-bit userland. It has a couple of unfortunate properties: firstly, it doesn't save or restore RFLAGS, so the kernel code has to take care of this (and be very careful not to clobber any of it before saving or after restoring it). Secondly, if you execute SYSENTER with the TF ("trap"/single-step flag) set by a debugger, the resulting debug trap's frame points at kernel code instead of the user code where it actually happened. The first one requires some careful gymnastics on the entry and return trampolines specifically for SYSENTER, while the second is a nasty case that is incidentally made easier by using IST. With IST, we can simply make the debug trap trampoline check for whether we took the trap in another trampoline's code, and reset %cr3 and the destination stack. This works for single-stepping into any of the handlers, not just the one for SYSENTER. To make debugging easier, we decided that traps like the debug/single-step trap (as well as faults like page faults, #GP, etc.) would push their interrupt frame in a different part of the KPTI state page to normal interrupts. We applied this change to all the traps that can interrupt another trampoline (based on the instructions we used). These "paranoid" traps also set a flag in the KPTI struct to mark it busy (and jump to the double-fault handler if it is), to work around some bugs where double-faults are not correctly generated. It's been a long and busy two months, with lots of time spent building, testing, and validating the code. We've run it on as many kinds of machines as we could get our hands on, to try to make sure we catch issues. The time we've spent on this has been validated several times in the process by finding bugs that could have been nasty in production. One great example: our patches on Westmere-EP Xeons were causing busy machines to throw a lot of L0 I-cache parity errors. This seemed very mysterious at first, and it took us a few times seeing it to believe that it was actually our fault. This was actually caused by the accidental activation of a CPU errata for Westmere (B52, "Memory Aliasing of Code Pages May Cause Unpredictable System Behaviour") -- it turned out we had made a typo and put the "cacheable" flag into a variable named flags instead of attrs where it belonged when setting up the page tables. This was causing performance degradation on other machines, but on Westmere it causes cache parity errors as well. This is a great example of the surprising consequences that small mistakes in this kind of code can end up having. In the end, I'm glad that that erratum existed, otherwise it may have been a long time before we caught that bug. As of this week, Mike and Philip have committed the OpenBSD patches for KPTI to their repository, and the patches for illumos are out for review. It's a nice kind of symmetry that the two projects who started on the work together after the public disclosure at the same time are both almost ready to ship at the same time at the other end. I'm feeling hopeful, and looking forward to further future collaborations like this with our cousins, the BSDs. The IllumOS work has since landed, on March 12th (https://github.com/joyent/illumos-joyent/commit/d85fbfe15cf9925f83722b6d62da49d549af615c) *** OpenBSD Email Service (https://github.com/vedetta-com/caesonia) Features Efficient: configured to run on min. 512MB RAM and 20GB SSD, a KVM (cloud) VPS for around $2.50/mo 15GB+ uncompressed Maildir, rivals top free-email providers (grow by upgrading SSD) Email messages are gzip compressed, at least 1/3 more space with level 6 default Server side full text search (headers and body) can be enabled (to use the extra space) Mobile data friendly: IMAPS connections are compressed Subaddress (+tag) support, to filter and monitor email addresses Virtual domains, aliases, and credentials in files, Berkeley DB, or SQLite3 Naive Bayes rspamd filtering with supervised learning: the lowest false positive spam detection rates Carefree automated Spam/ and Trash/ cleaning service (default: older than 30 days) Automated quota management, gently assists when over quota Easy backup MX setup: using the same configuration, install in minutes on a different host Worry-free automated master/master replication with backup MX, prevents accidental loss of email messages Resilient: the backup MX can be used as primary, even when the primary is not down, both perfect replicas Flexible: switching roles is easy, making the process of changing VPS hosts a breeze (no downtime) DMARC (with DKIM and SPF) email-validation system, to detect and prevent email spoofing Daily (spartan) stats, to keep track of things Your sieve scripts and managesieve configuration, let's get started Considerations By design, email message headers need to be public, for exchanges to happen. The body of the message can be encrypted by the user, if desired. Moreover, there is no way to prevent the host from having access to the virtual machine. Therefore, full disk encryption (at rest) may not be necessary. Given our low memory requirements, and the single-purpose concept of email service, Roundcube or other web-based IMAP email clients should be on a different VPS. Antivirus software users (usually) have the service running on their devices. ClamAV can easily be incorporated into this configuration, if affected by the types of malware it protects against, but will require around 1GB additional RAM (or another VPS). Every email message is important, if properly delivered, for Bayes classification. At least 200 ham and 200 spam messages are required to learn what one considers junk. By default (change to use case), a rspamd score above 50% will send the message to Spam/. Moving messages in and out of Spam/ changes this score. After 95%, the message is flagged as "seen" and can be safely ignored. Spamd is effective at greylisting and stopping high volume spam, if it becomes a problem. It will be an option when IPv6 is supported, along with bgp-spamd. System mail is delivered to an alias mapped to a virtual user served by the service. This way, messages are guaranteed to be delivered via encrypted connection. It is not possible for real users to alias, nor mail an external mail address with the default configuration. e.g. puffy@mercury.example.com is wheel, with an alias mapped to (virtual) puffy@example.com, and user (puffy) can be different for each. Interview - Ryan Zezeski - rpz@joyent.com (mailto:rpz@joyent.com) / @rzezeski (https://twitter.com/rzezeski) News Roundup John Carmack's programming retreat to hermit coding with OpenBSD (https://www.facebook.com/permalink.php?story_fbid=2110408722526967&id=100006735798590) After a several year gap, I finally took another week-long programming retreat, where I could work in hermit mode, away from the normal press of work. My wife has been generously offering it to me the last few years, but I'm generally bad at taking vacations from work. As a change of pace from my current Oculus work, I wanted to write some from-scratch-in-C++ neural network implementations, and I wanted to do it with a strictly base OpenBSD system. Someone remarked that is a pretty random pairing, but it worked out ok. Despite not having actually used it, I have always been fond of the idea of OpenBSD — a relatively minimal and opinionated system with a cohesive vision and an emphasis on quality and craftsmanship. Linux is a lot of things, but cohesive isn't one of them. I'm not a Unix geek. I get around ok, but I am most comfortable developing in Visual Studio on Windows. I thought a week of full immersion work in the old school Unix style would be interesting, even if it meant working at a slower pace. It was sort of an adventure in retro computing — this was fvwm and vi. Not vim, actual BSD vi. In the end, I didn't really explore the system all that much, with 95% of my time in just the basic vi / make / gdb operations. I appreciated the good man pages, as I tried to do everything within the self contained system, without resorting to internet searches. Seeing references to 30+ year old things like Tektronix terminals was amusing. I was a little surprised that the C++ support wasn't very good. G++ didn't support C++11, and LLVM C++ didn't play nicely with gdb. Gdb crashed on me a lot as well, I suspect due to C++ issues. I know you can get more recent versions through ports, but I stuck with using the base system. In hindsight, I should have just gone full retro and done everything in ANSI C. I do have plenty of days where, like many older programmers, I think “Maybe C++ isn't as much of a net positive as we assume...”. There is still much that I like, but it isn't a hardship for me to build small projects in plain C. Maybe next time I do this I will try to go full emacs, another major culture that I don't have much exposure to. I have a decent overview understanding of most machine learning algorithms, and I have done some linear classifier and decision tree work, but for some reason I have avoided neural networks. On some level, I suspect that Deep Learning being so trendy tweaked a little bit of contrarian in me, and I still have a little bit of a reflexive bias against “throw everything at the NN and let it sort it out!” In the spirit of my retro theme, I had printed out several of Yann LeCun's old papers and was considering doing everything completely off line, as if I was actually in a mountain cabin somewhere, but I wound up watching a lot of the Stanford CS231N lectures on YouTube, and found them really valuable. Watching lecture videos is something that I very rarely do — it is normally hard for me to feel the time is justified, but on retreat it was great! I don't think I have anything particularly insightful to add about neural networks, but it was a very productive week for me, solidifying “book knowledge” into real experience. I used a common pattern for me: get first results with hacky code, then write a brand new and clean implementation with the lessons learned, so they both exist and can be cross checked. I initially got backprop wrong both times, comparison with numerical differentiation was critical! It is interesting that things still train even when various parts are pretty wrong — as long as the sign is right most of the time, progress is often made. I was pretty happy with my multi-layer neural net code; it wound up in a form that I can just drop it into future efforts. Yes, for anything serious I should use an established library, but there are a lot of times when just having a single .cpp and .h file that you wrote ever line of is convenient. My conv net code just got to the hacky but working phase, I could have used another day or two to make a clean and flexible implementation. One thing I found interesting was that when testing on MNIST with my initial NN before adding any convolutions, I was getting significantly better results than the non-convolutional NN reported for comparison in LeCun ‘98 — right around 2% error on the test set with a single 100 node hidden layer, versus 3% for both wider and deeper nets back then. I attribute this to the modern best practices —ReLU, Softmax, and better initialization. This is one of the most fascinating things about NN work — it is all so simple, and the breakthrough advances are often things that can be expressed with just a few lines of code. It feels like there are some similarities with ray tracing in the graphics world, where you can implement a physically based light transport ray tracer quite quickly, and produce state of the art images if you have the data and enough runtime patience. I got a much better gut-level understanding of overtraining / generalization / regularization by exploring a bunch of training parameters. On the last night before I had to head home, I froze the architecture and just played with hyperparameters. “Training!” Is definitely worse than “Compiling!” for staying focused. Now I get to keep my eyes open for a work opportunity to use the new skills! I am dreading what my email and workspace are going to look like when I get into the office tomorrow. Stack-register Checking (https://undeadly.org/cgi?action=article;sid=20180310000858) Recently, Theo de Raadt (deraadt@) described a new type of mitigation he has been working on together with Stefan Kempf (stefan@): How about we add another new permission! This is not a hardware permission, but a software permission. It is opportunistically enforced by the kernel. The permission is MAP_STACK. If you want to use memory as a stack, you must mmap it with that flag bit. The kernel does so automatically for the stack region of a process's stack. Two other types of stack occur: thread stacks, and alternate signal stacks. Those are handled in clever ways. When a system call happens, we check if the stack-pointer register points to such a page. If it doesn't, the program is killed. We have tightened the ABI. You may no longer point your stack register at non-stack memory. You'll be killed. This checking code is MI, so it works for all platforms. For more detail, see Theo's original message (https://marc.info/?l=openbsd-tech&m=152035796722258&w=2). This is now available in snapshots, and people are finding the first problems in the ports tree already. So far, few issues have been uncovered, but as Theo points out, more testing is necessary: Fairly good results. A total of 4 problems have been found so far. go, SBCL, and two cases in src/regress which failed the new page-alignment requirement. The SBCL and go ones were found at buildtime, since they use themselves to complete build. But more page-alignment violations may be found in ports at runtime. This is something I worry about a bit. So please everyone out there can help: Use snapshots which contain the stack-check diff, update to new packages, and test all possible packages. Really need a lot of testing for this, so please help out. So, everybody, install the latest snapshot and try all your favorite ports. This is the time to report issues you find, so there is a good chance this additional security feature is present in 6.3 (and works with third party software from packages). NomadBSD 1.0 has been released (https://freeshell.de/~mk/projects/nomadbsd.html) NomadBSD is a live system for flash drives, based on FreeBSD® 11.1 (amd64) Change Log The setup process has been improved. Support for optional geli encryption of the home partition has been added Auto-detection of NVIDIA graphics cards and their corresponding driver has been added. (Thanks to holgerw and lme from BSDForen.de) An rc script to start the GEOM disk scheduler on the root device has been added. More software has been added: accessibility/redshift (starts automatically) audio/cantata audio/musicpd audio/ncmpc ftp/filezilla games/bsdtris mail/neomutt math/galculator net-p2p/transmission-qt5 security/fpm2 sysutils/bsdstats x11/metalock x11/xbindkeys Several smaller improvements and bugfixes. Screenshots https://freeshell.de/~mk/projects/nomadbsd-ss1.png https://freeshell.de/~mk/projects/nomadbsd-ss2.png https://freeshell.de/~mk/projects/nomadbsd-ss3.png https://freeshell.de/~mk/projects/nomadbsd-ss4.png https://freeshell.de/~mk/projects/nomadbsd-ss5.png https://freeshell.de/~mk/projects/nomadbsd-ss6.png Beastie Bits KnoxBug - Nagios (http://knoxbug.org/2018-03-27) vBSDcon videos landing (https://www.youtube.com/playlist?list=PLfJr0tWo35bc9FG_reSki2S5S0G8imqB4) AsiaBSDCon 2017 videos (https://www.youtube.com/playlist?list=PLnTFqpZk5ebBTyXedudGm6CwedJGsE2Py) DragonFlyBSD Adds New "Ptr_Restrict" Security Option (https://www.phoronix.com/scan.php?page=news_item&px=DragonFlyBSD-Ptr-Restrict) A Dexter needs your help (https://twitter.com/michaeldexter/status/975603855407788032) Mike Larkin at bhyvecon 2018: OpenBSD vmm(4) update (https://undeadly.org/cgi?action=article;sid=20180309064801) [HEADS UP] - OFED/RDMA stack update (https://lists.freebsd.org/pipermail/freebsd-arch/2018-March/018900.html) *** Feedback/Questions Ron - Interview someone using DragonflyBSD (http://dpaste.com/3BM6GSW#wrap) Brad - Gaming and all (http://dpaste.com/3X4ZZK2#wrap) Mohammad - Sockets vs TCP (http://dpaste.com/0PJMKRD#wrap) Paul - All or at least most of Bryan Cantrill's Talks (http://dpaste.com/2WXVR1X#wrap) ***

We cover an interview about Unix Architecture Evolution, another vBSDcon trip report, how to teach an old Unix about backspace, new NUMA support coming to FreeBSD, and stack pointer checking in OpenBSD. This episode was brought to you by Headlines Unix Architecture Evolution from the 1970 PDP-7 to the 2017 FreeBSD (https://fosdem.org/2018/interviews/diomidis-spinellis/) Q: Could you briefly introduce yourself? I'm a professor of software engineering, a programmer at heart, and a technology author. Currently I'm also the editor in chief of the IEEE Software magazine. I recently published the book Effective Debugging, where I detail 66 ways to debug software and systems. Q: What will your talk be about, exactly? I will describe how the architecture of the Unix operating system evolved over the past half century, starting from an unnamed system written in PDP-7 assembly language and ending with a modern FreeBSD system. My talk is based, first, on a GitHub repository where I tried to record the system's history from 1970 until today and, second, on the evolution of documented facilities (user commands, system calls, library functions) across revisions. I will thus present the early system's defining architectural features (layering, system calls, devices as files, an interpreter, and process management) and the important ones that followed in subsequent releases: the tree directory structure, user contributed code, I/O redirection, the shell as a user program, groups, pipes, scripting, and little languages. Q: Why this topic? Unix stands out as a major engineering breakthrough due to its exemplary design, its numerous technical contributions, its impact, its development model, and its widespread use. Furthermore, the design of the Unix programming environment has been characterized as one offering unusual simplicity, power, and elegance. Consequently, there are many lessons that we can learn by studying the evolution of the Unix architecture, which we can apply to the design of new systems. I often see modern systems that suffer from a bloat of architectural features and a lack of clear form on which functionality can be built. I believe that many of the modern Unix architecture defining features are excellent examples of what we should strive toward as system architects. Q: What do you hope to accomplish by giving this talk? What do you expect? I'd like FOSDEM attendees to leave the talk with their mind full with architectural features of timeless quality. I want them to realize that architectural elegance isn't derived by piling design patterns and does not need to be expensive in terms of resources. Rather, beautiful architecture can be achieved on an extremely modest scale. Furthermore, I want attendees to appreciate the importance of adopting flexible conventions rather than rigid enforcement mechanisms. Finally, I want to demonstrate through examples that the open source culture was part of Unix from its earliest days. Q: What are the most significant milestones in the development of Unix? The architectural development of Unix follows a path of continuous evolution, albeit at a slowing pace, so I don't see here the most important milestones. I would however define as significant milestones two key changes in the way Unix was developed. The first occurred in the late 1970s when significant activity shifted from a closely-knit team of researchers at the AT&T Bell Labs to the Computer Science Research Group in the University of California at Berkeley. This opened the system to academic contributions and growth through competitive research funding. The second took place in the late 1980s and the 1990s when Berkeley open-sourced the the code it had developed (by that time a large percentage of the system) and enthusiasts built on it to create complete open source operating system distributions: 386BSD, and then FreeBSD, NetBSD, OpenBSD, and others. Q: In which areas has the development of Unix stalled? The data I will show demonstrate that there were in the past some long periods where the number of C library functions and system calls remained mostly stable. Nowadays there is significant growth in the number of all documented facilities with the exception of file formats. I'm looking forward to a discussion regarding the meaning of these growth patterns in the Q&A session after the talk. Q: What are the core features that still link the 1970 PDP-7 system to the latest FreeBSD 11.1 release, almost half a century apart? Over the past half-century the Unix system has grown by four orders of magnitude from a few thousand lines of code to many millions. Nevertheless, looking at a 1970s architecture diagram and a current one reveals that the initial architectural blocks are still with us today. Furthermore, most system calls, user programs, and C library functions of that era have survived until today with essentially similar functionality. I've even found in modern FreeBSD some lines of code that have survived unchanged for 40 years. Q: Can we still add innovative changes to operating systems like FreeBSD without breaking the ‘Unix philosophy'? Will there be a moment where FreeBSD isn't recognizable anymore as a descendant of the 1970 PDP-7 system? There's a saying that “form liberates”. So having available a time-tested form for developing operating system functionality allows you to innovate in areas that matter rather than reinventing the wheel. Such concepts include having commands act as a filter, providing manual pages with a consistent structure, supplying build information in the form of a Makefile, installing files in a well-defined directory hierarchy, implementing filesystems with an standardized object-oriented interface, and packaging reusable functions as a library. Within this framework there's ample space for both incremental additions (think of jq, the JSON query command) and radical innovations (consider the Solaris-derived ZFS and dtrace functionality). For this reason I think that BSD and Linux systems will always be recognizable as direct or intellectual descendants of the 1970s Research Unix editions. Q: Have you enjoyed previous FOSDEM editions? Immensely! As an academic I need to attend many scientific conferences and meetings in order to present research results and interact with colleagues. This means too much time spent traveling and away from home, and a limited number of conferences I'm in the end able to attend. Nevertheless, attending FOSDEM is an easy decision due to the world-changing nature of its theme, the breadth of the topics presented, the participants' enthusiasm and energy, as well as the exemplary, very efficient conference organization. Another vBSDCon trip report we just found (https://www.weaponizedawesome.com/blog/?cat=53) We just got tipped about another trip report from vBSDCon, this time from one of the first time speakers: W. Dean Freeman Recently I had the honor of co-presenting on the internals of FreeBSD's Kernel RNG with John-Mark Gurney at the 3rd biennial vBSDCon, hosted in Reston, VA hosted by Verisign. I've been in and out of the FreeBSD community for about 20 years. As I've mentioned on here before, my first Unix encounter was FreeBSD 2.2.8 when I was in the 7th or 8th grade. However, for all that time I've never managed to get out to any of the cons. I've been to one or two BUG meetings and I've met some folks from IRC before, but nothing like this. A BSD conference is a very different experience than anything else out there. You have to try it, it is the only way to truly understand it. I'd also not had to do a stand-up presentation really since college before this. So, my first BSD con and my first time presenting rolled into one made for an interesting experience. See, he didn't say terrifying. It went very well. You should totally submit a talk for the next conference, even if it is your first. That said, it was amazing and invigorating experience. I got to meet a few big names in the FreeBSD community, discuss projects, ideas for FreeBSD, etc. I did seem to spend an unusual amount of time talking about FIPS and Common Criteria with folks, but to me that's a good sign and indicative that there is interest in working to close gaps between FreeBSD and the current requirements so that we can start getting FreeBSD and more BSD-based products into the government and start whittling away the domination of Linux (especially since Oracle has cut Solaris, SPARC and the ZFS storage appliance business units). There is nothing that can match the high bandwidth interchange of ideas in person. The internet has made all kinds of communication possible, and we use it all the time, but every once in a while, getting together in person is hugely valuable. Dean then went on to list some of the talks he found most valuable, including DTrace, Capsicum, bhyve, *BSD security tools, and Paul Vixie's talk about gets() I think the talk that really had the biggest impact on me, however, was Kyle Kneisl's talk on BSD community dynamics. One of the key points he asked was whether the things that drew us to the BSD community in the first place would be able to happen today. Obviously, I'm not a 12 or 13 year old kid anymore, but it really got me thinking. That, combined with getting face time with people I'd previously only known as screen names has recently drawn me back into participating in IRC and rejoining mailing lists (wdf on freenode. be on the lookout!) Then Dean covered some thoughts on his own talk: JMG and my talk seems to have been well received, with people paying lots of attention. I don't know what a typical number of questions is for one of these things, but on day one there weren't that many questions. We got about 5 during our question time and spent most of the rest of the day fielding questions from interested attendees. Getting a “great talk!” from GNN after coming down from the stage was probably one of the major highlights for me. I remember my first solo talk, and GNN asking the right question in the middle to get me to explain a part of it I had missed. It was very helpful. I think key to the interest in our presentation was that JMG did a good job framing a very complicated topic's importance in terms everyone could understand. It also helped that we got to drop some serious truth bombs. Final Thoughts: I met a lot of folks in person for the first time, and met some people I'd never known online before. It was a great community and I'm glad I got a chance to expand my network. Verisign were excellent hosts and they took good care of both speakers (covering airfare, rooms, etc.) and also conference attendees at large. The dinners that they hosted were quite good as well. I'm definitely interested in attending vBSDCon again and now that I've had a taste of meeting IRL with the community on scale of more than a handful, I have every intention of finally making it to BSDCan next year (I'd said it in 2017, but then moved to Texas for a new job and it wasn't going to be practical). This year for sure, though! Teaching an Almost 40-year Old UNIX about Backspace (https://virtuallyfun.com/2018/01/17/teaching_an_almost_40-year_old_unix_about_backspace/) Introduction I have been messing with the UNIX® operating system, Seventh Edition (commonly known as UNIX V7 or just V7) for a while now. V7 dates from 1979, so it's about 40 years old at this point. The last post was on V7/x86, but since I've run into various issues with it, I moved on to a proper installation of V7 on SIMH. The Internet has some really good resources on installing V7 in SIMH. Thus, I set out on my own journey on installing and using V7 a while ago, but that was remarkably uneventful. One convenience that I have been dearly missing since the switch from V7/x86 is a functioning backspace key. There seem to be multiple different definitions of backspace: BS, as in ASCII character 8 (010, 0x08, also represented as ^H), and DEL, as in ASCII character 127 (0177, 0x7F, also represented as ^?). V7 does not accept either for input by default. Instead, # is used as the erase character and @ is used as the kill character. These defaults have been there since UNIX V1. In fact, they have been “there” since Multics, where they got chosen seemingly arbitrarily. The erase character erases the character before it. The kill character kills (deletes) the whole line. For example, “ba##gooo#d” would be interpreted as “good” and “bad line@good line” would be interpreted as “good line”. There is some debate on whether BS or DEL is the correct character for terminals to send when the user presses the backspace key. However, most programs have settled on DEL today. tmux forces DEL, even if the terminal emulator sends BS, so simply changing my terminal to send BS was not an option. The change from the defaults outlined here to today's modern-day defaults occurred between 4.1BSD and 4.2BSD. enf on Hacker News has written a nice overview of the various conventions Getting the Diff For future generations as well as myself when I inevitably majorly break this installation of V7, I wanted to make a diff. However, my V7 is installed in SIMH. I am not a very intelligent man, I didn't keep backup copies of the files I'd changed. Getting data out of this emulated machine is an exercise in frustration. In the end, I printed everything on screen using cat(1) and copied that out. Then I performed a manual diff against the original source code tree because tabs got converted to spaces in the process. Then I applied the changes to clean copies that did have the tabs. And finally, I actually invoked diff(1). Closing Thoughts Figuring all this out took me a few days. Penetrating how the system is put together was surprisingly fairly hard at first, but then the difficulty curve eased up. It was an interesting exercise in some kind of “reverse engineering” and I definitely learned something about tty handling. I was, however, not pleased with using ed(1), even if I do know the basics. vi(1) is a blessing that I did not appreciate enough until recently. Had I also been unable to access recursive grep(1) on my host and scroll through the code, I would've probably given up. Writing UNIX under those kinds of editing conditions is an amazing feat. I have nothing but the greatest respect for software developers of those days. News Roundup New NUMA support coming to FreeBSD CURRENT (https://lists.freebsd.org/pipermail/freebsd-current/2018-January/068145.html) Hello folks, I am working on merging improved NUMA support with policy implemented by cpuset(2) over the next week. This work has been supported by Dell/EMC's Isilon product division and Netflix. You can see some discussion of these changes here: https://reviews.freebsd.org/D13403 https://reviews.freebsd.org/D13289 https://reviews.freebsd.org/D13545 The work has been done in user/jeff/numa if you want to look at svn history or experiment with the branch. It has been tested by Peter Holm on i386 and amd64 and it has been verified to work on arm at various points. We are working towards compatibility with libnuma and linux mbind. These commits will bring in improved support for NUMA in the kernel. There are new domain specific allocation functions available to kernel for UMA, malloc, kmem, and vmpage*. busdmamem consumers will automatically be placed in the correct domain, bringing automatic improvements to some device performance. cpuset will be able to constrains processes, groups of processes, jails, etc. to subsets of the system memory domains, just as it can with sets of cpus. It can set default policy for any of the above. Threads can use cpusets to set policy that specifies a subset of their visible domains. Available policies are first-touch (local in linux terms), round-robin (similar to linux interleave), and preferred. For now, the default is round-robin. You can achieve a fixed domain policy by using round-robin with a bitmask of a single domain. As the scheduler and VM become more sophisticated we may switch the default to first-touch as linux does. Currently these features are enabled with VMNUMAALLOC and MAXMEMDOM. It will eventually be NUMA/MAXMEMDOM to match SMP/MAXCPU. The current NUMA syscalls and VMNUMAALLOC code was 'experimental' and will be deprecated. numactl will continue to be supported although cpuset should be preferred going forward as it supports the full feature set of the new API. Thank you for your patience as I deal with the inevitable fallout of such sweeping changes. If you do have bugs, please file them in bugzilla, or reach out to me directly. I don't always have time to catch up on all of my mailing list mail and regretfully things slip through the cracks when they are not addressed directly to me. Thanks, Jeff Stack pointer checking – OpenBSD (https://marc.info/?l=openbsd-tech&m=151572838911297&w=2) Stefan (stefan@) and I have been working for a few months on this diff, with help from a few others. At every trap and system call, it checks if the stack-pointer is on a page that is marked MAPSTACK. execve() is changed to create such mappings for the process stack. Also, libpthread is taught the new MAPSTACK flag to use with mmap(). There is no corresponding system call which can set MAP_FLAG on an existing page, you can only set the flag by mapping new memory into place. That is a piece of the security model. The purpose of this change is to twart stack pivots, which apparently have gained some popularity in JIT ROP attacks. It makes it difficult to place the ROP stack in regular data memory, and then perform a system call from it. Workarounds are cumbersome, increasing the need for far more gadgetry. But also the trap case -- if any memory experiences a demand page fault, the same check will occur and potentially also kill the process. We have experimented a little with performing this check during device interrupts, but there are some locking concerns and performance may then become a concern. It'll be best to gain experience from handle of syncronous trap cases first. chrome and other applications I use run fine! I'm asking for some feedback to discover what ports this breaks, we'd like to know. Those would be ports which try to (unconventionally) create their stacks in malloc()'d memory or inside another Data structure. Most of them are probably easily fixed ... Qt 5.9 on FreeBSD (https://euroquis.nl/bobulate/?p=1768) Tobias and Raphael have spent the past month or so hammering on the Qt 5.9 branch, which has (finally!) landed in the official FreeBSD ports tree. This brings FreeBSD back up-to-date with current Qt releases and, more importantly, up-to-date with the Qt release KDE software is increasingly expecting. With Qt 5.9, the Elisa music player works, for instance (where it has run-time errors with Qt 5.7, even if it compiles). The KDE-FreeBSD CI system has had Qt 5.9 for some time already, but that was hand-compiled and jimmied into the system, rather than being a “proper” ports build. The new Qt version uses a new build system, which is one of the things that really slowed us down from a packaging perspective. Some modules have been reshuffled in the process. Some applications depending on Qt internal-private headers have been fixed along the way. The Telegram desktop client continues to be a pain in the butt that way. Following on from Qt 5.9 there has been some work in getting ready for Clang 6 support; in general the KDE and Qt stack is clean and modern C++, so it's more infrastructural tweaks than fixing code. Outside of our silo, I still see lots of wonky C++ code being fixed and plenty of confusion between pointers and integers and strings and chars and .. ugh. Speaking of ugh, I'm still planning to clean up Qt4 on ARM aarch64 for FreeBSD; this boils down to stealing suitable qatomic implementations from Arch Linux. For regular users of Qt applications on FreeBSD, there should be few to no changes required outside the regular upgrade cycle. For KDE Plasma users, note that development of the ports has changed branches; as we get closer to actually landing modern KDE bits, things have been renamed and reshuffled and mulled over so often that the old plasma5 branch wasn't really right anymore. The kde5-import branch is where it's at nowadays, and the instructions are the same: the x11/kde5 metaport will give you all the KDE Frameworks 5, KDE Plasma Desktop and modern KDE Applications you need. Adding IPv6 to an Nginx website on FreeBSD / FreshPorts (https://dan.langille.org/2018/01/13/adding-ipv6-to-an-nginx-website-on-freebsd-freshports/) FreshPorts recently moved to an IPv6-capable server but until today, that capability has not been utilized. There were a number of things I had to configure, but this will not necessarily be an exhaustive list for you to follow. Some steps might be missing, and it might not apply to your situation. All of this took about 3 hours. We are using: FreeBSD 11.1 Bind 9.9.11 nginx 1.12.2 Fallout I expect some monitoring fallout from this change. I suspect some of my monitoring assumes IP4 and now that IPv6 is available, I need to monitor both IP addresses. ZFS on TrueOS: Why We Love OpenZFS (https://www.trueos.org/blog/zfs-trueos-love-openzfs/) TrueOS was the first desktop operating system to fully implement the OpenZFS (Zettabyte File System or ZFS for short) enterprise file system in a stable production environment. To fully understand why we love ZFS, we will look back to the early days of TrueOS (formerly PC-BSD). The development team had been using the UFS file system in TrueOS because of its solid track record with FreeBSD-based computer systems and its ability to check file consistency with the built-in check utility fsck. However, as computing demands increased, problems began to surface. Slow fsck file verification on large file systems, slow replication speeds, and inconsistency in data integrity while using UFS logging / journaling began to hinder users. It quickly became apparent that TrueOS users would need a file system that scales with evolving enterprise storage needs, offers the best data protection, and works just as well on a hobbyist system or desktop computer. Kris Moore, the founder of the TrueOS project, first heard about OpenZFS in 2007 from chatter on the FreeBSD mailing lists. In 2008, the TrueOS development team was thrilled to learn that the FreeBSD Project had ported ZFS. At the time, ZFS was still unproven as a graphical desktop solution, but Kris saw a perfect opportunity to offer ZFS as a cutting-edge file system option in the TrueOS installer, allowing the TrueOS project to act as an indicator of how OpenZFS would fair in real-world production use. The team was blown away by the reception and quality of OpenZFS on FreeBSD-based systems. By its nature, ZFS is a copy-on-write (CoW) file system that won't move a block of data until it both writes the data and verifies its integrity. This is very different from most other file systems in use today. ZFS is able to assure that data stays consistent between writes by automatically comparing write checksums, which mitigates bit rot. ZFS also comes with native RaidZ functionality that allows for enterprise data management and redundancy without the need for expensive traditional RAID cards. ZFS snapshots allow for system configuration backups in a split-second. You read that right. TrueOS can backup or restore snapshots in less than a second using the ZFS file system. Given these advantages, the TrueOS team decided to use ZFS as its exclusive file system starting in 2013, and we haven't looked back since. ZFS offers TrueOS users the stable workstation experience they want, while simultaneously scaling to meet the increasing demands of the enterprise storage market. TrueOS users are frequently commenting on how easy it is to use ZFS snapshots with our built-in snapshot utility. This allows users the freedom to experiment with their system knowing they can restore it in seconds if anything goes wrong. If you haven't had a chance to try ZFS with TrueOS, browse to our download page and make sure to grab a copy of TrueOS. You'll be blown away by the ease of use, data protection functionality, and incredible flexibility of RaidZ. Beastie Bits Source Code Podcast Interview with Michael W Lucas (https://blather.michaelwlucas.com/archives/3099) Operating System of the Year 2017: NetBSD Third place (https://w3techs.com/blog/entry/web_technologies_of_the_year_2017) OPNsense 18.1-RC1 released (https://opnsense.org/opnsense-18-1-rc1-released/) Personal OpenBSD Wiki Notes (https://balu-wiki.readthedocs.io/en/latest/security/openbsd.html) BSD section can use some contribution (https://guide.freecodecamp.org/bsd-os/) The Third Research Edition Unix Programmer's Manual (now available in PDF) (https://github.com/dspinellis/unix-v3man) Feedback/Questions Alex - my first freebsd bug (http://dpaste.com/3DSV7BC#wrap) John - Suggested Speakers (http://dpaste.com/2QFR4MT#wrap) Todd - Two questions (http://dpaste.com/2FQ450Q#wrap) Matthew - CentOS to FreeBSD (http://dpaste.com/3KA29E0#wrap) Brian - Brian - openbsd 6.2 and enlightenment .17 (http://dpaste.com/24DYF1J#wrap) ***

TrueOS stable 17.12 is out, we have an OpenBSD workstation guide for you, learnings from the PDP-11, FreeBSD 2017 Releng recap and Duo SSH. This episode was brought to you by Headlines TrueOS stable release 17.12 (https://www.trueos.org/blog/trueos-17-12-release/) We are pleased to announce a new release of the 6-month STABLE version of TrueOS! This release cycle focused on lots of cleanup and stabilization of the distinguishing features of TrueOS: OpenRC, boot speed, removable-device management, SysAdm API integrations, Lumina improvements, and more. We have also been working quite a bit on the server offering of TrueOS, and are pleased to provide new text-based server images with support for Virtualization systems such as bhyve! This allows for simple server deployments which also take advantage of the TrueOS improvements to FreeBSD such as: Sane service management and status reporting with OpenRC Reliable, non-interactive system update mechanism with fail-safe boot environment support. Graphical management of remote TrueOS servers through SysAdm (also provides a reliable API for administrating systems remotely). LibreSSL for all base SSL support. Base system managed via packages (allows for additional fine-tuning). Base system is smaller due to the removal of the old GCC version in base. Any compiler and/or version may be installed and used via packages as desired. Support for newer graphics drivers and chipsets (graphics, networking, wifi, and more) TrueOS Version 17.12 (2017, December) is now available for download from the TrueOS website. Both the STABLE and UNSTABLE package repositories have also been updated in-sync with each other, so current users only need to follow the prompts about updating their system to run the new release. We are also pleased to announce the availability of TrueOS Sponsorships! If you would like to help contribute to the project financially we now have the ability to accept both one-time donations as well as recurring monthly donations which wil help us advocate for TrueOS around the world. Thank you all for using and supporting TrueOS! Notable Changes: Over 1100 OpenRC services have been created for 3rd-party packages. This should ensure the functionality of nearly all available 3rd-party packages that install/use their own services. The OpenRC services for FreeBSD itself have been overhauled, resulting in significantly shorter boot times. Separate install images for desktops and servers (server image uses a text/console installer) Bhyve support for TrueOS Server Install FreeBSD base is synced with 12.0-CURRENT as of December 4th, 2017 (Github commit: 209d01f) FreeBSD ports tree is synced as of November 30th (pre-FLAVOR changes) Lumina Desktop has been updated/developed from 1.3.0 to 1.4.1 PCDM now supports multiple simultaneous graphical sessions Removable devices are now managed through the “automounter” service. Devices are “announced” as available to the system via *.desktop shortcuts in /media. These shortcuts also contain a variety of optional “Actions” that may be performed on the device. Devices are only mounted while they are being used (such as when browsing via the command line or a file manager). Devices are automatically unmounted as soon as they stop being accessed. Integrated support for all major filesystems (UFS, EXT, FAT, NTFS, ExFAT, etc..) NOTE: The Lumina desktop is the only one which supports this functionality at the present time. The TrueOS update system has moved to an “active” update backend. This means that the user will need to actually start the update process by clicking the “Update Now” button in SysAdm, Lumina, or PCDM (as well as the command-line option). The staging of the update files is still performed automatically by default but this (and many other options) can be easily changed in the “Update Manager” settings as desired. Known Errata: [VirtualBox] Running FreeBSD within a VirtualBox VM is known to occasionally receive non-existent mouse clicks – particularly when using a scroll wheel or two-finger scroll. Quick Links: TrueOS Forums (https://discourse.trueos.org/) TrueOS Bugs (https://github.com/trueos/trueos-core/issues) TrueOS Handbook (https://www.trueos.org/handbook/trueos.html) TrueOS Community Chat on Telegram (https://t.me/TrueOSCommunity) *** OpenBSD Workstation Guide (https://begriffs.com/posts/2017-05-17-linux-workstation-guide.html) Design Goals User actions should complete instantaneously. While I understand if compiling code and rendering videos takes time, opening programs and moving windows should have no observable delay. The system should use minimalist tools. Corollary: cache data offline when possible. Everything from OpenStreetMaps to StackExchange can be stored locally. No reason to repeatedly hit the internet to query them. This also improves privacy because the initial download is indiscriminate and doesn't reveal personal queries or patterns of computer activity. No idling program should use a perceptible amount of CPU. Why does CalendarAgent on my Macbook sometimes use 150% CPU for fifteen minutes? Who knows. Why are background ChromeHelpers chugging along at upper-single-digit CPU? I didn't realize that holding a rendered DOM could be so challenging. Avoid interpreted languages, web-based desktop apps, and JavaScript garbage. There, I said it. Take your Electron apps with you to /dev/null! Stability. Old fashioned programs on a conservative OS on quality mainstream hardware. There are enough challenges to tackle without a bleeding edge system being one of them. Delegate to quality hardware components. Why use a janky ncurses software audio mixer when you can use…an actual audio mixer? Hardware privacy. No cameras or microphones that I can't physically disconnect. Also real hardware protection for cryptographic keys. Software privacy. Commercial software and operating systems have gotten so terrible about this. I even catch Mac command line tools trying to call Google Analytics. Sorry homebrew, your cute emojis don't make up for the surveillance. The Hardware Core To get the best hardware for the money I'm opting for a desktop computer. Haven't had one since the early 2000s and it feels anachronistic, but it will outperform a laptop of similar cost. After much searching, I found the HP Z240 Tower Workstation. It's no-nonsense and supports exactly the customizations I was looking for: No operating system pre-loaded (Cut out the “Windows tax”) Intel Xeon E3-1270 v6 processor (Supports ECC ram) 16 GB (2x8 GB) DDR4-2400 ECC Unbuffered memory (2400Mhz is the full memory clock speed supported by the Xeon) 256 GB HP Z Turbo Drive G2 PCIe SSD (Uses NVMe rather than SATA for faster throughput, supported by nvme(4)) No graphics card (We'll add our own) Intel® Ethernet I210-T1 PCIe (Supported by em(4)) A modest discrete video card will enable 2D Glamor acceleration on X11. The Radeon HD 6450 (sold separately) is fanless and listed as supported by radeon(4). Why build a solid computer and not protect it? Externally, the APC BR1300G UPS will protect the system from power surges and abrupt shutdowns. Peripherals The Matias Ergo Pro uses mechanical switches for that old fashioned clicky sound. It also includes dedicated buttons along the side for copying and pasting. Why is that cool? Well, it improves secondary selection, a technique that Sun computers used but time forgot. Since we're talking about a home office workstation, you may want a printer. The higher quality printers speak PostScript and PDF natively. Unix machines connect to them on TCP port 9100 and send PostScript commands directly. (You can print via telnet if you know the commands!) The Brother HL-L5100DN is a duplex LaserJet which allows that “raw” TCP printing. Audio/Video I know a lot of people enjoy surrounding themselves with a wall of monitors like they're in the heart of NASA Mission Control, but I find multi-monitor setups slightly disorienting. It introduces an extra bit of cognitive overhead to determine which monitor is for what exactly. That's why I'd go with a modest, crisp Dell UltraSharp 24" U2417H. It's 1080p and yeah there are 4k monitors nowadays, but text and icons are small enough as it is for me! If I ever considered a second monitor it would be e-ink for comfortably reading electronic copies of books or long articles. The price is currently too high to justify the purchase, but the most promising monitor seems to be the Dasung Paperlike. In the other direction, video input, it's more flexible to use a general-purpose HDMI capture box like the Rongyuxuan than settle on a particular webcam. This allows hooking up a real camera, or any other video device. Although the motherboard for this system has built-in audio, we should use a card with better OpenBSD support. The WBTUO PCIe card uses a C-Media CMI8768 chipset, handled by cmpci(4). The card provides S/PDIFF in and out ports if you ever want to use an external DAC or ADC. The way to connect it with other things is with a dedicated hardware mixer. The Behringer Xenyx 802 has all the connections needed, and the ability to route audio to and from the computer and a variety of devices at once. The mixer may seem an odd peripheral, but I want to mix the computer with an old fashioned CD player, ham radio gear, and amplifier so this unifies the audio setup. When doing remote pair programming or video team meetings it's nice to have a quality microphone. The best ones for this kind of work are directional, with a cardioid reception pattern. The MXL 770 condenser mic is perfect, and uses a powered XLR connection supplied by the mixer. Backups We're going dead simple and old-school, back to tapes. There are a set of tape standards called LTO-n. As n increases the tape capacity gets bigger, but the tape drive gets more expensive. In my opinion the best balance these days for the home user is LTO-3. You can usually find an HP Ultrium 960 LTO-3 on eBay for 150 dollars. The cartridges hold 800GB and are about 15 dollars apiece. Hard drives keep coming down in price, but these tapes are very cheap and simpler than keeping a bunch of disk drives. Also tape has proven longevity, and good recoverability. To use old fashioned tech like this you need a SCSI host bus adapter like the Adaptec 29320LPE, supported by ahd(4). Cryptography You don't want to generate and store secret keys on a general purpose network attached computer. The attack surface is a mile wide. Generating or manipulating “offline” secret keys needs to happen on a separate computer with no network access. Little boards like the Raspberry Pi would be good except they use ARM processors (incompatible with Tails OS) and have wifi. The JaguarBoard is a small x86 machine with no wireless capability. Just switch the keyboard and monitor over to this machine for your “cleanroom.” jaguar board: Generating keys requires entropy. The Linux kernel on Tails samples system properties to generate randomness, but why not help it out with a dedicated true random number generator (TRNG)? Bit Babbler supplies pure randomness at a high bitrate through USB. (OneRNG works better on the OpenBSD main system, via uonerng(4).) bit babbler: This little computer will save its results onto a OpenPGP Smartcard V2.1. This card provides write-only access to keys, and computes cryptographic primitives internally to sign and encrypt messages. To use it with a regular computer, hook up a Cherry ST2000 card reader. This reader has a PIN pad built in, so no keylogger on the main computer could even obtain your decryption PIN. The Software We take the beefed up hardware above and pair it with ninja-fast software written in C. Some text-based, others raw X11 graphical apps unencumbered by ties to any specific window manager. I'd advise OpenBSD for the underlying operating system, not a Linux. OpenBSD has greater internal consistency, their man pages are impeccable, and they make it a priority to prune old code to keep the system minimal. What Have We Learned from the PDP-11? (https://dave.cheney.net/2017/12/04/what-have-we-learned-from-the-pdp-11) The paper I have chosen tonight is a retrospective on a computer design. It is one of a series of papers by Gordon Bell, and various co-authors, spanning the design, growth, and eventual replacement of the companies iconic line of PDP-11 mini computers. This year represents the 60th anniversary of the founding of the company that produced the PDP-11. It is also 40 years since this paper was written, so I thought it would be entertaining to review Bell's retrospective through the lens of our own 20/20 hindsight. To set the scene for this paper, first we should talk a little about the company that produced the PDP-11, the Digital Equipment Corporation of Maynard, Massachusetts. Better known as DEC. It's also worth noting that the name PDP is an acronym for “Programmed Data Processor”, as at the time, computers had a reputation of being large, complicated, and expensive machines, and DEC's venture capitalists would not support them if they built a “computer” A computer is not solely determined by its architecture; it reflects the technological, economic, and human aspects of the environment in which it was designed and built. […] The finished computer is a product of the total design environment. “Right from the get go, Bell is letting us know that the success of any computer project is not abstractly building the best computer but building the right computer, and that takes context.” It is the nature of computer engineering to be goal-oriented, with pressure to produce deliverable products. It is therefore difficult to plan for an extensive lifetime. Because of the open nature of the PDP-11, anything which interpreted the instructions according to the processor specification, was a PDP-11, so there had been a rush within DEC, once it was clear that the PDP-11 market was heating up, to build implementations; you had different groups building fast, expensive ones and cost reduced slower ones The first weakness of minicomputers was their limited addressing capability. The biggest (and most common) mistake that can be made in a computer design is that of not providing enough address bits for memory addressing and management. A second weakness of minicomputers was their tendency not to have enough registers. This was corrected for the PDP-11 by providing eight 16-bit registers. Later, six 32-bit registers were added for floating-point arithmetic. […] More registers would increase the multiprogramming context switch time and confuse the user. “It's also interesting to note Bell's concern that additional registers would confuse the user. In the early 1970's the assumption that the machine would be programmed directly in assembly was still the prevailing mindset.” A third weakness of minicomputers was their lack of hardware stack capability. In the PDP-11, this was solved with the autoincrement/autodecrement addressing mechanism. This solution is unique to the PDP-11 and has proven to be exceptionally useful. (In fact, it has been copied by other designers.) “Nowadays it's hard to imagine hardware that doesn't have a notion of a stack, but consider that a stack isn't important if you don't need recursion.” “The design for the PDP-11 was laid down in 1969 and if we look at the programming languages of the time, FORTRAN and COBOL, neither supported recursive function calls. The function call sequence would often store the return address at a blank word at the start of the procedure making recursion impossible.” A fourth weakness, limited interrupt capability and slow context switching, was essentially solved with the device of UNIBUS interrupt vectors, which direct device interrupts. The basic mechanism is very fast, requiring only four memory cycles from the time an interrupt request is issued until the first instruction of the interrupt routine begins execution. A fifth weakness of prior minicomputers, inadequate character-handling capability, was met in the PDP-11 by providing direct byte addressing capability. “Strings and character handling were of increasing importance during the 1960's as scientific and business computing converged. The predominant character encodings at the time were 6 bit character sets which provided just enough space for upper case letters, the digits 0 to 9, space, and a few punctuation characters sufficient for printing financial reports.” “Because memory was so expensive, placing one 6 bit character into a 12 or 18 bit word was simply unacceptable so characters would be packed into words. This proved efficient for storage, but complex for operations like move, compare, and concatenate, which had to account for a character appearing in the top or bottom of the word, expending valuable words of program storage to cope.” “The problem was addressed in the PDP-11 by allowing the machine to operate on memory as both a 16-bit word, and the increasingly popular 8-bit byte. The expenditure of 2 additional bits per character was felt to be worth it for simpler string handling, and also eased the adoption of the increasingly popular 7-bit ASCII standard of which DEC were a proponent at the time. Bell concludes this point with the throw away line:” Although string instructions are not yet provided in the hardware, the common string operations (move, compare, concatenate) can be programmed with very short loops. A sixth weakness, the inability to use read-only memories, was avoided in the PDP-11. Most code written for the PDP-11 tends to be pure and reentrant without special effort by the programmer, allowing a read-only memory (ROM) to be used directly. A seventh weakness, one common to many minicomputers, was primitive I/O capabilities. A ninth weakness of minicomputers was the high cost of programming them. Many users program in assembly language, without the comfortable environment of editors, file systems, and debuggers available on bigger systems. The PDP-11 does not seem to have overcome this weakness, although it appears that more complex systems are being built successfully with the PDP-11 than with its predecessors, the PDP-8 and PDP-15. The problems faced by computer designers can usually be attributed to one of two causes: inexperience or second-systemitis Before the PDP-11, there was no UNIX. Before the PDP-11, there was no C, this is the computer that C was designed on. If you want to know why the classical C int is 16 bits wide, it's because of the PDP-11. UNIX bought us ideas such as pipes, everything is a file, and interactive computing. UNIX, which had arrived at Berkley in 1974 aboard a tape carried by Ken Thompson, would evolve into the west coast flavoured Berkley Systems Distribution. Berkeley UNIX had been ported to the VAX by the start of the 1980's and was thriving as the counter cultural alternative to DEC's own VMS operating system. Berkeley UNIX spawned a new generation of hackers who would go on to form companies like Sun micro systems, and languages like Self, which lead directly to the development of Java. UNIX was ported to a bewildering array of computer systems during the 80's and the fallout from the UNIX wars gave us the various BSD operating systems who continue to this day. The article, and the papers it is summarizing, contain a lot more than we could possibly dig into even if we dedicated the entire show to the topic *** News Roundup Two-factor authentication SSH with Duo in FreeBSD 11 (https://www.teachnix.com/2017/11/29/configuring-two-factor-authentication-on-freebsd-with-duo/) This setup uses an SSH key as the first factor of authentication. Please watch Part 1 on setting up SSH keys and how to scp it to your server. Video guide (https://www.youtube.com/watch?v=E5EuvF-iaV0) Register for a free account at Duo.com Install the Duo package on your FreeBSD server pkg install -y duo Log into the Duo site > Applications > Protect an Application > Search for Unix application > Protect this Application This will generate the keys we need to configure Duo. Edit the Duo config file using the course notes template vi /usr/local/etc/pam_duo.conf Example config [duo] ; Duo integration key ikey = Integration key goes here ; Duo secret key skey = Secret key goes here ; Duo API host host = API hostname goes here Change the permissions of the Duo config file. If the permissions are not correct then the service will not function properly. chmod 600 /usr/local/etc/pam_duo.conf Edit the SSHD config file using the course notes template vi /etc/ssh/sshd_config Example config ListenAddress 0.0.0.0 Port 22 PasswordAuthentication no UsePAM yes ChallengeResponseAuthentication yes UseDNS no PermitRootLogin yes AuthenticationMethods publickey,keyboard-interactive Edit PAM to configure SSHD for Duo using the course notes template Example config ``` # auth auth sufficient pamopie.so nowarn nofakeprompts auth requisite pamopieaccess.so nowarn allowlocal auth required /usr/local/lib/security/pamduo.so # session # session optional pamssh.so wantagent session required pam_permit.so # password # password sufficient pamkrb5.so nowarn tryfirstpass password required pamunix.so nowarn tryfirstpass ``` Restart the sshd service service sshd restart SSH into your FreeBSD server and follow the link it outputs to enroll your phone with Duo. ssh server.example.com SSH into your server again ssh server.example.com Choose your preferred method and it should log you into your server. FreeBSD 2017 Release Engineering Recap (https://www.freebsdfoundation.org/blog/2017-release-engineering-recap/) This past year was undoubtedly a rather busy and successful year for the Release Engineering Team. Throughout the year, development snapshot builds for FreeBSD-CURRENT and supported FreeBSD-STABLE branches were continually provided. In addition, work to package the base system using pkg(8) continued throughout the year and remains ongoing. The FreeBSD Release Engineering Team worked on the FreeBSD 11.1-RELEASE, with the code slush starting mid-May. The FreeBSD 11.1-RELEASE cycle stayed on schedule, with the final release build starting July 21, and the final release announcement following on July 25, building upon the stability and reliability of 11.0-RELEASE. Milestones during the 11.1-RELEASE cycle can be found on the 11.1 schedule page (https://www.freebsd.org/releases/11.1R/schedule.html). The final announcement is available here (https://www.freebsd.org/releases/11.1R/announce.html). The FreeBSD Release Engineering Team started the FreeBSD 10.4-RELEASE cycle, led by Marius Strobl. The FreeBSD 10.4-RELEASE cycle continued on schedule, with the only adjustments to the schedule being the addition of BETA4 and the removal of RC3. FreeBSD 10.4-RELEASE builds upon the stability and reliability of FreeBSD 10.3-RELEASE, and is planned to be the final release from the stable/10 branch. Milestones during the 10.4-RELEASE cycle can be found on the 10.4 schedule page (https://www.freebsd.org/releases/10.4R/schedule.html). The final announcement is available here (https://www.freebsd.org/releases/10.4R/announce.html). In addition to these releases, support for additional arm single-board computer images were added, notably Raspberry Pi 3 and Pine64. Additionally, release-related documentation effective 12.0-RELEASE and later has been moved from the base system repository to the documentation repository, making it possible to update related documentation as necessary post-release. Additionally, the FreeBSD Release Engineering article in the Project Handbook had been rewritten to outline current practices used by the Release Engineering Team. For more information on the procedures and processes the FreeBSD Release Engineering Team follows, the new article is available here and continually updated as procedures change. Finally, following the availability of FreeBSD 11.1-RELEASE, Glen Barber attended the September Developer Summit hosted at vBSDCon in Reston, VA, USA, where he gave a brief talk comprising of several points relating directly to the 11.1-RELEASE cycle. In particular, some of the points covered included what he felt went well during the release cycle, what did not go as well as it could have, and what we, as a Project, could do better to improve the release process. The slides from the talk are available in the FreeBSD Wiki. During the question and answer time following the talk, some questions asked included: Q: Should developers use the ‘Relnotes' tag in the Subversion commit template more loosely, at risk of an increase in false positives. A: When asked when the tag in the template was initially added, the answer would have been “no”, however in hindsight it is easier to sift through the false positives, than to comb through months or years of commit logs. Q: What issues are present preventing moving release-related documentation to the documentation repository? A: There were some rendering issues last time it was investigated, but it is really nothing more than taking the time to fix those issues. (Note, that since this talk, the migration of the documentation in question had moved.) Q: Does it make sense to extend the timeframe between milestone builds during a release cycle from one week to two weeks, to allow more time for testing, for example, RC1 versus RC2? A: No. It would extend the length of the release cycle with no real benefit between milestones since as we draw nearer to the end of a given release cycle, the number of changes to that code base significantly reduce. FLIMP - GIMP Exploit on FreeBSD (https://flimp.fuzzing-project.org) In 2014, when starting the Fuzzing Project (https://fuzzing-project.org/), Hanno Böck did some primitive fuzzing on GIMP and reported two bugs. They weren't fixed and were forgotten in the public bug tracker. Recently Tobias Stöckmann found one of these bugs (https://bugzilla.gnome.org/show_bug.cgi?id=739133) (CVE-2017-17785) and figured out that it's easy to exploit. What kind of bug is that? It's a classic heap buffer overflow in the FLIC parser. FLIC is a file format for animations and was introduced by Autodesk Animator. How does the exploit work? Tobias has created a detailed writeup (https://flimp.fuzzing-project.org/exploit.html). The exploit doesn't work for me! We figured out it's unreliable and the memory addresses are depending on many circumstances. The exploit ZIP comes with two variations using different memory addresses. Try both of them. We also noticed putting the files in a subdirectory sometimes made the exploit work. Anything more to tell about the GIMP? There's a wide variety of graphics formats. GIMP tries to support many of them, including many legacy formats that nobody is using any more today. While this has obvious advantages - you can access the old images you may find on a backup CD from 1995 - it comes with risks. Support for many obscure file formats means many parsers that hardly anyone ever looks at. So... what about the other parsers? The second bug (https://bugzilla.gnome.org/show_bug.cgi?id=739134) (CVE-2017-17786), which is a simple overread, was in the TGA parser. Furthermore we found buffer overreads in the XCF parser (https://bugzilla.gnome.org/show_bug.cgi?id=790783) (CVE-2017-17788), the Gimp Brush (GBR) parser (https://bugzilla.gnome.org/show_bug.cgi?id=790784) (CVE-2017-17784) and the Paint Shop Pro (PSP) parser (https://bugzilla.gnome.org/show_bug.cgi?id=790849) (CVE-2017-17789). We found another Heap buffer overflow (https://bugzilla.gnome.org/show_bug.cgi?id=790849) in the Paint Shop Pro parser (CVE-2017-17787) which is probably also exploitable. In other words: The GIMP import parsers are full of memory safety bugs. What should happen? First of all obviously all known memory safety bugs should be fixed. Furthermore we believe the way GIMP plugins work is not ideal for security testing. The plug-ins are separate executables, however they can't be executed on their own, as they communicate with the main GIMP process. Ideally either these plug-ins should be changed in a way that allows running them directly from the command line or - even better - they should be turned into libraries. The latter would also have the advantage of making the parser code useable for other software projects. Finally it might be a good idea to sandbox the import parsers. Dell FS12-NV7 Review – Bargain FreeBSD/ZFS box (http://blog.frankleonhardt.com/2017/dell-fs12-nv7-review-bargain-freebsdzfs-box/) It seems just about everyone selling refurbished data centre kit has a load of Dell FS12-NV7's to flog. Dell FS-what? You won't find them in the Dell catalogue, that's for sure. They look a bit like C2100s of some vintage, and they have a lot in common. But on closer inspection they're obviously a “special” for an important customer. Given the number of them knocking around, it's obviously a customer with big data, centres stuffed full of servers with a lot of processing to do. Here's a hint: It's not Google or Amazon. So, should you be buying a weirdo box with no documentation whatsoever? I'd say yes, definitely. If you're interests are anything like mine. In a 2U box you can get twin 4-core CPUs and 64Gb of RAM for £150 or less. What's not to like? Ah yes, the complete lack of documentation. Over the next few weeks I intend to cover that. And to start off this is my first PC review for nearly twenty years. As I mentioned, it's a 2U full length heavy metal box on rails. On the back there are the usual I/O ports: a 9-way RS-232, VGA, two 1Gb Ethernet, two USB2 and a PS/2 keyboard and mouse. The front is taken up by twelve 3.5″ hard drive bays, with the status lights and power button on one of the mounting ears to make room. Unlike other Dell servers, all the connections are on the back, only. So, in summary, you're getting a lot for your money if its the kind of thing you want. It's ideal as a high-performance Unix box with plenty of drive bays (preferably running BSD and ZFS). In this configuration it really shifts. Major bang-per-buck. Another idea I've had is using it for a flight simulator. That's a lot of RAM and processors for the money. If you forego the SAS controllers in the PCIe slots and dump in a decent graphics card and sound board, it's hard to see what's could be better (and you get jet engine sound effects without a speaker). So who should buy one of these? BSD geeks is the obvious answer. With a bit of tweaking they're a dream. It can build-absolutely-everything in 20-30 minutes. For storage you can put fast SAS drives in and it goes like the wind, even at 3Gb bandwidth per drive. I don't know if it works with FreeNAS but I can't see why not – I'm using mostly FreeBSD 11.1 and the generic kernel is fine. And if you want to run a load of weird operating systems (like Windows XP) in VM format, it seems to work very well with the Xen hypervisor and Dom0 under FreeBSD. Or CentOS if you prefer. So I shall end this review in true PCW style: Pros: Cheap Lots of CPUs, Lots of RAM Lots of HD slots Great for BSD/ZFS or VMs Cons: Noisy no AES-NI SAS needs upgrading Limited PCI slots As I've mentioned, the noise and SAS are easy and relatively cheap to fix, and thanks to BitCoin miners, even the PCI slot problem can be sorted. I'll talk about this in a later post. Beastie Bits Reflections on Hackathons (https://undeadly.org/cgi?action=article;sid=20171126090055) 7-Part Video Crash Course on SaltStack For FreeBSD (https://www.youtube.com/watch?v=HijG0hWebZk&list=PL5yV8umka8YQOr1wm719In5LITdGzQMOF) The LLVM Thread Sanitizer has been ported to NetBSD (https://blog.netbsd.org/tnf/entry/the_llvm_thread_sanitizer_has) The First Unix Port (1998) (http://bitsavers.informatik.uni-stuttgart.de/bits/Interdata/32bit/unix/univWollongong_v6/miller.pdf) arm64 platform now officially supported [and has syspatch(8)] (https://undeadly.org/cgi?action=article;sid=20171208082238) BSDCan 2018 Call for Participation (https://www.freebsdfoundation.org/news-and-events/call-for-papers/bsdcan-2018-call-for-participation/) AsiaBSDCon 2018 Call for Papers (https://www.freebsdfoundation.org/news-and-events/call-for-papers/asiabsdcon-2018-call-for-papers/) *** Feedback/Questions Shawn - DragonFlyBSD vagrant images (http://dpaste.com/3PRPJHG#wrap) Ben - undermydesk (http://dpaste.com/0AZ32ZB#wrap) Ken - Conferences (http://dpaste.com/3E8FQC6#wrap) Ben - ssh keys (http://dpaste.com/0E4538Q#wrap) SSH Chaining (https://www.bsdnow.tv/tutorials/ssh-chaining) ***

We recap EuroBSDcon in Paris, tell the story behind a pf PR, and show you how to do screencasting with OpenBSD. This episode was brought to you by Headlines Recap of EuroBSDcon 2017 in Paris, France (https://2017.eurobsdcon.org) EuroBSDcon was held in Paris, France this year, which drew record numbers this year. With over 300 attendees, it was the largest BSD event I have ever attended, and I was encouraged by the higher than expected number of first time attendees. The FreeBSD Foundation held a board meeting on Wednesday afternoon with the members who were in Paris. Topics included future conferences (including a conference kit we can mail to people who want to represent FreeBSD) and planning for next year. The FreeBSD Devsummit started on Thursday at the beautiful Mozilla Office in Paris. After registering and picking up our conference bag, everyone gathered for a morning coffee with lots of handshaking and greeting. We then gathered in the next room which had a podium with microphone, screens as well as tables and chairs. After developers sat down, Benedict opened the devsummit with a small quiz about France for developers to win a Mogics Power Bagel (https://www.mogics.com/?page_id=3824). 45 developers participated and DES won the item in the end. After introductions and collecting topics of interest from everyone, we started with the Work in Progress (WIP) session. The WIP session had different people present a topic they are working on in 7 minute timeslots. Topics ranged from FreeBSD Forwarding Performance, fast booting options, and a GELI patch under review to attach multiple providers. See their slides on the FreeBSD wiki (https://wiki.freebsd.org/DevSummit/201709). After lunch, the FreeBSD Foundation gave a general update on staff and funding, as well as a more focused presentation about our partnership with Intel. People were interested to hear what was done so far and asked a few questions to the Intel representative Glenn Weinberg. After lunch, developers worked quietly on their own projects. The mic remained open and occasionally, people would step forward and gave a short talk without slides or motivated a discussion of common interest. The day concluded with a dinner at a nice restaurant in Paris, which allowed to continue the discussions of the day. The second day of the devsummit began with a talk about the CAM-based SDIO stack by Ilya Bakulin. His work would allow access to wifi cards/modules on embedded boards like the Raspberry Pi Zero W and similar devices as many of these are using SDIO for data transfers. Next up was a discussion and Q&A session with the FreeBSD core team members who were there (missing only Benno Rice, Kris Moore, John Baldwin, and Baptiste Daroussin, the latter being busy with conference preparations). The new FCP (FreeBSD community proposals) were introduced for those who were not at BSDCan this year and the hows and whys about it. Allan and I were asked to describe our experiences as new members of core and we encouraged people to run for core when the next election happens. After a short break, Scott Long gave an overview of the work that's been started on NUMA (Non-Uniform Memory Architecture), what the goals of the project are and who is working on it. Before lunch, Christian Schwarz presented his work on zrepl, a new ZFS replication solution he developed using Go. This sparked interest in developers, a port was started (https://reviews.freebsd.org/D12462) and people suggested to Christian that he should submit his talk to AsiaBSDcon and BSDCan next year. Benedict had to leave before lunch was done to teach his Ansible tutorial (which was well attended) at the conference venue. There were organized dinners, for those two nights, quite a feat of organization to fit over 100 people into a restaurant and serve them quickly. On Saturday, there was a social event, a river cruise down the Seine. This took the form of a ‘standing' dinner, with a wide selection of appetizer type dishes, designed to get people to walk around and converse with many different people, rather than sit at a table with the same 6-8 people. I talked to a much larger group of people than I had managed to at the other dinners. I like having both dinner formats. We would also like to thank all of the BSDNow viewers who attended the conference and made the point of introducing themselves to us. It was nice to meet you all. The recordings of the live video stream from the conference are available immediately, so you can watch the raw versions of the talks now: Auditorium Keynote 1: Software Development in the Age of Heroes (https://youtu.be/4iR8g9-39LM?t=179) by Thomas Pornin (https://twitter.com/BearSSLnews) Tuning FreeBSD for routing and firewalling (https://youtu.be/4iR8g9-39LM?t=1660) by Olivier Cochard-Labbé (https://twitter.com/ocochardlabbe) My BSD sucks less than yours, Act I (https://youtu.be/4iR8g9-39LM?t=7040) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_) My BSD sucks less than yours, Act II (https://youtu.be/4iR8g9-39LM?t=14254) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_) Reproducible builds on NetBSD (https://youtu.be/4iR8g9-39LM?t=23351) by Christos Zoulas Your scheduler is not the problem (https://youtu.be/4iR8g9-39LM?t=26845) by Martin Pieuchot Keynote 2: A French story on cybercrime (https://youtu.be/4iR8g9-39LM?t=30540) by Éric Freyssinet (https://twitter.com/ericfreyss) Case studies of sandboxing base system with Capsicum (https://youtu.be/jqdHYEH_BQY?t=731) by Mariusz Zaborski (https://twitter.com/oshogbovx) OpenBSD's small steps towards DTrace (a tale about DDB and CTF) (https://youtu.be/jqdHYEH_BQY?t=6030) by Jasper Lievisse Adriaanse The Realities of DTrace on FreeBSD (https://youtu.be/jqdHYEH_BQY?t=13096) by George Neville-Neil (https://twitter.com/gvnn3) OpenSMTPD, current state of affairs (https://youtu.be/jqdHYEH_BQY?t=16818) by Gilles Chehade (https://twitter.com/PoolpOrg) Hoisting: lessons learned integrating pledge into 500 programs (https://youtu.be/jqdHYEH_BQY?t=21764) by Theo de Raadt Keynote 3: System Performance Analysis Methodologies (https://youtu.be/jqdHYEH_BQY?t=25463) by Brendan Gregg (https://twitter.com/brendangregg) Closing Session (https://youtu.be/jqdHYEH_BQY?t=29355) Karnak “Is it done yet ?” The never ending story of pkg tools (https://youtu.be/1hjzleqGRYk?t=71) by Marc Espie (https://twitter.com/espie_openbsd) A Tale of six motherboards, three BSDs and coreboot (https://youtu.be/1hjzleqGRYk?t=7498) by Piotr Kubaj and Katarzyna Kubaj State of the DragonFly's graphics stack (https://youtu.be/1hjzleqGRYk?t=11475) by François Tigeot From NanoBSD to ZFS and Jails – FreeBSD as a Hosting Platform, Revisited (https://youtu.be/1hjzleqGRYk?t=16227) by Patrick M. Hausen Bacula – nobody ever regretted making a backup (https://youtu.be/1hjzleqGRYk?t=20069) by Dan Langille (https://twitter.com/DLangille) Never Lose a Syslog Message (https://youtu.be/qX0BS4P65cQ?t=325) by Alexander Bluhm Running CloudABI applications on a FreeBSD-based Kubernetes cluster (https://youtu.be/qX0BS4P65cQ?t=5647) by Ed Schouten (https://twitter.com/EdSchouten) The OpenBSD web stack (https://youtu.be/qX0BS4P65cQ?t=13255) by Michael W. Lucas (https://twitter.com/mwlauthor) The LLDB Debugger on NetBSD (https://youtu.be/qX0BS4P65cQ?t=16835) by Kamil Rytarowski What's in store for NetBSD 8.0? (https://youtu.be/qX0BS4P65cQ?t=21583) by Alistair Crooks Louxor A Modern Replacement for BSD spell(1) (https://youtu.be/6Nen6a1Xl7I?t=156) by Abhinav Upadhyay (https://twitter.com/abhi9u) Portable Hotplugging: NetBSD's uvm_hotplug(9) API development (https://youtu.be/6Nen6a1Xl7I?t=5874) by Cherry G. Mathew Hardening pkgsrc (https://youtu.be/6Nen6a1Xl7I?t=9343) by Pierre Pronchery (https://twitter.com/khorben) Discovering OpenBSD on AWS (https://youtu.be/6Nen6a1Xl7I?t=14874) by Laurent Bernaille (https://twitter.com/lbernail) OpenBSD Testing Infrastructure Behind bluhm.genua.de (https://youtu.be/6Nen6a1Xl7I?t=18639) by Jan Klemkow The school of hard knocks – PT1 (https://youtu.be/8wuW8lfsVGc?t=276) by Sevan Janiyan (https://twitter.com/sevanjaniyan) 7 years of maintaining firefox, and still looking ahead (https://youtu.be/8wuW8lfsVGc?t=5321) by Landry Breuil Branch VPN solution based on OpenBSD, OSPF, RDomains and Ansible (https://youtu.be/8wuW8lfsVGc?t=12385) by Remi Locherer Running BSD on AWS (https://youtu.be/8wuW8lfsVGc?t=15983) by Julien Simon and Nicolas David Getting started with OpenBSD device driver development (https://youtu.be/8wuW8lfsVGc?t=21491) by Stefan Sperling A huge thanks to the organizers, program committee, and sponsors of EuroBSDCon. Next year, EuroBSDcon will be in Bucharest, Romania. *** The story of PR 219251 (https://www.sigsegv.be//blog/freebsd/PR219251) The actual story I wanted Kristof to tell, the pf bug he fixed at the Essen Hackathon earlier this summer. As I threatened to do in my previous post, I'm going to talk about PR 219251 for a bit. The bug report dates from only a few months ago, but the first report (that I can remeber) actually came from Shawn Webb on Twitter, of all places Despite there being a stacktrace it took quite a while (nearly 6 months in fact) before I figured this one out. It took Reshad Patuck managing to distill the problem down to a small-ish test script to make real progress on this. His testcase meant that I could get core dumps and experiment. It also provided valuable clues because it could be tweaked to see what elements were required to trigger the panic. This test script starts a (vnet) jail, adds an epair interface to it, sets up pf in the jail, and then reloads the pf rules on the host. Interestingly the panic does not seem to occur if that last step is not included. Obviously not the desired behaviour, but it seems strange. The instances of pf in the jails are supposed to be separate. We try to fetch a counter value here, but instead we dereference a bad pointer. There's two here, so already we need more information. Inspection of the core dump reveals that the state pointer is valid, and contains sane information. The rule pointer (rule.ptr) points to a sensible location, but the data is mostly 0xdeadc0de. This is the memory allocator being helpful (in debug mode) and writing garbage over freed memory, to make use-after-free bugs like this one easier to find. In other words: the rule has been free()d while there was still a state pointing to it. Somehow we have a state (describing a connection pf knows about) which points to a rule which no longer exists. The core dump also shows that the problem always occurs with states and rules in the default vnet (i.e. the host pf instance), not one of the pf instances in one of the vnet jails. That matches with the observation that the test script does not trigger the panic unless we also reload the rules on the host. Great, we know what's wrong, but now we need to work out how we can get into this state. At this point we're going to have to learn something about how rules and states get cleaned up in pf. Don't worry if you had no idea, because before this bug I didn't either. The states keep a pointer to the rule they match, so when rules are changed (or removed) we can't just delete them. States get cleaned up when connections are closed or they time out. This means we have to keep old rules around until the states that use them expire. When rules are removed pfunlinkrule() adds then to the Vpfunlinkedrules list (more on that funny V prefix later). From time to time the pf purge thread will run over all states and mark the rules that are used by a state. Once that's done for all states we know that all rules that are not marked as in-use can be removed (because none of the states use it). That can be a lot of work if we've got a lot of states, so pfpurgethread() breaks that up into smaller chuncks, iterating only part of the state table on every run. We iterate over all of our virtual pf instances (VNETFOREACH()), check if it's active (for FreeBSD-EN-17.08, where we've seen this code before) and then check the expired states with pfpurgeexpiredstates(). We start at state 'idx' and only process a certain number (determined by the PFTMINTERVAL setting) states. The pfpurgeexpiredstates() function returns a new idx value to tell us how far we got. So, remember when I mentioned the odd V_ prefix? Those are per-vnet variables. They work a bit like thread-local variables. Each vnet (virtual network stack) keeps its state separate from the others, and the V_ variables use a pointer that's changed whenever we change the currently active vnet (say with CURVNETSET() or CURVNETRESTORE()). That's tracked in the 'curvnet' variable. In other words: there are as many Vpfvnetactive variables as there are vnets: number of vnet jails plus one (for the host system). Why is that relevant here? Note that idx is not a per-vnet variable, but we handle multiple pf instances here. We run through all of them in fact. That means that we end up checking the first X states in the first vnet, then check the second X states in the second vnet, the third X states in the third and so on and so on. That of course means that we think we've run through all of the states in a vnet while we really only checked some of them. So when pfpurgeunlinkedrules() runs it can end up free()ing rules that actually are still in use because pfpurgethread() skipped over the state(s) that actually used the rule. The problem only happened if we reloaded rules in the host, because the active ruleset is never free()d, even if there are no states pointing to the rule. That explains the panic, and the fix is actually quite straightforward: idx needs to be a per-vnet variable, Vpfpurge_idx, and then the problem is gone. As is often the case, the solution to a fairly hard problem turns out to be really simple. As you might expect, finding the problem takes a lot more work that fixing it Thanks to Kristof for writing up this detailed post explaining how the problem was found, and what caused it. *** vBSDcon 2017: BSD at Work (https://www.ixsystems.com/blog/vbsdcon-2017-dexter/) The third biennial vBSDcon hosted by Verisign took place September 7th through 9th with the FreeBSD Developer Summit taking place the first day. vBSDcon and iXsystems' MeetBSD event have been alternating between the East and West coasts of the U.S.A. and these two events play vital roles in reaching Washington, DC-area and Bay Area/Silicon Valley audiences. Where MeetBSD serves many BSD Vendors, vBSDcon attracts a unique government and security industry demographic that isn't found anywhere else. Conference time and travel budgets are always limited and bringing these events to their attendees is a much-appreciated service provided by their hosts. The vBSDcon FreeBSD DevSummit had a strong focus on OpenZFS, the build system and networking with the FreeBSD 12 wish list of features in mind. How to best incorporate the steady flow of new OpenZFS features into FreeBSD such as dataset-level encryption was of particular interest. This feature from a GNU/Linux-based storage vendor is tribute to the growth of the OpenZFS community which is vital in light of the recent “Death of Solaris and ZFS” at Oracle. There has never been more demand for OpenZFS on FreeBSD and the Oracle news further confirms our collective responsibility to meet that demand. The official conference opened with my talk on “Isolated BSD Build Environments” in which I explained how the bhyve hypervisor can be used to effortlessly tour FreeBSD 5.0-onward and build specific source releases on demand to trace regressions to their offending commit. I was followed by a FreeNAS user who made the good point that FreeNAS is an exemplary “entry vector” into Unix and Enterprise Storage fundamentals, given that many of the vectors our generation had are gone. Where many of us discovered Unix and the Internet via console terminals at school or work, smart phones are only delivering the Internet without the Unix. With some irony, both iOS and Android are Unix-based yet offer few opportunities for their users to learn and leverage their Unix environments. The next two talks were The History and Future of Core Dumps in FreeBSD by Sam Gwydir and Using pkgsrc for multi-platform deployments in heterogeneous environments by G. Clifford Williams. I strongly recommend that anyone wanting to speak at AsiaBSDCon read Sam's accompanying paper on core dumps because I consider it the perfect AsiaBSDCon topic and his execution is excellent. Core dumps are one of those things you rarely think about until they are a DROP EVERYTHING! priority. G. Clifford's talk was about what I consider a near-perfect BSD project: pkgsrc, the portable BSD package manager. I put it up there with OpenSSH and mandoc as projects that have provided significant value to other Open Source operating systems. G. Clifford's real-world experiences are perfectly inline with vBSDcon's goal to be more production-oriented than other BSDCons. Of the other talks, any and all Dtrace talks are always appreciated and George Neville-Neil's did not disappoint. He based it on his experiences with the Teach BSD project which is bringing FreeBSD-based computer science education to schools around the world. The security-related talks by John-Mark Gurney, Dean Freeman and Michael Shirk also represented vBSDcon's consideration of the local community and made a convincing point that the BSDs should make concerted efforts to qualify for Common Criteria, FIPS, and other Government security requirements. While some security experts will scoff at these, they are critical to the adoption of BSD-based products by government agencies. BSD Now hosts Allan Jude and Benedict Reuschling hosted an OpenZFS BoF and Ansible talk respectively and I hosted a bhyve hypervisor BoF. The Hallway Track and food at vBSDcon were excellent and both culminated with an after-dinner dramatic reading of Michael W. Lucas' latest book that raised money for the FreeBSD Foundation. A great time was had by all and it was wonderful to see everyone! News Roundup FreeBSD 10.4-RC2 Available (https://lists.freebsd.org/pipermail/freebsd-stable/2017-September/087848.html) FreeBSD 10.4 will be released soon, this is the last chance to find bugs before the official release is cut. Noteworthy Changes Since 10.4-RC1: Given that the amd64 disc1 image was overflowing, more of the base components installed into the disc1 (live) file systems had to be disabled. Most notably, this removed the compiler toolchain from the disc1 images. All disabled tools are still available with the dvd1 images, though. The aesni(4) driver now no longer shares a single FPU context across multiple sessions in multiple threads, addressing problems seen when employing aesni(4) for ipsec(4). Support for netmap(4) by the ixgbe(4) driver has been brought into line with the netmap(4) API present in stable/10. Also, ixgbe(4) now correctly handles VFs in its netmap(4) support again instead of treating these as PFs. During the creation of amd64 and i386 VM images, etcupdate(8) and mergemaster(8) databases now are bootstrapped, akin to what happens along the extraction of base.txz as part of a new installation via bsdinstall(8). This change allows for both of these tools to work out-of-box on the VM images and avoids errors seen when upgrading these images via freebsd-update(8). If you are still on the stable/10 branch, you should test upgrading to 10.4, and make sure there are no problems with your workload Additional testing specifically of the features that have changed since 10.4-BETA1 would also be most helpful This will be the last release from the stable/10 branch *** OpenBSD changes of note 628 (https://www.tedunangst.com/flak/post/openbsd-changes-of-note-628) EuroBSDCon in two weeks. Be sure to attend early and often. Many and various documentation improvements for libcrypto. New man pages, rewrites, expanded bugs sections, and more. Only allow upward migration in vmd. There's a README for the syspatch build system if you want to run your own. Move the kernel relinking code from /etc/rc into a seperate script usable by syspatch. Kernel patches can now be reduced to just the necessary files. Make the callers of sogetopt() responsible for allocating memory. Now allocation and free occur in the same place. Use waitpid() instead of wait() in most programs to avoid accidentally collecting the wrong child. Have cu call isatty() before making assumptions. Switch mandoc rendering of mathematical symbols and greek letters from trying to imitate the characters' graphical shapes, which resulted in unintelligible renderings in many cases, to transliterations conveying the characters' meanings. Update libexpat to 2.2.4. Fix copying partial UTF-8 characters. Sigh, here we go again. Work around bug in F5's handling of the supported elliptic curves extension. RFC 4492 only defines elliptic_curves for ClientHello. However, F5 is sending it in ServerHello. We need to skip over it since our TLS extension parsing code is now more strict. After a first install, run syspatch -c to check for patches. If SMAP is present, clear PSL_AC on kernel entry and interrupt so that only the code in copy{in,out}* that need it run with it set. Panic if it's set on entry to trap() or syscall(). Prompted by Maxime Villard's NetBSD work. Errata. New drivers for arm: rktemp, mvpinctrl, mvmpic, mvneta, mvmdio, mvpxa, rkiic, rkpmic. No need to exec rm from within mandoc. We know there's exactly one file and directory to remove. Similarly with running cmp. Revert to Mesa 13.0.6 to hopefully address rendering issues a handful of people have reported with xpdf/fvwm on ivy bridge with modesetting driver. Rewrite ALPN extension using CBB/CBS and the new extension framework. Rewrite SRTP extension using CBB/CBS and the new extension framework. Revisit 2q queue sizes. Limit the hot queue to 1/20th the cache size up to a max of 4096 pages. Limit the warm and cold queues to half the cache. This allows us to more effectively notice re-interest in buffers instead of losing it in a large hot queue. Add glass console support for arm64. Probably not yet for your machine, though. Replace heaps of hand-written syscall stubs in ld.so with a simpler framework. 65535 is a valid port to listen on. When xinit starts an X server that listens only on UNIX socket, prefer DISPLAY=unix:0 rather than DISPLAY=:0. This will prevent applications from ever falling back to TCP if the UNIX socket connection fails (such as when the X server crashes). Reverted. Add -z and -Z options to apmd to auto suspend or hibernate when low on battery. Remove the original (pre-IETF) chacha20-poly1305 cipher suites. Add urng(4) which supports various USB RNG devices. Instead of adding one driver per device, start bundling them into a single driver. Remove old deactivated pledge path code. A replacement mechanism is being brewed. Fix a bug from the extension parsing rewrite. Always parse ALPN even if no callback has been installed to prevent leaving unprocessed data which leads to a decode error. Clarify what is meant by syslog priorities being ordered, since the numbers and priorities are backwards. Remove a stray setlocale() from ksh, eliminating a lot of extra statically linked code. Unremove some NPN symbols from libssl because ports software thinks they should be there for reasons. Fix saved stack location after resume. Somehow clang changed it. Resume works again on i386. Improve error messages in vmd and vmctl to be more informative. Stop building the miniroot installer for OMAP3 Beagleboards. It hasn't worked in over a year and nobody noticed. Have the callers of sosetopt() free the mbuf for symmetry. On octeon, let the kernel use the hardware FPU even if emulation is compiled in. It's faster. Fix support for 486DX CPUs by not calling cpuid. I used to own a 486. Now I don't. Merge some drm fixes from linux. Defer probing of floppy drives, eliminating delays during boot. Better handling of probes and beacons and timeouts and scans in wifi stack to avoid disconnects. Move mutex, condvar, and thread-specific data routes, pthreadonce, and pthreadexit from libpthread to libc, along with low-level bits to support them. Let's thread aware (but not actually threaded) code work with just libc. New POSIX xlocale implementation. Complete as long as you only use ASCII and UTF-8, as you should. Round and round it goes; when 6.2 stops, nobody knows. A peak at the future? *** Screencasting with OpenBSD (http://eradman.com/posts/screencasting.html) USB Audio Any USB microphone should appear as a new audio device. Here is the dmesg for my mic by ART: uaudio0 at uhub0 port 2 configuration 1 interface 0 "M-One USB" rev 1.10/0.01 addr 2 uaudio0: audio rev 1.00, 8 mixer controls audio1 at uaudio0 audioctl can read off all of the specific characterisitcs of this device $ audioctl -f /dev/audio1 | grep record mode=play,record record.rate=48000 record.channels=1 record.precision=16 record.bps=2 record.msb=1 record.encoding=slinear_le record.pause=0 record.active=0 record.block_size=1960 record.bytes=0 record.errors=0 Now test the recording from the second audio device using aucat(1) aucat -f rsnd/1 -o file.wav If the device also has a headset audio can be played through the same device. aucat -f rsnd/1 -i file.wav Screen Capture using Xvfb The rate at which a framebuffer for your video card is a feature of the hardware and software your using, and it's often very slow. x11vnc will print an estimate of the banwidth for the system your running. x11vnc ... 09/05/2012 22:23:45 fb read rate: 7 MB/sec This is about 4fps. We can do much better by using a virtual framebuffer. Here I'm setting up a new screen, setting the background color, starting cwm and an instance of xterm Xvfb :1 -screen 0 720x540x16 & DISPLAY=:1 xsetroot -solid steelblue & DISPLAY=:1 cwm & DISPLAY=:1 xterm +sb -fa Hermit -fs 14 & Much better! Now we're up around 20fps. x11vnc -display :1 & ... 11/05/2012 18:04:07 fb read rate: 168 MB/sec Make a connection to this virtual screen using raw encoding to eliminate time wasted on compression. vncviewer localhost -encodings raw A test recording with sound then looks like this ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 ~/out.avi Note: always stop the recording and playback using q, not Ctrl-C so that audio inputs are shut down properly. Screen Capture using Xephyr Xephyr is perhaps the easiest way to run X with a shadow framebuffer. This solution also avoids reading from the video card's RAM, so it's reasonably fast. Xephyr -ac -br -noreset -screen 800x600 :1 & DISPLAY=:1 xsetroot -solid steelblue & DISPLAY=:1 cwm & DISPLAY=:1 xrdb -load ~/.Xdefaults & DISPLAY=:1 xterm +sb -fa "Hermit" -fs 14 & Capture works in exactally the same way. This command tries to maintain 12fps. ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi To capture keyboard and mouse input press Ctrl then Shift. This is very handy for using navigating a window manager in the nested X session. Arranging Windows I have sometimes found it helpful to launch applications and arrange them in a specific way. This will open up a web browser listing the current directory and position windows using xdotool DISPLAY=:1 midori "file:///pwd" & sleep 2 DISPLAY=:1 xdotool search --name "xterm" windowmove 0 0 DISPLAY=:1 xdotool search --class "midori" windowmove 400 0 DISPLAY=:1 xdotool search --class "midori" windowsize 400 576 This will position the window precisely so that it appears to be in a tmux window on the right. Audio/Video Sync If you find that the audio is way out of sync with the video, you can ajust the start using the -ss before the audio input to specify the number of seconds to delay. My final recording command line, that delays the audio by 0.5 seconds, writing 12fps ffmpeg -ss 0.5 -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi Sharing a Terminal with tmux If you're trying to record a terminal session, tmux is able to share a session. In this way a recording of an X framebuffer can be taken without even using the screen. Start by creating the session. tmux -2 -S /tmp/tmux0 Then on the remote side connect on the same socket tmux -2 -S /tmp/tmux0 attach Taking Screenshots Grabbing a screenshots on Xvfb server is easily accomplished with ImageMagick's import command DISPLAY=:1 import -window root screenshot.png Audio Processing and Video Transcoding The first step is to ensure that the clip begins and ends where you'd like it to. The following will make a copy of the recording starting at time 00:00 and ending at 09:45 ffmpeg -i interactive-sql.avi -vcodec copy -acodec copy -ss 00:00:00 -t 00:09:45 interactive-sql-trimmed.avi mv interactive-sql-trimmed.avi interactive-sql.avi Setting the gain correctly is very important with an analog mixer, but if you're using a USB mic there may not be a gain option; simply record using it's built-in settings and then adjust the levels afterwards using a utility such as normalize. First extact the audio as a raw PCM file and then run normalize ffmpeg -i interactive-sql.avi -c:a copy -vn audio.wav normalize audio.wav Next merge the audio back in again ffmpeg -i interactive-sql.avi -i audio.wav -map 0:0 -map 1:0 -c copy interactive-sql-normalized.avi The final step is to compress the screencast for distribution. Encoding to VP8/Vorbis is easy: ffmpeg -i interactive-sql-normalized.avi -c:v libvpx -b:v 1M -c:a libvorbis -q:a 6 interactive-sql.webm H.264/AAC is tricky. For most video players the color space needs to be set to yuv420p. The -movflags puts the index data at the beginning of the file to enable streaming/partial content requests over HTTP: ffmpeg -y -i interactive-sql-normalized.avi -c:v libx264 -preset slow -crf 14 -pix_fmt yuv420p -movflags +faststart -c:a aac -q:a 6 interactive-sql.mp4 TrueOS @ Ohio Linuxfest '17! (https://www.trueos.org/blog/trueos-ohio-linuxfest-17/) Dru Lavigne and Ken Moore are both giving presentations on Saturday the 30th. Sit in and hear about new developments for the Lumina and FreeNAS projects. Ken is offering Lumina Rising: Challenging Desktop Orthodoxy at 10:15 am in Franklin A. Hear his thoughts about the ideas propelling desktop environment development and how Lumina, especially Lumina 2, is seeking to offer a new model of desktop architecture. Elements discussed include session security, application dependencies, message handling, and operating system integration. Dru is talking about What's New in FreeNAS 11 at 2:00 pm in Franklin D. She'll be providing an overview of some of the new features added in FreeNAS 11.0, including: Alert Services Starting specific services at boot time AD Monitoring to ensure the AD service restarts if disconnected A preview of the new user interface support for S3-compatible storage and the bhyve hypervisor She's also giving a sneak peek of FreeNAS 11.1, which has some neat features: A complete rewrite of the Jails/Plugins system as FreeNAS moves from warden to iocage Writing new plugins with just a few lines of code A brand new asynchronous middleware API Who's going? Attending this year are: Dru Lavigne (dlavigne): Dru leads the technical documentation team at iX, and contributes heavily to open source documentation projects like FreeBSD, FreeNAS, and TrueOS. Ken Moore (beanpole134): Ken is the lead developer of Lumina and a core contributor to TrueOS. He also works on a number of other Qt5 projects for iXsystems. J.T. Pennington (q5sys): Some of you may be familiar with his work on BSDNow, but J.T. also contributes to the TrueOS, Lumina, and SysAdm projects, helping out with development and general bug squashing. *** Beastie Bits Lumina Development Preview: Theme Engine (https://www.trueos.org/blog/lumina-development-preview-theme-engine/) It's happening! Official retro Thinkpad lappy spotted in the wild (https://www.theregister.co.uk/2017/09/04/retro_thinkpad_spotted_in_the_wild/) LLVM libFuzzer and SafeStack ported to NetBSD (https://blog.netbsd.org/tnf/entry/llvm_libfuzzer_and_safestack_ported) Remaining 2017 FreeBSD Events (https://www.freebsdfoundation.org/news-and-events/event-calendar/2017-openzfs-developer-summit/) *** Feedback/Questions Andrew - BSD Teaching Material (http://dpaste.com/0YTT0VP) Seth - Switching to Tarsnap after Crashplan becomes no more (http://dpaste.com/1SK92ZX#wrap) Thomas - Native encryption in ZFS (http://dpaste.com/02KD5FX#wrap) Coding Cowboy - Coding Cowboy - Passwords and clipboards (http://dpaste.com/31K0E40#wrap) ***

We recap vBSDcon, give you the story behind a PF EN, reminisce in Solaris memories, and show you how to configure different DEs on FreeBSD. This episode was brought to you by Headlines [vBSDCon] vBSDCon was held September 7 - 9th. We recorded this only a few days after getting home from this great event. Things started on Wednesday night, as attendees of the thursday developer summit arrived and broke into smallish groups for disorganized dinner and drinks. We then held an unofficial hacker lounge in a medium sized seating area, working and talking until we all decided that the developer summit started awfully early tomorrow. The developer summit started with a light breakfast and then then we dove right in Ed Maste started us off, and then Glen Barber gave a presentation about lessons learned from the 11.1-RELEASE cycle, and comparing it to previous releases. 11.1 was released on time, and was one of the best releases so far. The slides are linked on the DevSummit wiki page (https://wiki.freebsd.org/DevSummit/20170907). The group then jumped into hackmd.io a collaborative note taking application, and listed of various works in progress and upstreaming efforts. Then we listed wants and needs for the 12.0 release. After lunch we broke into pairs of working groups, with additional space for smaller meetings. The first pair were, ZFS and Toolchain, followed by a break and then a discussion of IFLIB and network drivers in general. After another break, the last groups of the day met, pkgbase and secure boot. Then it was time for the vBSDCon reception dinner. This standing dinner was a great way to meet new people, and for attendees to mingle and socialize. The official hacking lounge Thursday night was busy, and included some great storytelling, along with a bunch of work getting done. It was very encouraging to watch a struggling new developer getting help from a seasoned veteran. Watching the new developers eyes light up as the new information filled in gaps and they now understood so much more than just a few minutes before, and they raced off to continue working, was inspirational, and reminded me why these conferences are so important. The hacker lounge shut down relatively early by BSD conference standards, but, the conference proper started at 8:45 sharp the next morning, so it made sense. Friday saw a string of good presentations, I think my favourite was Jonathan Anderson's talk on Oblivious sandboxing. Jonathan is a very energetic speaker, and was able to keep everyone focused even during relatively complicated explanations. Friday night I went for dinner at ‘Big Bowl', a stir-fry bar, with a largish group of developers and users of both FreeBSD and OpenBSD. The discussions were interesting and varied, and the food was excellent. Benedict had dinner with JT and some other folks from iXsystems. Friday night the hacker lounge was so large we took over a bigger room (it had better WiFi too). Saturday featured more great talks. The talk I was most interested in was from Eric McCorkle, who did the EFI version of my GELIBoot work. I had reviewed some of the work, but it was interesting to hear the story of how it happened, and to see the parallels with my own story. My favourite speaker was Paul Vixie, who gave a very interesting talk about the gets() function in libc. gets() was declared unsafe before the FreeBSD project even started. The original import of the CSRG code into FreeBSD includes the compile time, and run-time warnings against using gets(). OpenBSD removed gets() in version 5.6, in 2014. Following Paul's presentation, various patches were raised, to either cause use of gets() to crash the program, or to remove gets() entirely, causing such programs to fail to link. The last talk before the closing was Benedict's BSD Systems Management with Ansible (https://people.freebsd.org/~bcr/talks/vBSDcon2017_Ansible.pdf). Shortly after, Allan won a MacBook Pro by correctly guessing the number of components in a jar that was standing next to the registration desk (Benedict was way off, but had a good laugh about the unlikely future Apple user). Saturday night ended with the Conference Social, and excellent dinner with more great conversations On Sunday morning, a number of us went to the Smithsonian Air and Space Museum site near the airport, and saw a Concorde, an SR-71, and the space shuttle Discovery, among many other exhibits. Check out the full photo album by JT (https://t.co/KRmSNzUSus), our producer. Thanks to all the sponsors for vBSDcon and all the organizers from Verisign, who made it such a great event. *** The story behind FreeBSD-EN-17.08.pf (https://www.sigsegv.be//blog/freebsd/FreeBSD-EN-17.08.pf) After our previous deep dive on a bug in episode 209, Kristof Provost, the maintainer of pf on FreeBSD (he is going to hate me for saying that) has written the story behind a recent ERRATA notice for FreeBSD First things first, so I have to point out that I think Allan misremembered things. The heroic debugging story is PR 219251, which I'll try to write about later. FreeBSD-EN-17:08.pf is an issue that affected some FreeBSD 11.x systems, where FreeBSD would panic at startup. There were no reports for CURRENT. There's very little to go on here, but we do know the cause of the panic ("integer divide fault"), and that the current process was "pf purge". The pf purge thread is part of the pf housekeeping infrastructure. It's a housekeeping kernel thread which cleans up things like old states and expired fragments. The lack of mention of pf functions in the backtrace is a hint unto itself. It suggests that the error is probably directly in pfpurgethread(). It might also be in one of the static functions it calls, because compilers often just inline those so they don't generate stack frames. Remember that the problem is an "integer divide fault". How can integer divisions be a problem? Well, you can try to divide by zero. The most obvious suspect for this is this code: idx = pfpurgeexpiredstates(idx, pfhashmask / (Vpfdefaultrule.timeout[PFTMINTERVAL] * 10)); However, this variable is both correctly initialised (in pfattachvnet()) and can only be modified through the DIOCSETTIMEOUT ioctl() call and that one checks for zero. At that point I had no idea how this could happen, but because the problem did not affect CURRENT I looked at the commit history and found this commit from Luiz Otavio O Souza: Do not run the pf purge thread while the VNET variables are not initialized, this can cause a divide by zero (if the VNET initialization takes to long to complete). Obtained from: pfSense Sponsored by: Rubicon Communications, LLC (Netgate) That sounds very familiar, and indeed, applying the patch fixed the problem. Luiz explained it well: it's possible to use Vpfdefaultrule.timeout before it's initialised, which caused this panic. To me, this reaffirms the importance of writing good commit messages: because Luiz mentioned both the pf purge thread and the division by zero I was easily able to find the relevant commit. If I hadn't found it this fix would have taken a lot longer. Next week we'll look at the more interesting story I was interested in, which I managed to nag Kristof into writing *** The sudden death and eternal life of Solaris (http://dtrace.org/blogs/bmc/2017/09/04/the-sudden-death-and-eternal-life-of-solaris/) A blog post from Bryan Cantrill about the death of Solaris As had been rumored for a while, Oracle effectively killed Solaris. When I first saw this, I had assumed that this was merely a deep cut, but in talking to Solaris engineers still at Oracle, it is clearly much more than that. It is a cut so deep as to be fatal: the core Solaris engineering organization lost on the order of 90% of its people, including essentially all management. Of note, among the engineers I have spoken with, I heard two things repeatedly: “this is the end” and (from those who managed to survive Friday) “I wish I had been laid off.” Gone is any of the optimism (however tepid) that I have heard over the years — and embarrassed apologies for Oracle's behavior have been replaced with dismay about the clumsiness, ineptitude and callousness with which this final cut was handled. In particular, that employees who had given their careers to the company were told of their termination via a pre-recorded call — “robo-RIF'd” in the words of one employee — is both despicable and cowardly. To their credit, the engineers affected saw themselves as Sun to the end: they stayed to solve hard, interesting problems and out of allegiance to one another — not out of any loyalty to the broader Oracle. Oracle didn't deserve them and now it doesn't have them — they have been liberated, if in a depraved act of corporate violence. Assuming that this is indeed the end of Solaris (and it certainly looks that way), it offers a time for reflection. Certainly, the demise of Solaris is at one level not surprising, but on the other hand, its very suddenness highlights the degree to which proprietary software can suffer by the vicissitudes of corporate capriciousness. Vulnerable to executive whims, shareholder demands, and a fickle public, organizations can simply change direction by fiat. And because — in the words of the late, great Roger Faulkner — “it is easier to destroy than to create,” these changes in direction can have lasting effect when they mean stopping (or even suspending!) work on a project. Indeed, any engineer in any domain with sufficient longevity will have one (or many!) stories of exciting projects being cancelled by foolhardy and myopic management. For software, though, these cancellations can be particularly gutting because (in the proprietary world, anyway) so many of the details of software are carefully hidden from the users of the product — and much of the innovation of a cancelled software project will likely die with the project, living only in the oral tradition of the engineers who knew it. Worse, in the long run — to paraphrase Keynes — proprietary software projects are all dead. However ubiquitous at their height, this lonely fate awaits all proprietary software. There is, of course, another way — and befitting its idiosyncratic life and death, Solaris shows us this path too: software can be open source. In stark contrast to proprietary software, open source does not — cannot, even — die. Yes, it can be disused or rusty or fusty, but as long as anyone is interested in it at all, it lives and breathes. Even should the interest wane to nothing, open source software survives still: its life as machine may be suspended, but it becomes as literature, waiting to be discovered by a future generation. That is, while proprietary software can die in an instant, open source software perpetually endures by its nature — and thrives by the strength of its communities. Just as the existence of proprietary software can be surprisingly brittle, open source communities can be crazily robust: they can survive neglect, derision, dissent — even sabotage. In this regard, I speak from experience: from when Solaris was open sourced in 2005, the OpenSolaris community survived all of these things. By the time Oracle bought Sun five years later in 2010, the community had decided that it needed true independence — illumos was born. And, it turns out, illumos was born at exactly the right moment: shortly after illumos was announced, Oracle — in what remains to me a singularly loathsome and cowardly act — silently re-proprietarized Solaris on August 13, 2010. We in illumos were indisputably on our own, and while many outsiders gave us no chance of survival, we ourselves had reason for confidence: after all, open source communities are robust because they are often united not only by circumstance, but by values, and in our case, we as a community never lost our belief in ZFS, Zones, DTrace and myriad other technologies like MDB, FMA and Crossbow. Indeed, since 2010, illumos has thrived; illumos is not only the repository of record for technologies that have become cross-platform like OpenZFS, but we have also advanced our core technologies considerably, while still maintaining highest standards of quality. Learning some of the mistakes of OpenSolaris, we have a model that allows for downstream innovation, experimentation and differentiation. For example, Joyent's SmartOS has always been focused on our need for a cloud hypervisor (causing us to develop big features like hardware virtualization and Linux binary compatibility), and it is now at the heart of a massive buildout for Samsung (who acquired Joyent a little over a year ago). For us at Joyent, the Solaris/illumos/SmartOS saga has been formative in that we have seen both the ill effects of proprietary software and the amazing resilience of open source software — and it very much informed our decision to open source our entire stack in 2014. Judging merely by its tombstone, the life of Solaris can be viewed as tragic: born out of wedlock between Sun and AT&T and dying at the hands of a remorseless corporate sociopath a quarter century later. And even that may be overstating its longevity: Solaris may not have been truly born until it was made open source, and — certainly to me, anyway — it died the moment it was again made proprietary. But in that shorter life, Solaris achieved the singular: immortality for its revolutionary technologies. So while we can mourn the loss of the proprietary embodiment of Solaris (and we can certainly lament the coarse way in which its technologists were treated!), we can rejoice in the eternal life of its technologies — in illumos and beyond! News Roundup OpenBSD on the Lenovo Thinkpad X1 Carbon (5th Gen) (https://jcs.org/2017/09/01/thinkpad_x1c) Joshua Stein writes about his experiences running OpenBSD on the 5th generation Lenovo Thinkpad X1 Carbon: ThinkPads have sort of a cult following among OpenBSD developers and users because the hardware is basic and well supported, and the keyboards are great to type on. While no stranger to ThinkPads myself, most of my OpenBSD laptops in recent years have been from various vendors with brand new hardware components that OpenBSD does not yet support. As satisfying as it is to write new kernel drivers or extend existing ones to make that hardware work, it usually leaves me with a laptop that doesn't work very well for a period of months. After exhausting efforts trying to debug the I2C touchpad interrupts on the Huawei MateBook X (and other 100-Series Intel chipset laptops), I decided to take a break and use something with better OpenBSD support out of the box: the fifth generation Lenovo ThinkPad X1 Carbon. Hardware Like most ThinkPads, the X1 Carbon is available in a myriad of different internal configurations. I went with the non-vPro Core i7-7500U (it was the same price as the Core i5 that I normally opt for), 16Gb of RAM, a 256Gb NVMe SSD, and a WQHD display. This generation of X1 Carbon finally brings a thinner screen bezel, allowing the entire footprint of the laptop to be smaller which is welcome on something with a 14" screen. The X1 now measures 12.7" wide, 8.5" deep, and 0.6" thick, and weighs just 2.6 pounds. While not available at initial launch, Lenovo is now offering a WQHD IPS screen option giving a resolution of 2560x1440. Perhaps more importantly, this display also has much better brightness than the FHD version, something ThinkPads have always struggled with. On the left side of the laptop are two USB-C ports, a USB-A port, a full-size HDMI port, and a port for the ethernet dongle which, despite some reviews stating otherwise, is not included with the laptop. On the right side is another USB-A port and a headphone jack, along with a fan exhaust grille. On the back is a tray for the micro-SIM card for the optional WWAN device, which also covers the Realtek microSD card reader. The tray requires a paperclip to eject which makes it inconvenient to remove, so I think this microSD card slot is designed to house a card semi-permanently as a backup disk or something. On the bottom are the two speakers towards the front and an exhaust grille near the center. The four rubber feet are rather plastic feeling, which allows the laptop to slide around on a desk a bit too much for my liking. I wish they were a bit softer to be stickier. Charging can be done via either of the two USB-C ports on the left, though I wish more vendors would do as Google did on the Chromebook Pixel and provide a port on both sides. This makes it much more convenient to charge when not at one's desk, rather than having to route a cable around to one specific side. The X1 Carbon includes a 65W USB-C PD with a fixed USB-C cable and removable country-specific power cable, which is not very convenient due to its large footprint. I am using an Apple 61W USB-C charger and an Anker cable which charge the X1 fine (unlike HP laptops which only work with HP USB-C chargers). Wireless connectivity is provided by a removable Intel 8265 802.11a/b/g/n/ac WiFi and Bluetooth 4.1 card. An Intel I219-V chip provides ethernet connectivity and requires an external dongle for the physical cable connection. The screen hinge is rather tight, making it difficult to open with one hand. The tradeoff is that the screen does not wobble in the least bit when typing. The fan is silent at idle, and there is no coil whine even under heavy load. During a make -j4 build, the fan noise is reasonable and medium-pitched, rather than a high-pitched whine like on some laptops. The palm rest and keyboard area remain cool during high CPU utilization. The full-sized keyboard is backlit and offers two levels of adjustment. The keys have a soft surface and a somewhat clicky feel, providing very quiet typing except for certain keys like Enter, Backspace, and Escape. The keyboard has a reported key travel of 1.5mm and there are dedicated Page Up and Page Down keys above the Left and Right arrow keys. Dedicated Home, End, Insert, and Delete keys are along the top row. The Fn key is placed to the left of Control, which some people hate (although Lenovo does provide a BIOS option to swap it), but it's in the same position on Apple keyboards so I'm used to it. However, since there are dedicated Page Up, Page Down, Home, and End keys, I don't really have a use for the Fn key anyway. Firmware The X1 Carbon has a very detailed BIOS/firmware menu which can be entered with the F1 key at boot. F12 can be used to temporarily select a different boot device. A neat feature of the Lenovo BIOS is that it supports showing a custom boot logo instead of the big red Lenovo logo. From Windows, download the latest BIOS Update Utility for the X1 Carbon (my model was 20HR). Run it and it'll extract everything to C:driversflash(some random string). Drop a logo.gif file in that directory and run winuptp.exe. If a logo file is present, it'll ask whether to use it and then write the new BIOS to its staging area, then reboot to actually flash it. + OpenBSD support Secure Boot has to be disabled in the BIOS menu, and the "CSM Support" option must be enabled, even when "UEFI/Legacy Boot" is left on "UEFI Only". Otherwise the screen will just go black after the OpenBSD kernel loads into memory. Based on this component list, it seems like everything but the fingerprint sensor works fine on OpenBSD. *** Configuring 5 different desktop environments on FreeBSD (https://www.linuxsecrets.com/en/entry/51-freebsd/2017/09/04/2942-configure-5-freebsd-x-environments) This fairly quick tutorial over at LinuxSecrets.com is a great start if you are new to FreeBSD, especially if you are coming from Linux and miss your favourite desktop environment It just goes to show how easy it is to build the desktop you want on modern FreeBSD The tutorial covers: GNOME, KDE, Xfce, Mate, and Cinnamon The instructions for each boil down to some variation of: Install the desktop environment and a login manager if it is not included: > sudo pkg install gnome3 Enable the login manager, and usually dbus and hald: > sudo sysrc dbusenable="YES" haldenable="YES" gdmenable="YES" gnomeenable="YES"? If using a generic login manager, add the DE startup command to your .xinitrc: > echo "exec cinnamon" > ~/.xinitrc And that is about it. The tutorial goes into more detail on other configuration you can do to get your desktop just the way you like it. To install Lumina: > sudo pkg install lumina pcbsd-utils-qt5 This will install Lumina and the pcbsd utilities package which includes pcdm, the login manager. In the near future we hear the login manager and some of the other utilities will be split into separate packages, making it easier to use them on vanilla FreeBSD. > sudo sysrc pcdmenable=”YES” dbusenable="YES" hald_enable="YES" Reboot, and you should be greeted with the graphical login screen *** A return-oriented programming defense from OpenBSD (https://lwn.net/Articles/732201/) We talked a bit about RETGUARD last week, presenting Theo's email announcing the new feature Linux Weekly News has a nice breakdown on just how it works Stack-smashing attacks have a long history; they featured, for example, as a core part of the Morris worm back in 1988. Restrictions on executing code on the stack have, to a great extent, put an end to such simple attacks, but that does not mean that stack-smashing attacks are no longer a threat. Return-oriented programming (ROP) has become a common technique for compromising systems via a stack-smashing vulnerability. There are various schemes out there for defeating ROP attacks, but a mechanism called "RETGUARD" that is being implemented in OpenBSD is notable for its relative simplicity. In a classic stack-smashing attack, the attack code would be written directly to the stack and executed there. Most modern systems do not allow execution of on-stack code, though, so this kind of attack will be ineffective. The stack does affect code execution, though, in that the call chain is stored there; when a function executes a "return" instruction, the address to return to is taken from the stack. An attacker who can overwrite the stack can, thus, force a function to "return" to an arbitrary location. That alone can be enough to carry out some types of attacks, but ROP adds another level of sophistication. A search through a body of binary code will turn up a great many short sequences of instructions ending in a return instruction. These sequences are termed "gadgets"; a large program contains enough gadgets to carry out almost any desired task — if they can be strung together into a chain. ROP works by locating these gadgets, then building a series of stack frames so that each gadget "returns" to the next. There is, of course, a significant limitation here: a ROP chain made up of exclusively polymorphic gadgets will still work, since those gadgets were not (intentionally) created by the compiler and do not contain the return-address-mangling code. De Raadt acknowledged this limitation, but said: "we believe once standard-RET is solved those concerns become easier to address separately in the future. In any case a substantial reduction of gadgets is powerful". Using the compiler to insert the hardening code greatly eases the task of applying RETGUARD to both the OpenBSD kernel and its user-space code. At least, that is true for code written in a high-level language. Any code written in assembly must be changed by hand, though, which is a fair amount of work. De Raadt and company have done that work; he reports that: "We are at the point where userland and base are fully working without regressions, and the remaining impacts are in a few larger ports which directly access the return address (for a variety of reasons)". It can be expected that, once these final issues are dealt with, OpenBSD will ship with this hardening enabled. The article wonders about applying the same to Linux, but notes it would be difficult because the Linux kernel cannot currently be compiled using LLVM If any benchmarks have been run to determine the cost of using RETGUARD, they have not been publicly posted. The extra code will make the kernel a little bigger, and the extra overhead on every function is likely to add up in the end. But if this technique can make the kernel that much harder to exploit, it may well justify the extra execution overhead that it brings with it. All that's needed is somebody to actually do the work and try it out. Videos from BSDCan have started to appear! (https://www.youtube.com/playlist?list=PLeF8ZihVdpFfVEsCxNWGDmcATJfRZacHv) Henning Brauer: tcp synfloods - BSDCan 2017 (https://www.youtube.com/watch?v=KuHepyI0_KY) Benno Rice: The Trouble with FreeBSD - BSDCan 2017 (https://www.youtube.com/watch?v=1DM5SwoXWSU) Li-Wen Hsu: Continuous Integration of The FreeBSD Project - BSDCan 2017 (https://www.youtube.com/watch?v=SCLfKWaUGa8) Andrew Turner: GENERIC ARM - BSDCan 2017 (https://www.youtube.com/watch?v=gkYjvrFvPJ0) Bjoern A. Zeeb: From the outside - BSDCan 2017 (https://www.youtube.com/watch?v=sYmW_H6FrWo) Rodney W. Grimes: FreeBSD as a Service - BSDCan 2017 (https://www.youtube.com/watch?v=Zf9tDJhoVbA) Reyk Floeter: The OpenBSD virtual machine daemon - BSDCan 2017 (https://www.youtube.com/watch?v=Os9L_sOiTH0) Brian Kidney: The Realities of DTrace on FreeBSD - BSDCan 2017 (https://www.youtube.com/watch?v=NMUf6VGK2fI) The rest will continue to trickle out, likely not until after EuroBSDCon *** Beastie Bits Oracle has killed sun (https://meshedinsights.com/2017/09/03/oracle-finally-killed-sun/) Configure Thunderbird to send patch friendly (http://nanxiao.me/en/configure-thunderbird-to-send-patch-friendly/) FreeBSD 10.4-BETA4 Available (https://www.freebsd.org/news/newsflash.html#event20170909:01) iXsystems looking to hire kernel and zfs developers (especially Sun/Oracle Refugees) (https://www.facebook.com/ixsystems/posts/10155403417921508) Speaking of job postings, UnitedBSD.com has few job postings related to BSD (https://unitedbsd.com/) Call for papers USENIX FAST ‘18 - February 12-15, 2018, Due: September 28 2017 (https://www.freebsdfoundation.org/news-and-events/call-for-papers/usenix-fast-18-call-for-papers/) Scale 16x - March 8-11, 2018, Due: October 31, 2017 (https://www.freebsdfoundation.org/news-and-events/call-for-papers/scale-16x-call-for-participation/) FOSDEM ‘18 - February 3-4, 2018, Due: November 3 2017 (https://www.freebsdfoundation.org/news-and-events/call-for-papers/fosdem-18-call-for-participation/) Feedback/Questions Jason asks about cheap router hardware (http://dpaste.com/340KRHG) Prashant asks about latest kernels with freebsd-update (http://dpaste.com/2J7DQQ6) Matt wants know about VM Performance & CPU Steal Time (http://dpaste.com/1H5SZ81) John has config questions regarding Dell precision 7720, FreeBSD, NVME, and ZFS (http://dpaste.com/0X770SY) ***

DragonflyBSD 4.8.1 has been released, we explore how the X11 clipboard works, and look at OpenBSD gaming resources. This episode was brought to you by Headlines LLVM, Clang and compiler-rt support enhancements (https://blog.netbsd.org/tnf/entry/llvm_clang_and_compiler_rt) In the last month I started with upstream of the code for sanitizers: the common layer and ubsan. I worked also on the elimination of unexpected failures in LLVM and Clang. I've managed to achieve, with a pile of local patches, the number of 0 unexpected bugs within LLVM (check-llvm) and 3 unexpected bugs within Clang (check-clang) (however these ones were caused by hardcoded environment -lstdc++ vs -lc++). The number of failures in sanitizers (check-sanitizer) is also low, it's close to zero. LLVM In order to achieve the goals of testability concerning the LLVM projects, I had to prepare a new pkgsrc-wip package called llvm-all-in-one that contains 12 active LLVM projects within one tree. The set of these projects is composed of: llvm, clang, compiler-rt, libcxx, libcxxabi, libunwind, test-suite, openmp, llgo, lld, lldb, clang-tools-extra. These were required to build and execute test-suites in the LLVM's projects. Ideally the tests should work in standalone packages - built out-of-LLVM-sources - and with GCC/Clang, however the real life is less bright and this forced me to use Clang as the system compiler an all-in-one package in order to develop the work environment with the ability to build and execute unit tests. There were four threads within LLVM: Broken std::callonce with libstdc++. This is an old and well-known bug, which was usually worked around with a homegrown implementation llvm::callonce. I've discovered that the llvm::callonce workaround isn't sufficient for the whole LLVM functionality, as std::callonce can be called internally inside the libstdc++ libraries - like within the C++11 futures interface. This bug has been solved by Joerg Sonnenberger in the ELF dynamic linker. Unportable shell construct hardcoded in tests ">&". This has been fixed upstream. LLVM JIT. The LLVM Memory generic allocator (or page mapper) was designed to freely map pages with any combination of the protection bits: R,W,X. This approach breaks on NetBSD with PaX MPROTECT and requires redesign of the interfaces. This is the continuation of the past month AllocateRWX and ReleaseRWX compatibility with NetBSD improvements. I've prepared few variations of local patches addressing these issues and it's still open for discussion with upstream. My personal preference is to remove the current API entirely and introduce a newer one with narrowed down functionality to swap between readable (R--), writable (RW-) and executable (R-X) memory pages. This would effectively enforce W^X. Sanitizers support. Right now, I keep the patches locally in order to upstream the common sanitizer code in compiler-rt. The LLVM JIT API is the last cause of unexpected failures in check-llvm. This breaks MCJIT, ORCJIT and ExecutionEngine libraries and causes around 200 unexpected failures within tests. Clang I've upstreamed a patch that enables ubsan and asan on Clang's frontend for NetBSD/amd64. This support isn't complete, and requires sanitizers' support code upstreamed to compiler-rt. compiler-rt The current compiler-rt tasks can be divided into: upstream sanitizer common code shared with POSIX platforms upstream sanitizer common code shared with Linux and FreeBSD upstream sanitizer common code shared with FreeBSD upstream sanitizer common code specific to NetBSD build, execute and pass tests for sanitizer common code in check-santizer This means that ubsan, asan and the rest of the specific sanitizers wait in queue. All the mentioned tasks are being worked on simultaneously, with a soft goal to finish them one after another from the first to the last one. The last point with check-sanitizer unveiled so far two generic bugs on NetBSD: Return errno EFAULT instead of EACCES on memory fault with read(2)/write(2)-like syscalls. Honor PTHREADDESTRUCTORITERATIONS in libpthread. These bugs are not strictly real bugs, but they were introducing needless differences with other modern POSIX systems. The fixes were introduced by Christos Zoulas and backported to NetBSD-8. Plan for the next milestone I have decided not to open new issues in with the coming month and focus on upstreaming the remaining LLVM code. The roadmap for the next month is to continue working on the goals of the previous months. std::call_once is an example that every delayed bug keeps biting again and again in future. LLVM 5.0.0 is planned to be released this month (August) and there is a joint motivation with the upstream maintainer to push compatibility fixes for LLVM JIT. There is an option to submit a workaround now and introduce refactoring for the trunk and next version (6.0.0). This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: http://netbsd.org/donations/#how-to-donate *** DragonFly BSD 4.8.1 released (http://lists.dragonflybsd.org/pipermail/commits/2017-August/626150.html) +Updates by dev: + Antonio Huete Jimenez (1): + libc/gmon: Replace sbrk() with mmap() + Francois Tigeot (3): + drm: bring in Linux compability changes from master + drm/linux: make flushwork() more robust + drm/i915: Update to Linux 4.7.10 + Imre Vadász (4): + drm - Fix hrtimer, don't reset timer->function to NULL in timeout handler. + sound - Delete devfs clone handler for /dev/dsp and /dev/mixer on unload. + ifvtnet - Allocate struct vtnettxheader entries from a queue. + Make sure that cam(4)'s dashutdown handler runs before DEVICESHUTDOWN(). + Matthew Dillon (24): + kernel - MFC b48dd28447fc (sigtramp workaround) + kernel - Fix deadlock in sound system + kernel - Fix broken wakeup in crypto code + kernel - Add KERNPROCSIGTRAMP + gcc - Adjust the unwind code to use the new sigtramp probe sysctl + kernel - Implement NX + kernel - Implement NX (2) + kernel - Implement machdep.pmapnxenable TUNABLE + kernel - Implement NX (3) - cleanup + kernel - Temporarily set the default machdep.pmapnxenable to 0 + param - Change _DragonFlyversion to 400801 + kernel - Fix i915 deadlock + pthreads - Change PTHREADSTACKMIN + libc - Fix bug in rcmdsh() + ppp - Fix minor overflow in protocol search + libtelnet - Fix improper statement construction (not a bug in the binary) + libdevstat - Limit sscanf field, fix redundant condition + openssh - Fix a broken assignment + window - Fix Graphics capability enable test + kernel - Fix event preset + mfiutil - Fix static buffer overflow + mixer - Fix sscanf() overflow + gcore - fix overflow in sscanf + kernel - Fix improper parens + Sascha Wildner (17): + libkvm: Fix char pointer dereference. + Fix some cases where an index was used before its limits check. + Really ensure that our world/kernel are built under POSIX locale ("C"). + zoneinfo: Create a /usr/share/zoneinfo/UTC link. + kernel/cam: Add CAMSCSIITNEXUSLOST (in preparation for virtioscsi(4)). + kernel: Add FreeBSD's virtioscsi(4) driver. + ccdconfig(8): Add missing free(). + libpuffs: Fix two asserts. + kernel/acpi: Untangle the wakecode generation during buildkernel. + kernel/acpica: Better check AcpiOsPredefinedOverride()'s InitVal argument + kernel/acpica: ACPITHREADID is unsigned. + kernel/acpica: Return curthread as thread id from AcpiOsGetThreadId(). + kernel/acpica: Remove no longer needed #include. + kernel/acpi: Call AcpiInitializeSubsystem() before AcpiInitializeTables(). + kernel/urtwn: Add missing braces. + kernel/ieee80211: Add missing braces. + libthreadxu: Fix checking of pthreadbarrierinit()'s count argument. + Sepherosa Ziehau (7): + sound/hda: Sync device ID table with FreeBSD + inet6: Restore mbuf hash after defragmentation. + pf: Normalized, i.e. defragged, packets requiring rehash. + em: Enable MSI by default on devices has PCI advanced features capability. + sched: Change CPU_SETSIZE to signed int, same as FreeBSD/Linux. + usched: Allow process to change self cpu affinity + ix: Fixup TX/RX ring settings for X550, which supports 64/64 TX/RX rings. + zrj (1): + Revert "Always use unix line endings" Porting Unix to the 386: A Practical Approach (http://www.informatica.co.cr/unix-source-code/research/1991/0101.html) The University of California's Berkeley Software Distribution (BSD) has been the catalyst for much of the innovative work done with the UNIX operating system in both the research and commercial sectors. Encompassing over 150 Mbytes (and growing) of cutting-edge operating systems, networking, and applications software, BSD is a fully functional and nonproprietary complete operating systems software distribution (see Figure 1). In fact, every version of UNIX available from every vendor contains at least some Berkeley UNIX code, particularly in the areas of filesystems and networking technologies. However, unless one could pay the high cost of site licenses and equipment, access to this software was simply not within the means of most individual programmers and smaller research groups. The 386BSD project was established in the summer of 1989 for the specific purpose of porting BSD to the Intel 80386 microprocessor platform so that the tools this software offers can be made available to any programmer or research group with a 386 PC. In coordination with the Computer Systems Research Group (CSRG) at the University of California at Berkeley, we successively ported a basic research system to a common AT class machine (see, Figure 2), with the result that approximately 65 percent of all 32-bit systems could immediately make use of this new definition of UNIX. We have been refining and improving this base port ever since. By providing the base 386BSD port to CSRG, our hope is to foster new interest in Berkeley UNIX technology and to speed its acceptance and use worldwide. We hope to see those interested in this technology build on it in both commercial and noncommercial ventures. In this and following articles, we will examine the key aspects of software, strategy, and experience that encompassed a project of this magnitude. We intend to explore the process of the 386BSD port, while learning to effectively exploit features of the 386 architecture for use with an advanced operating system. We also intend to outline some of the tradeoffs in implementation goals which must be periodically reexamined. Finally, we will highlight extensions which remain for future work, perhaps to be done by some of you reading this article today. Note that we are assuming familiarity with UNIX, its concepts and structures, and the basic functions of the 386, so we will not present exhaustive coverage of these areas. In this installment, we discuss the beginning of our project and the initial framework that guided our efforts, in particular, the development of the 386BSD specification. Future articles will address specific topics of interest and actual nonproprietary code fragments used in 386BSD. Among the future areas to be covered are: 386BSD process context switching Executing the first 386BSD process on the PC 386BSD kernel interrupt and exception handling 386BSD INTERNET networking ISA device drivers and system support 386BSD bootstrap process *** X11: How does “the” clipboard work (https://www.uninformativ.de/blog/postings/2017-04-02/0/POSTING-en.html) > If you have used another operating system before you switched to something that runs X11, you will have noticed that there is more than one clipboard: > Sometimes, you can use the mouse to select some text, switch to another window, and then hit the middle mouse button to paste text. > Sometimes, you can select text, then hit some hotkey, e.g. Ctrl+C, switch to another window, hit another hotkey, e.g. Ctrl+V, and paste said text. > Sometimes, you can do both. > Selections as a form of IPC First things first, in X11 land, “clipboards” are called “selections”. Yes, there is more than one selection and they all work independently. In fact, you can use as many selections as you wish. In theory, that is. When using selections, you make different clients communicate with each other. This means that those clients have to agree on which selections to use. You can't just invent your own selection and then expect Firefox to be compatible with it. How are selections identified? There are three “standard” selection names: PRIMARY: The “middle mouse clipboard” SECONDARY: Virtually unused these days CLIPBOARD: The “Ctrl+C clipboard” Program 1: Query selection owners Content type and conversion Program 2: Get clipboard as UTF-8 Program 3: Owning a selection Program 4: Content type TARGETS Handling binary data using xclip Large amounts of data Clipboard managers Summary News Roundup TrueOS Documentation: A great way to give back! (https://www.trueos.org/blog/trueos-documentation-great-way-give-back/) The TrueOS project is always looking for community contribution. Documentation changes are a great way for users to not only make a solid contribution to the project, but learn more about it too! Over the last few months, many users have asked for both simple and detailed instructions on making documentation changes. These are now added to the TrueOS handbook in the Contributing to TrueOS section. If interested in making a small alteration to the TrueOS handbook, here are some instructions for submitting a patch through the GitHub website. These instructions are also applicable to the Lumina and SysAdm handbooks. Lumina documentation is in the the lumina-docs repository, and SysAdm guides are in sysadm-docs. Make a Doc change! A GitHub account is required to submit patches to the TrueOS docs. Open a web browser and sign in to GitHub or make a new account. When making a new account, be sure to use an often checked email address, as all communication regarding patches and pull requests are sent to this address. Navigate to the trueos-docs GitHub repository. Click on the trueos-handbook directory to view all the documentation files. Open the .rst file corresponding to the chapter needing an update. The chapter names are reflected in the title of the .rst files. For example, open install.rst to fix an error spotted in handbook chapter 3: “Install”. This first image shows the trueos-docs repository and the contents of the trueos-handbook directory Open the desired chapter file by clicking its entry in the list. The trueos.rst file is an index file and should be ignored. Begin editing the file by clicking the Pencil icon in the upper right corner above the file's text. The file moves to edit mode, where it is now possible to make changes, as the next image shows. Editing install.rst with GitHub When making a simple change, it is recommended to avoid adjusting the specific formatting elements and instead work within or around them. Once satisfied, scroll to the bottom of the page and write a detailed commit summary of the new changes. Click Propose file change (green button), then Create pull request to submit the changes to the project. GitHub then does an automated merge check. Click Create pull request again to submit the change to the repository. In the final step, a developer or project committer reviews the changes, merging them into the project or asking for more changes as necessary. Learn more about TrueOS documentation To learn more about the underlying structure of TrueOS documentation like the Sphinx Documentation Generator and reStructuredText markup, browse the Advanced Documentation Changes section of the TrueOS handbook. This section also contains instructions for forking the repository and configuring a local clone, build testing, updating the translation files, and other useful information. The Sphinx website is also a valuable resource. libHijack Revival (https://www.soldierx.com/news/Hijack-Revival) Over a decade ago, while standing naked and vulnerable in the comfort of my steaming hot shower, I gathered my thoughts as humans typically attempt to do in the wee hours of the morning. Thoughts of a post-exploitation exercise raced in my mind, the same thoughts that made sleeping the night before difficult. If only I could inject into Apache some code that would allow me to hook into its parsing engine without requiring persistance. Putting a file-backed entry into /proc/pid/maps would tip off the security team to a compromise. The end-goal was to be able to send Apache a special string and have Apache perform a unique action based on the special string. FelineMenace's Binary Protection Schemes whitepaper provided inspiration. Silvio Cesare paved the way into PLT/GOT redirection attacks. Various Phrack articles selflessly contributed to the direction I was to head. Alas, in the aforementioned shower, an epiphany struck me. I jumped as an awkward stereotypical geek does: like an elaborate Elaine Benes dance rehearsal in the air. If I used PTrace, ELF, and the PLT/GOT to my advantage, I could cause the victim application to allocate anonymous memory mappings arbitrarily. In the newly-created memory mapping, I could inject arbitrary code. Since a typical operating system treats debuggers as God-like applications, the memory mapping could be mapped without write access, but as read and execute only. Thus enabling the stealth that I sought. The project took a few years to develop in my spare time. I ended up creating several iterations, taking a rough draft/Proof-of-Concept style code and rewriting it to be more efficient and effective. I had toyed with FreeBSD off-and-on for over a decade by this point, but by-and-large I was still mostly using Linux. FreeBSD gained DTrace and ZFS support, winning me over from the Linux camp. I ported libhijack to FreeBSD, giving it support for both Linux and FreeBSD simultaneously. In 2013, I started work on helping Oliver Pinter with his ASLR implementation, which was originally destined to be upstreamed to FreeBSD. It took a lot of work, and my interest in libhijack faded. As a natural consequence, I handed libhijack over to SoldierX, asking the community to take it and enhance it. Over four years went by without a single commit. The project was essentially abandoned. My little baby was dead. This past week, I wondered if libhijack could even compile on FreeBSD anymore. Given that four years have passed by and major changes have happened in those four years, I thought libhijack would need a major overhaul just to compile, let alone function. Imagine my surprise when libhijack needed only a few fixups to account for changes in FreeBSD's RTLD. Today, I'm announcing the revival of libhijack. No longer is it dead, but very much alive. In order to develop the project faster, I've decided to remove support for Linux, focusing instead on FreeBSD. I've removed hundreds of lines of code over the past few days. Supporting both FreeBSD and Linux meant some code had to be ugly. Now the beautification process has begun. I'm announcing the availability of libhijack 0.7.0 today. The ABI and API should be considered unstable as they may change without notice. Note that HardenedBSD fully mitigates libhijack from working with two security features: setting security.bsd.unprivilegedprocdebug to 0 by default and the implementation of PaX NOEXEC. The security.bsd.unprivilegedprocdebug sysctl node prevents PTrace access for applications the debugger itself did not fork+execve for unprivileged (non-root) users. Privileged users (the root account) can use PTrace to its fullest extent. HardenedBSD's implementation of PaX NOEXEC prevents the creation of memory mappings that are both writable and executable. It also prevents using mprotect to toggle between writable and executable. In libhijack's case, FreeBSD grants libhijack the ability to write to memory mappings that are not marked writable. Debuggers do this to set breakpoints. HardenedBSD behaves differently due to PaX NOEXEC. Each memory mapping has a notion of a maximum protection level. When a memory mapping is created, if the write bit is set, then HardenedBSD drops the execute bit from the maximum protection level. When the execute bit is set at memory mapping creation time, then the write bit is dropped from the maximum protection level. If both the write and execute bits are set, then the execute bit is silently dropped from both the mapping creation request and the maximum protection level. The maximum protection level is always obeyed, even for debuggers. Thus we see that PaX NOEXEC is 100% effective in preventing libhijack from injecting code into a process. Here is a screenshot showing PaX NOEXEC preventing libhijack from injecting shellcode into a newly-created memory mapping. What's next for libhijack? Here's what we have planned, in no particular order: Python bindings Port to arm64 This requires logic for handling machine-dependent code. High priority. Finish anonymous shared object injection. This requires implementing a custom RTLD from within libhijack. More cleanups. Adhere to style(9). libhijack can be found on GitHub @ https://github.com/SoldierX/libhijack *** Contributing to FreeBSD (https://blather.michaelwlucas.com/archives/2988) I've talked to a whole bunch of folks who say things like “I'm a junior programmer. I'm looking for a way to help. I have no specific expertise, but I'm willing to learn.” Today, I present such junior programmers with an opportunity. An opportunity for you to learn skills that will be incredibly valuable to your career, and will simultaneously expand your career opportunities. For decades, FreeBSD has relied on its users for testing. They expect users to install pre-release versions of the OS and exercise them to identify regressions. That's necessary, but it's nowhere near enough. The FreeBSD Testing Project is building an automated test suite for the entire operating system. They have a whole mess of work to do. There's only four people on the team, so each additional person that contributes can have a serious impact. They have tutorials on how to write tests, and sample tests. There's a whole bunch of tests left to be written. You have an almost open field. They need tests for everything from ls(1) to bhyve. (Yes, ls(1) broke at one point in the last few years.) Everything needs testing. Learning to write, submit, and commit small tests is valuable experience for developing the big tests. What's more, learning to write tests for a system means learning the system. Developing tests will transform you into a FreeBSD expert. Once you've demonstrated your competence, worth, and ability to work within the project, other FreeBSD teams will solicit your help and advice. The Project will suck you in. Testing is perhaps the most valuable contribution anyone can make to an open source project. And this door into the FreeBSD Project is standing wide, wide open. OpenBSD Gaming Resource (https://mrsatterly.com/openbsd_games.html) > What isn't there to love about playing video games on your favorite operating system? OpenBSD and video games feels like a natural combination to me. My resource has software lists, links to free games not in ports, lists of nonfree games, and recommendations. The Table of Contents has these high-level items for you: > General Resources > OpenBSD Exclusive > Ports > Network Clients > Browser Games > Game Engines > Multiple Game Engines > Multiple System Emulation > Computer Emulation > Game Console Emulation > Live Media Emulation > Operating System Emulation > Games in Other Software Have fun with these games! *** Beastie Bits Dragonfly introduces kcollect(8) (https://www.dragonflydigest.com/2017/08/07/20061.html) The Faces of Open Source (http://facesofopensource.com/unix/) Edgemesh CEO, Jake Loveless and Joyent CTO, Bryan Cantrill join together for a fireside chat to discuss distributed caching at scale, Docker, Node.js, Mystery Science Theater 3000, and more! (https://www.joyent.com/blog/joyent-edgemesh-cache-me-if-you-can) UFS: Place the information needed to find alternate superblocks to the end of the area reserved for the boot block (https://svnweb.freebsd.org/base?view=revision&revision=322297) Let ‘localhost' be localhost (https://tools.ietf.org/html/draft-west-let-localhost-be-localhost-04) Hurry up and register for vBSDCon September 7-9 (http://www.verisign.com/en_US/internet-technology-news/verisign-events/vbsdcon/index.xhtml?dmn=vBSDcon.com) and EuroBSDCon September 21-24 (https://2017.eurobsdcon.org/) *** Feedback/Questions Morgan - btrfs deprecated (http://dpaste.com/0JEYE1K) Ben - UEFI, GELI, BEADM, and more (http://dpaste.com/2TP90HD) Brad - Hostname Clarification (http://dpaste.com/1MQH1BD) M Rod - BSD Laptop (http://dpaste.com/39C6PGN) Jeremy - Contributing to BSDs (http://dpaste.com/3SVP5SF) ***

We recap our devsummit experiences at BSDCambridge, share why memcmp is more complicated than expected, explore Docker on FreeBSD, and we look at a retro terminal. This episode was brought to you by Headlines BSDCam recap (https://wiki.freebsd.org/DevSummit/201708) The 2017 Cambridge DevSummit took place from 2-4 August 2017. The event took place over three days including a formal dinner at St John's College, and was attended by 55 registered developers and guests. Prior to the start of the conference, we had a doc hacking lounge, the computer lab provided a room where we could meet and try to spend some time on documentation. Sevan walked two interested people through the process of creating a documentation patch and submitting it for the first time. In the process, found ways to improve the documentation on how to write documentation. The event is run "un-conference style" in that we brainstorm the actual session schedule on the first morning, with a focus on interactive topics that reflect the interests and exploit the knowledge of the attendees. The idea is to maximize the amount of discussion and decisions that can be made while we are all in the same room The first morning, we all gather in the slightly too small, and even more slightly under air conditioned FW11 classroom. We go around the room introducing ourselves, and listing a few topics we would be interested in discussing. Eventually the whiteboard is full of topics, with various numbers of ticks beside them to indicate the number of interested people There are breakout rooms of all sizes, so even topics with only a small group of interested folks can get a lot accomplished The most difficult is trying to schedule the sessions, as there is much overlap and people usually want to be in concurrent sessions, or someone's schedule means they won't be available that day, etc. This years working groups: Toolchain (Compilers, Linkers, External Toolchain, Static analysis and sanitizers) Virtualization (bhyve, xen, jails, docker) Transport (TCP) and Network Performance Security and mitigations (W^X, noexec stack, CFI, ASLR, KASLR, Safe Stack, etc) Testing (Status, What to test, How to test, QA for releases) Capsicum (Automation with LLVM etc, Casper, Namespacing, “Services”, capsh) Desktop / WiFi (drm-next, drivers, resume, power, installer, desktop, OOB Experience) Tracing (Blackbox, DTrace, KTR, ptrace, truss, hardware tracing) Packaging and Packaged Base (Sets, Kernels, Ports & flavours, sub-packages, privlib) Architectural Security Features (CPU Features: SGX, PXN/PAN, Pointer Authentication, AMD Memory Encryption, Libcrunch, RISC-V, CheriABI) Architectures and Embedded systems (RISC-V, ARM, ARM64, MIPS(64), SPARC64) Teaching (Audiences, Objectives, Targets, Material, future directions) Provisioning and Management Tools (CfgMgmt tools, Image building, VM/bhyve orchestration, Preconfigured VMs for testing, Wishlist) Storage (ZFS status update, ZFS encryption infrastructure, ZFS Zero Copy / Sendfile, Acceleration of checksums and raidz parity calculations, sesutil, mpsutil) And that wasn't everything. We then had a series of short talklets: Enhancing and replacing mmap() SDIO support eBPF support for FreeBSD Tracing + Virtualization Practical DMA Attack Protection On Thursday night there was a special dinner at St John's College Overall it was a great DevSummit, and I even managed to get some of the work assigned to me finished. Shortly I will commit an update to the boot loader menu that will automatically populate the kernel selection menu with the automatically detected list of installed kernels. The list is also properly refreshed when you switch boot environments. *** Hosts/BSD – for when you need to run your BSD inside a penguin (https://wiki.qemu.org/index.php/Hosts/BSD) This wiki provides details on how to run each of the various BSDs under QEMU The target audience is Linux developers looking to test their apps etc under BSD The wiki is in need of some love, there are some option questions, and it lacks some polish There are instructions on building qemu from source, but it should likely mention the qemu-devel port There should probably also be instructions on using other architectures, like ARM/MIPS etc If you have used QEMU, or would like to spend the time to learn how, please help update this wiki *** memcmp -- more complicated than you might expect (http://trust-in-soft.com/memcmp-requires-pointers-to-fully-valid-buffers/) “A suspicious pattern in open-source software” One bug recently found by John using tis-interpreter on a widely used open-source library involved the comparison of strings with memcmp. The unexpected condition was that memcmp was, in one case, called with a pointer to a buffer shorter than the length passed as third argument, breaking one of the two symmetrical pre-conditions in the function's ACSL contract A reason that may have made this use of memcmp look okay to the developer is that the buffers being passed to it always differed before the end of the buffers were reached. a memcmp implementation based on stopping as soon as a difference is found, would not have caused any out-of-bounds read access The first question raised was whether the pattern memcmp("a", "bc", 3) was problematic according to the letter of the C standard. If it was, the second question was whether the busy maintainer of one of the Open Source packages that make the Internet tick should be bothered with a bug report. I would like to be able to say that memcmp's ACSL contract was the product of careful deliberation, but unfortunately this is not the case: many standard function contracts were written quickly in order to get most of the standard library covered, and have not been tested by time. Anyway, upon proofreading the relevant clause in the C11 standard, my feeling was that the ACSL formalization was, in this particular case, right, and that it was undefined behavior to pass as memcmp argument a buffer that wasn't fully valid, even if the implementation sort-of needs to read the buffer's characters in order for the purpose of finding the first mismatch. The post then goes on to look at the memcmp code in glibc There are two distinct optimizations for long buffers, one that applies when both buffers start at the same offset modulo the word size, memcmpcommonalignment, and one that applies when they don't, memcmpnotcommonalignment. The function memcmpcommonalignment is relatively well-behaved: it reads from the two buffers aligned word by aligned word, and thus reads the entire words that contain differing bytes. If the caller passed buffers that aren't valid after the differing byte, this amounts to reading out of bounds, but this sort of out-of-bounds access is not detected by the typical MMU, which works at the scale of the page. The “notcommon_alignment” case, however, tells a different story. When passed the carefully (mis-)aligned buffers t1 and (char*)t2+1, although these buffers differ in the 8th byte, Glibc's implementation of memcmp reads 8 bytes beyond the end of t1. By making the 16th byte differ instead of the 8th one, it is also possible to make Glibc's implementation of memcmp read 16 bytes beyond the end of t1. In conclusion, yes, some implementations of memcmp will crash when invoked with buffers that aren't valid for the full length, even if they differ early. The circumstances are rare (probably the reason this bug was still there to be found in a library that had already been tested with all the available techniques) but outside the programmer's control. The pattern described in this post should be reported as a bug when found. It is interesting to read the detailed analysis of a bug in such a basic libc feature *** News Roundup Docker on FreeBSD (http://daemon-notes.com/articles/network/docker) There are two approaches to running Docker on FreeBSD. First one was created back in 2015 and it was a native port of Docker engine to FreeBSD. It was an ambitious project but nobody stepped forward to continuously port the never-ending flow of upstream code to FreeBSD. So the port still exists (sysutils/docker-freebsd) but it wasn't updated since 2015 and it is Docker v1 (it is v17 as of 2017). The other approach is to use official way of running Docker on platforms other than Linux. Well, somewhat official as Docker still does not support FreeBSD as a host officially. This is docker-machine tool which in turn will use VirtualBox to run a virtual machine with Linux and Docker engine. docker utility on the host will communicate with the engine inside VB where all the work will be done. This article describes what needs to be done to start using it. Before we begin you need VirtualBox installed. Do not skip adding /boot/loader.conf and /etc/rc.conf lines mentioned on that page. You won't need user inteface or anything, docker-machine will do all the work, just make sure VirtualBox is present and ready to be used. `pkg install docker docker-machine docker-compose' Docker will store its stuff in ~/.docker. You might not want the virtual machine image files to live in your home, in this case just create a symlink: mkdir ~/.docker ln -s /storage/docker ~/.docker/machine docker-machine create --driver virtualbox --virtualbox-memory 2048 --virtualbox-cpu-count 2 --virtualbox-disk-size 102400 --virtualbox-hostonly-cidr "10.2.1.1/24" docker1 Here's the example. We are creating machine named docker1. It is using VirtualBox driver, the vm has 2G of memory, 2 cores and 100G of disk space. docker-machine setups VirtualBox to use host-only network adapter (it will create vboxnet0 interface on the host automatically) and we are instructing it to use 10.2.1.1/24 as the address of this adapter — change it to what suits your needs or omit this flag (default is 192.168.99.1/24). And basically that is all. Check if it is running: docker-machine ls If you do open VirtualBox interface you will find a virtual machine named docker1 running. You can start/stop/whatever your machine using docker-machine utility. Here's how you can connect to the machine: docker utility by default tries to talk to Docker engine running on the same host. However with specific environment variables you can instruct it to talk to other host. docker-machine can export these variables for you. eval docker-machine env docker1 docker run hello-world There was quite a bit of discussion about docker at the FreeBSD developers summit in Cambridge during the first week of August. Two docker developers who had worked on the Mac OS X port, one of whom is an OpenBSD advocate, explained how docker has evolved, and the linux-isms have been abstracted away such that a truly native docker solution for FreeBSD can be built and maintained with a lot less headache than before I look forward to seeing if we can't make that happen *** The POSIX Shell And Utilities (http://shellhaters.org/) The POSIX Shell And Utilities Compiled for The Shell Hater's Handbook *** PostgreSQL – logging to a file (http://dan.langille.org/2017/07/31/postgresql-logging-to-a-file/) These steps were carried out on FreeBSD 11.0 with PostgreSQL 9.6 (two of my favorite tools). I like logging. I like logging PostgreSQL. With logs, you can see what happened. Without, you can only guess. Setting up logging for PostgreSQL involves several parts, each of which must be completed or else I don't get what I want. This is not a criticism of PostgreSQL. It's a feature. I am documenting this because each time I configure a new PostgreSQL instance, it takes me more than one iteration to get it working. The goal: this post lets both you and me get it right the first time. The parts include: + Telling PostgreSQL to log via syslog + Telling FreeBSD to local postgres to /var/log/postgres.log (my preference). + Telling PostgreSQL the things you want logged. + Changes to postgresql.conf The file location varies with the version installed. For PostgreSQL 9.6 on FreeBSD, the file is /var/db/postgres/data96/postgresql.conf (adjust 96 according to the version installed). I made these changes to that file. log_destination = 'syslog' log_min_messages = notice log_min_error_statement = notice log_checkpoints = on log_lock_waits = on log_timezone = 'UTC' By default, PostgreSQL logs to the local0 facility and is controlled by the syslog_facility in postgresql.conf. This will be used in syslog.conf (see the next section of this post). The above mentioned changes require a reload: service postgresql reload Changes to /etc/syslog.conf Now that we have PostgreSQL logging to syslog, we want to tell syslog where to put those messages. I changed this line in /etc/syslog.conf:*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err /var/log/messages With .notice pulling in some local0 messages, adding local0.none to the line will free the messages up for later use in the configuration file. Otherwise, the PostgreSQL messages will be in /var/log/messages. The changed line is: `.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err;local0.none /var/log/messages Then, to get the messages into my preferred location, I added this to the file: local0.* /var/log/postgresql.log` Log file rotation For rotating my log file, I added a new file: /usr/local/etc/newsyslog.conf.d/postgresql96 /var/log/postgresql.log pgsql:wheel 640 7 * $D0 GB /var/db/postgres/data96/postmaster.pid 30 Before restarting syslog, I did this, so the destination file existed. This isn't always/strictly necessary, but because the ownership is not chown root:wheel, I do it to get that part set. touch /var/log/postgresql.log chown pgsql:wheel Restarting syslog: sudo kill -HUP `sudo cat /var/run/syslog.pid ` That's it Now you should see PostgreSQL logging in /var/log/postgresql.log. mandoc-1.14.2 released (http://undeadly.org/cgi?action=article&sid=20170729122350) i just released portable mandoc-1.14.2. It is available now from http://mandoc.bsd.lv/ (http://mandoc.bsd.lv/). ```From: Ingo Schwarze schwarze@usta.de Date: Fri, 28 Jul 2017 20:12:44 +0200 To: discuss@mandoc.bsd.lv Subject: mandoc-1.14.2 released Hi, i just released portable mandoc-1.14.2. It is available now from http://mandoc.bsd.lv/ . All downstream maintainers are encouraged to update their ports and packages from 1.14.1 to 1.14.2. Mandoc 1.14.2 is a feature release introducing: a new -Tmarkdown output mode anchors for deep linking into -Thtml manual pages a superset of the functionality of the former mdoclint(1) utility a new -Wstyle message level with several new messages automatic line breaking inside individual tbl(7) cells a rewrite of the eqn(7) lexer, and some eqn(7) rendering improvements support for many additional low-level roff(7) features and various smaller features and bug fixes. For more details, see: http://mandoc.bsd.lv/NEWS With the improved mandoc features, only twenty-five out of the ten thousand software packages in the OpenBSD ports tree still need groff to format their manual pages. Since the project has been called "mandoc" rather than "mdocml" for several years now, the website, the distribution tarball, and the source extraction directory are now also called "mandoc" rather than "mdocml". The release was tested on the following systems: + OpenBSD-current and OpenBSD-stable + NetBSD-current + illumos + Debian Linux + Void Linux x86_64 glibc and musl + Crux Linux + SunOS 5.11.2, 5.10, and 5.9 As before, catman(8) and the regression suite cannot be used on SunOS 5.10 and SunOS 5.9. A big thanks to everybody who provided patches, bug reports, feature suggestions, advice, and help with testing! Yours, Ingo``` Beastie Bits A good looking terminal emulator which mimics the old cathode display. Available in x11/cool-retro-terminal (https://github.com/Swordfish90/cool-retro-term) Milestone Complete! OpenRC conversion (https://www.trueos.org/blog/milestone-complete-openrc-conversion/) Healthy developer interaction between FreeBSD and IllumOS re: mdb (https://illumos.topicbox.com/groups/developer/discussions/T5eae6079331c4df4) Large Batch of Kernel Errata Patches Released (http://undeadly.org/cgi?action=article&sid=20170804053102) opnsense 17.7 released (https://opnsense.org/opnsense-17-7-released/) Twitter Co-Founder and CEO states “FreeBSD rules them all” (https://twitter.com/jack/status/892605692317650944) Hurry up and register for vBSDCon September 7-9 (http://www.verisign.com/en_US/internet-technology-news/verisign-events/vbsdcon/index.xhtml?dmn=vBSDcon.com) and EuroBSDCon September 21-24 (https://2017.eurobsdcon.org/) *** Feedback/Questions Dominik - Monitoring Software (http://dpaste.com/08971FQ) Darren - Wonderful Awk (http://dpaste.com/0YCS4DN) Andrew - Thanks (http://dpaste.com/0ZREKTV) Jens - Migration Questions (http://dpaste.com/1GVZNWN) ***

We look at an OpenBSD setup on a new laptop, revel in BSDCan trip reports, and visit daemons and friendly ninjas. This episode was brought to you by Headlines OpenBSD and the modern laptop (http://bsdly.blogspot.de/2017/07/openbsd-and-modern-laptop.html) Peter Hansteen has a new blog post about OpenBSD (http://www.openbsd.org/) on laptops: Did you think that OpenBSD is suitable only for firewalls and high-security servers? Think again. Here are my steps to transform a modern mid to high range laptop into a useful Unix workstation with OpenBSD. One thing that never ceases to amaze me is that whenever I'm out and about with my primary laptop at conferences and elsewhere geeks gather, a significant subset of the people I meet have a hard time believing that my laptop runs OpenBSD, and that it's the only system installed. and then it takes a bit of demonstrating that yes, the graphics runs with the best available resolution the hardware can offer, the wireless network is functional, suspend and resume does work, and so forth. And of course, yes, I do use that system when writing books and articles too. Apparently heavy users of other free operating systems do not always run them on their primary workstations. Peter goes on to describe the laptops he's had over the years (all running OpenBSD) and after BSDCan 2017, he needed a new one due to cracks in the display. So the time came to shop around for a replacement. After a bit of shopping around I came back to Multicom, a small computers and parts supplier outfit in rural Åmli in southern Norway, the same place I had sourced the previous one. One of the things that attracted me to that particular shop and their own-branded offerings is that they will let you buy those computers with no operating system installed. That is of course what you want to do when you source your operating system separately, as we OpenBSD users tend to do. The last time around I had gone for a "Thin and lightweight" 14 inch model (Thickness 20mm, weight 2.0kg) with 16GB RAM, 240GB SSD for system disk and 1TB HD for /home (since swapped out for a same-size SSD, as the dmesg will show). Three years later, the rough equivalent with some added oomph for me to stay comfortable for some years to come ended me with a 13.3 inch model, 18mm and advertised as 1.3kg (but actually weighing in at 1.5kg, possibly due to extra components), 32GB RAM, 512GB SSD and 2TB harddisk. For now the specification can be viewed online here (https://www.multicom.no/systemconfigurator.aspx?q=st:10637291;c:100559;fl:0#4091-10500502-1;4086-10637290-1;4087-8562157-2;4088-9101982-1;4089-9101991-1) (the site language is Norwegian, but product names and units of measure are not in fact different). The OpenBSD installer is a wonder of straightforward, no-nonsense simplicity that simply gets the job done. Even so, if you are not yet familiar with OpenBSD, it is worth spending some time reading the OpenBSD FAQ's installation guidelines and the INSTALL.platform file (in our case, INSTALL.amd64) to familiarize yourself with the procedure. If you're following this article to the letter and will be installing a snapshot, it is worth reading the notes on following -current too. The main hurdle back when I was installing the 2014-vintage 14" model was getting the system to consider the SSD which showed up as sd1 the automatic choice for booting (I solved that by removing the MBR, setting the size of the MBR on the hard drive that showed up as sd0 to 0 and enlarging the OpenBSD part to fill the entire drive). + He goes on to explain the choices he made in the installer and settings made after the reboot to set up his work environment. Peter closes with: If you have any questions on running OpenBSD as a primary working environment, I'm generally happy to answer but in almost all cases I would prefer that you use the mailing lists such as misc@openbsd.org or the OpenBSD Facebook (https://www.facebook.com/groups/2210554563/) group so the question and hopefully useful answers become available to the general public. Browsing the slides for my recent OpenBSD and you (https://home.nuug.no/~peter/openbsd_and_you/) user group talk might be beneficial if you're not yet familiar with the system. And of course, comments on this article are welcome. BSDCan 2017 Trip Report: Roller Angel (https://www.freebsdfoundation.org/blog/2017-bsdcan-trip-report-roller-angel/) We could put this into next week's show, because we have another trip report already that's quite long. After dropping off my luggage, I headed straight over to the Goat BoF which took place at The Royal Oak. There were already a number of people there engaged in conversation with food and drink. I sat down at a table and was delighted that the people sitting with me were also into the BSD's and were happy to talk about it the whole time. I felt right at home from the start as people were very nice to me, and were interested in what I was working on. I honestly didn't know that I would fit in so well. I had a preconceived notion that people may be a bit hard to approach as they are famous and so technically advanced. At first, people seemed to only be working in smaller circles. Once you get more familiar with the faces, you realize that these circles don't always contain the same people and that they are just people talking about specific topics. I found that it was easy to participate in the conversation and also found out that people are happy to get your feedback on the subject as well. I was actually surprised how easily I got along with everyone and how included I felt in the activities. I volunteered to help wherever possible and got to work on the video crew that recorded the audio and slides of the talks. The people at BSDCan are incredibly easy to talk to, are actually interested in what you're doing with BSD, and what they can do to help. It's nice to feel welcome in the community. It's like going home. Dan mentioned in his welcome on the first day of BSDCan that the conference is like home for many in the community. The trip report is very detailed and chronicles the two days of the developer summit, and the two days of the conference There was some discussion about a new code of conduct by Benno Rice who mentioned that people are welcome to join a body of people that is forming that helps work out issues related to code of conduct and forwards their recommendations on to core. Next, Allan introduced the idea of creating a process for formally discussing big project changes or similar discussions that is going to be known as FCP or FreeBSD Community Proposal. In Python we have the Python Enhancement Proposal or PEP which is very similar to the idea of FCP. I thought this idea is a great step for FreeBSD to be implementing as it has been a great thing for Python to have. There was some discussion about taking non-code contributions from people and how to recognize those people in the project. There was a suggestion to have a FreeBSD Member status created that can be given to people whose non-code contributions are valuable to the project. This idea seemed to be on a lot of people's minds as something that should be in place soon. The junior jobs on the FreeBSD Wiki were also brought up as a great place to look for ideas on how to get involved in contributing to FreeBSD. Roller wasted no time, and started contributing to EdgeBSD at the conference. On the first day of BSDCan I arrived at the conference early to coordinate with the team that records the talks. We selected the rooms that each of us would be in to do the recording and set up a group chat via WhatsApp for coordination. Thanks to Roller, Patrick McAvoy, Calvin Hendryx-Parker, and all of the others who volunteered their time to run the video and streaming production at BSDCan, as well as all others who volunteered, even if it was just to carry a box. BSDCan couldn't happen without the army of volunteers. After the doc lounge, I visited the Hacker Lounge. There were already several tables full of people talking and working on various projects. In fact, there was a larger group of people who were collaborating on the new libtrue library that seemed to be having a great time. I did a little socializing and then got on my laptop and did some more work on the documentation using my new skills. I really enjoyed having a hacker lounge to go to at night. I want to give a big thank you to the FreeBSD Foundation for approving my travel grant. It was a great experience to meet the community and participate in discussions. I'm very grateful that I was able to attend my first BSDCan. After visiting the doc lounge a few times, I managed to get comfortable using the tools required to edit the documentation. By the end of the conference, I had submitted two documentation patches to the FreeBSD Bugzilla with several patches still in progress. Prior to the conference I expected that I would be spending a lot of time working on my Onion Omega and Edge Router Lite projects that I had with me, but I actually found that there was always something fun going on that I would rather do or work on. I can always work on those projects at home anyway. I had a good time working with the FreeBSD community and will continue working with them by editing the documentation and working with Bugzilla. One of the things I enjoy about these trip reports is when they help convince other people to make the trip to their first conference. Hopefully by sharing their experience, it will convince you to come to the next conference: vBSDCon in Virginia, USA: Sept 7-9 EuroBSDCon in Paris, France: Sept 21-24 BSDTW in Taipei, Taiwan: November 11-12 (CFP ends July 31st) *** BSDCan 2017 - Trip report double-p (http://undeadly.org/cgi?action=article&sid=20170629150641) Prologue Most overheard in Tokyo was "see you in Ottawaaaaah", so with additional "personal item" being Groff I returned home to plan the trip to BSDCan. Dan was very helpful with getting all the preparations (immigration handling), thanks for that. Before I could start, I had to fix something: the handling of the goat. With a nicely created harness, I could just hang it along my backpack. Done that it went to the airport of Hamburg and check-in for an itinerary of HAM-MUC-YUL. While the feeder leg was a common thing, boarding to YUL was great - cabin-crew likes Groff :) Arriving in Montreal was like entering a Monsoon zone or something, sad! After the night the weather was still rain-ish but improving and i shuttled to Dorval VIARail station to take me to Ottawa (ever avoid AirCanada, right?). Train was late, but the conductor (or so) was nice to talk to - and wanted to know about Groff's facebook page :-P. Picking a cab in Ottawa to take me to "Residence" was easy at first - just that it was the wrong one. Actually my fault and so I had a "nice, short" walk to the actual one in the rain with wrong directions. Eventually I made it and after unpacking, refreshment it was time to hit the Goat BOF! Day 1 Since this was my first BSDCan I didnt exactly knew what to expect from this BOF. But it was like, we (Keeper, Dan, Allan, ..) would talk about "who's next" and things like that. How mistaken I was :). Besides the sheer amount of BSD people entering the not-so-yuuge Oak some Dexter sneaked in camouflage. The name-giver got a proper position to oversee the mess and I was glad I did not leave him behind after almost too many Creemores. Day 2 Something happened it's crystal blue on the "roof" and sun is trying its best to wake me up. To start the day, I pick breakfast at 'Father+Sons' - I can really recommend that. Very nice home made fries (almost hashbrowns) and fast delivery! Stuffed up I trott along to get to phessler's tutorial about BGP-for-sysadmins-and-developers. Peter did a great job, but the "lab" couldn't happen, since - oh surprise - the wifi was sluggish as hell. Must love the first day on a conference every time. Went to Hackroom in U90 afterwards, just to fix stuff "at home". IPsec giving pains again. Time to pick food+beer afterwards and since it's so easy to reach, we went to the Oak again. Having a nice backyard patio experience it was about time to meet new people. Cheers to Tom, Aaron, Nick, Philip and some more, we'd an awesome night there. I also invited some not-really-computer local I know by other means who was completly overwhelmed by what kind of "nerds" gather around BSD. He planned to stay "a beer" - and it was rather some more and six hours. Looks like "we" made some impression on him :). Day 3 Easy day, no tutorials at hand, so first picking up breakfast at F+S again and moving to hackroom in U90. Since I promised phessler to help with an localized lab-setup, I started to hack on a quick vagrant/ansible setup to mimic his BGP-lab and went quickly through most of it. Plus some more IPsec debugging and finally fixing it, we went early in the general direction of the Red Lion to pick our registration pack. But before that could happen it was called to have shawarma at 3brothers along. Given a tight hangover it wasn't the brightest idea to order a poutine m-(. Might be great the other day, it wasn't for me at the very time and had to throw away most of it :(. Eventually passing on to the Red Lion I made the next failure with just running into the pub - please stay at the front desk until "seated". I never get used to this concept. So after being "properly" seated, we take our beers and the registration can commence after we had half of it. So I register myself; btw it's a great idea to grant "not needed" stuff to charity. So dont pick "just because", think about it if you really need this or that gadget. Then I register Groff - he really needs badges - just to have Dru coming back to me some minutes later one to hand me the badge for Henning. That's just "amazing"; I dont know IF i want to break this vicious circle the other day, since it's so funny. Talked to Theo about the ongoing IPsec problems and he taught me about utrace(2) which looks "complicated" but might be an end of the story the other day. Also had a nice talk to Peter (H.) about some other ideas along books. BTW, did I pay for ongoing beers? I think Tom did - what a guy :). Arriving at the Residence, I had to find my bathroom door locked (special thing).. crazy thing is they dont have a master key at the venue, but to have to call in one from elsewhere. Short night shortened by another 30minutes :(. Day 4 Weather is improving into beach+sun levels - and it's Conference Day! The opening keynote from Geist was very interesting ("citation needed"). Afterwards I went to zfs-over-ssh, nothing really new (sorry Allan). But then Jason had a super interesting talk on how about to apply BSD for the health-care system in Australia. I hope I can help him with the last bits (rdomain!) in the end. While lunch I tried to recall my memories about utrace(2) while talking to Theo. Then it was about to present my talk and I think it was well perceipted. One "not so good" feedback was about not taking the audience more into account. I think I was asking every other five slides or so - but, well. The general feedback (in spoken terms) was quite good. I was a bit "confused" and I did likely a better job in Tokyo, but well. Happened we ended up in the Oak again.. thanks to mwl, shirkdog, sng, pitrh, kurtm for having me there :) Day 5 While the weather had to decide "what next", I rushed to the venue just to gather Reyk's talk about vmd(8). Afterwards it was MSTP from Paeps which was very interesting and we (OpenBSD) should look into it. Then happened BUG BOF and I invite all "coastal Germans" to cbug.de :) I had to run off for other reasons and came back to Dave's talk which was AWESOME. Following was Rod's talk.. well. While I see his case, that was very poor. The auction into closing was awesome again, and I spend $50 on a Tshirt. :) + Epilogue I totally got the exit dates wrong. So first cancel a booking of an Hotel and then rebook the train to YUL. So I have plenty of time "in the morning" to get breakfast with the local guy. After that he drives me to VIARail station and I dig into "business" cussions. Well, see you in Ottawa - or how about Paris, Taipei? Bind Broker (http://www.tedunangst.com/flak/post/bind-broker) Ted Unangst writes about an interesting idea he has He has a single big server, and lots of users who would like to share it, many want to run web servers. This would be great, but alas, archaic decisions made long ago mean that network sockets aren't really files and there's this weird concept of privileged ports. Maybe we could assign each user a virtual machine and let them do whatever they want, but that seems wasteful. Think of the megabytes! Maybe we could setup nginx.conf to proxy all incoming connections to a process of the user's choosing, but that only works for web sites and we want to be protocol neutral. Maybe we could use iptables, but nobody wants to do that. What we need is a bind broker. At some level, there needs to be some kind of broker that assigns IPs to users and resolves conflicts. It should be possible to build something of this nature given just the existing unix tools we have, instead of changing system design. Then we can deploy our broker to existing systems without upgrading or disrupting their ongoing operation. The bind broker watches a directory for the creation, by users, of unix domain sockets. Then it binds to the TCP port of the same name, and transfers traffic between them. A more complete problem specification is as follows. A top level directory, which contains subdirectories named after IP addresses. Each user is assigned a subdirectory, which they have write permission to. Inside each subdirectory, the user may create unix sockets named according to the port they wish to bind to. We might assign user alice the IP 10.0.0.5 and the user bob the IP 10.0.0.10. Then alice could run a webserver by binding to net/10.0.0.5/80 and bob could run a mail server by binding to net/10.0.0.10/25. This maps IP ownership (which doesn't really exist in unix) to the filesystem namespace (which does have working permissions). So this will be a bit different than jails. The idea is to use filesystem permissions to control which users can bind to which IP addresses and ports The broker is responsible for watching each directory. As new sockets are created, it should respond by binding to the appropriate port. When a socket is deleted, the network side socket should be closed as well. Whenever a connection is accepted on the network side, a matching connection is made on the unix side, and then traffic is copied across. A full set of example code is provided There's no completely portable way to watch a directory for changes. I'm using a kevent extension. Otherwise we might consider a timeout and polling with fstat, or another system specific interface (or an abstraction layer over such an interface). Otherwise, if one of our mappings is ready to read (accept), we have a new connection to handle. The first half is straightforward. We accept the connection and make a matching connect call to the unix side. Then I broke out the big cheat stick and just spliced the sockets together. In reality, we'd have to set up a read/copy/write loop for each end to copy traffic between them. That's not very interesting to read though. The full code, below, comes in at 232 lines according to wc. Minus includes, blank lines, and lines consisting of nothing but braces, it's 148 lines of stuff that actually gets executed by the computer. Add some error handling, and working read/write code, and 200 lines seems about right. A very interesting idea. I wonder about creating a virtual file system that would implement this and maybe do a bit more to fully flesh out this idea. What do you think? *** News Roundup Daemons and friendly Ninjas (https://euroquis.nl/bobulate/?p=1600) There's quite a lot of software that uses CMake as a (meta-)buildsystem. A quick count in the FreeBSD ports tree shows me 1110 ports (over a thousand) that use it. CMake generates buildsystem files which then direct the actual build — it doesn't do building itself. There are multiple buildsystem-backends available: in regular usage, CMake generates Makefiles (and does a reasonable job of producing Makefiles that work for GNU Make and for BSD Make). But it can generate Ninja, or Visual Studio, and other buildsystem files. It's quite flexible in this regard. Recently, the KDE-FreeBSD team has been working on Qt WebEngine, which is horrible. It contains a complete Chromium and who knows what else. Rebuilding it takes forever. But Tobias (KDE-FreeBSD) and Koos (GNOME-FreeBSD) noticed that building things with the Ninja backend was considerably faster for some packages (e.g. Qt WebEngine, and Evolution data-thingy). Tobias wanted to try to extend the build-time improvements to all of the CMake-based ports in FreeBSD, and over the past few days, this has been a success. Ports builds using CMake now default to using Ninja as buildsystem-backend. Here's a bitty table of build-times. These are one-off build times, so hardly scientifically accurate — but suggestive of a slight improvement in build time. Name Size GMake Ninja liblxt 50kB 0:32 0:31 llvm38 1655kB * 19:43 musescore 47590kB 4:00 3:54 webkit2-gtk3 14652kB 44:29 37:40 Or here's a much more thorough table of results from tcberner@, who did 5 builds of each with and without ninja. I've cut out the raw data, here are just the average-of-five results, showing usually a slight improvement in build time with Ninja. Name av make av ninj Delta D/Awo compiler-rt 00:08 00:07 -00:01 -14% openjpeg 00:06 00:07 +00:01 +17% marble 01:57 01:43 -00:14 -11% uhd 01:49 01:34 -00:15 -13% opencacscade 04:08 03:23 -00:45 -18% avidemux 03:01 02:49 -00:12 – 6% kdevelop 01:43 01:33 -00:10 – 9% ring-libclient 00:58 00:53 -00:05 – 8% Not everything builds properly with Ninja. This is usually due to missing dependencies that CMake does not discover; this shows up when foo depends on bar but no rule is generated for it. Depending on build order and speed, bar may be there already by the time foo gets around to being built. Doxygen showed this, where builds on 1 CPU core were all fine, but 8 cores would blow up occasionally. In many cases, we've gone and fixed the missing implicit dependencies in ports and upstreams. But some things are intractable, or just really need GNU Make. For this, the FreeBSD ports infrastructure now has a knob attached to CMake for switching a port build to GNU Make. Normal: USES=cmake Out-of-source: USES=cmake:outsource GNU Make: USES=cmake:noninja gmake OoS, GMake: USES=cmake:outsource,noninja gmake Bad: USES=cmake gmake For the majority of users, this has no effect, but for our package-building clusters, and for KDE-FreeBSD developers who build a lot of CMake-buildsystem software in a day it may add up to an extra coffee break. So I'll raise a shot of espresso to friendship between daemons and ninjas. Announcing the pkgsrc-2017Q2 release (http://mail-index.netbsd.org/pkgsrc-users/2017/07/10/msg025237.html) For the 2017Q2 release we welcome the following notable package additions and changes to the pkgsrc collection: Firefox 54 GCC 7.1 MATE 1.18 Ruby 2.4 Ruby on Rails 4.2 TeX Live 2017 Thunderbird 52.1 Xen 4.8 We say goodbye to: Ruby 1.8 Ruby 2.1 The following infrastructure changes were introduced: Implement optional new pkgtasks and init infrastructure for pkginstall. Various enhancements and fixes for building with ccache. Add support to USE_LANGUAGES for newer C++ standards. Enhanced support for SSP, FORTIFY, and RELRO. The GitHub mirror has migrated to https://github.com/NetBSD/pkgsrc In total, 210 packages were added, 43 packages were removed, and 1,780 package updates were processed since the pkgsrc-2017Q1 release. *** OpenBSD changes of note 624 (http://www.tedunangst.com/flak/post/openbsd-changes-of-note-624) There are a bunch, but here are a few that jump out: Start plugging some leaks. Compile kernels with umask 007. Install them minus read permissions. Pure preprocessor implementation of the roff .ec and .eo requests, though you are warned that very bad things will happen to anybody trying to use these macros in OpenBSD manuals. Random linking for arm64. And octeon. And alpha. And hppa. There's some variation by platform, because every architecture has the kernel loaded with different flavors of initial physical and virtual mappings. And landisk. And loongson. And sgi. And macppc. And a gap file for sparc64, but nobody yet dares split locore. And arm7. Errata for perl File::Path race condition. Some fixes for potential link attacks against cron. Add pledge violations to acct reporting. Take random linking to the next stage. More about KARL - kernel address randomized link. As noted, a few difficulties with hibernate and such, but the plan is coming together. Add a new function reorder_kernel() that relinks and installs the new kernel in the background on system startup. Add support for the bootblocks to detect hibernate and boot the previous kernel. Remove the poorly described “stuff” from ksh. Replace usage of TIOCSTI in csh using a more common IO loop. Kind of like the stuff in ksh, but part of the default command line editing and parsing code, csh would read too many characters, then send the ones it didn't like back into the terminal. Which is weird, right? Also, more importantly, eliminating the code that uses TIOCSTI to inject characters into ttys means that maybe TIOCSTI can be removed. Revamp some of the authentication logging in ssh. Add a verbose flag to rm so you can panic immediately upon seeing it delete the wrong file instead of waiting to discover your mistake after the fact. Update libexpat to version 2.2.1 which has some security fixes. Never trust an expat, that's my motto. Update inteldrm to code based on Linux 4.4.70. This brings us support for Skylake and Cherryview and better support for Broadwell and Valleyview. Also adds MST support. Fun times for people with newish laptops. *** OPNsense 17.1.9 released (https://opnsense.org/opnsense-17-1-9-released/) firewall: move gateway switching from system to firewall advanced settings firewall: keep category selection when changing tabs firewall: do not skip gateway switch parsing too early (contributed by Stephane Lesimple) interfaces: show VLAN description during edit firmware: opnsense-revert can now handle multiple packages at once firmware: opnsense-patch can now handle permission changes from patches dnsmasq: use canned –bogus-priv for noprivatereverse dnsmasq: separate log file, ACL and menu entries dynamic dns: fix update for IPv6 (contributed by Alexander Leisentritt) dynamic dns: remove usage of CURLAUTH_ANY (contributed by Alexander Leisentritt) intrusion detection: suppress “fast mode available” boot warning in PCAP mode openvpn: plugin framework adaption unbound: add local-zone type transparent for PTR zone (contributed by Davide Gerhard) unbound: separate log file, ACL and menu entries wizard: remove HTML from description strings mvc: group relation to something other than uuid if needed mvc: rework “item in” for our Volt templates lang: Czech to 100% translated (contributed by Pavel Borecki) plugins: zabbix-agent 1.1 (contributed by Frank Wall) plugins: haproxy 1.16 (contributed by Frank Wall) plugins: acme-client 1.8 (contributed by Frank Wall) plugins: tinc fix for switch mode (contributed by Johan Grip) plugins: monit 1.3 (contributed by Frank Brendel) src: support dhclient supersede statement for option 54 (contributed by Fabian Kurtz) src: add Intel Atom Cherryview SOC HSUART support src: add the ID for the Huawei ME909S LTE modem src: HardenedBSD Stack Clash mitigations[1] ports: sqlite 3.19.3[2] ports: openvpn 2.4.3[3] ports: sudo 1.8.20p2[4] ports: dnsmasq 2.77[5] ports: openldap 2.4.45[6] ports: php 7.0.20[7] ports: suricata 3.2.2[8] ports: squid 3.5.26[9] ports: carootnss 3.31 ports: bind 9.11.1-P2[10] ports: unbound 1.6.3[11] ports: curl 7.54.1[12] *** Beastie Bits Thinkpad x230 - trying to get TrackPoint / Touchpad working in X (http://lists.dragonflybsd.org/pipermail/users/2017-July/313519.html) FreeBSD deprecates all r-cmds (rcp, rlogin, etc.) (http://marc.info/?l=freebsd-commits-all&m=149918307723723&w=2) Bashfill - art for your terminal (https://max.io/bash.html) Go 1.9 release notes: NetBSD support is broken, please help (https://github.com/golang/go/commit/32002079083e533e11209824bd9e3a797169d1c4) Jest, A ReST api for creating and managing FreeBSD jails written in Go (https://github.com/altsrc-io/Jest) *** Feedback/Questions John - zfs send/receive (http://dpaste.com/3ANETHW#wrap) Callum - laptops (http://dpaste.com/11TV0BJ) & An update (http://dpaste.com/3A14BQ6#wrap) Lars - Snapshot of VM datadisk (http://dpaste.com/0MM37NA#wrap) Daryl - Jail managers (http://dpaste.com/0CDQ9EK#wrap) ***

In which we interview a unicorn, FreeNAS 11.0 is out, show you how to run Nextcloud in a FreeBSD jail, and talk about the connection between oil changes and software patches. This episode was brought to you by Headlines FreeNAS 11.0 is Now Here (http://www.freenas.org/blog/freenas-11-0/) The FreeNAS blog informs us: After several FreeNAS Release Candidates, FreeNAS 11.0 was released today. This version brings new virtualization and object storage features to the World's Most Popular Open Source Storage Operating System. FreeNAS 11.0 adds bhyve virtual machines to its popular SAN/NAS, jails, and plugins, letting you use host web-scale VMs on your FreeNAS box. It also gives users S3-compatible object storage services, which turns your FreeNAS box into an S3-compatible server, letting you avoid reliance on the cloud. FreeNAS 11.0 also introduces the beta version of a new administration GUI. The new GUI is based on the popular Angular framework and the FreeNAS team expects the GUI to be themeable and feature complete by 11.1. The new GUI follows the same flow as the existing GUI, but looks better. For now, the FreeNAS team has released it in beta form to get input from the FreeNAS community. The new GUI, as well as the classic GUI, are selectable from the login screen. Also new in FreeNAS 11 is an Alert Service page which configures the system to send critical alerts from FreeNAS to other applications and services such as Slack, PagerDuty, AWS, Hipchat, InfluxDB, Mattermost, OpsGenie, and VictorOps. FreeNAS 11.0 has an improved Services menu that adds the ability to manage which services and applications are started at boot. The FreeNAS community is large and vibrant. We invite you to join us on the FreeNAS forum (https://forums.freenas.org/index.php) and the #freenas IRC channel on Freenode. To download FreeNAS and sign-up for the FreeNAS Newsletter, visit freenas.org/download (http://www.freenas.org/download/). Building an IPsec Gateway With OpenBSD (https://www.exoscale.ch/syslog/2017/06/26/building-an-ipsec-gateway-with-openbsd/) Pierre-Yves Ritschard wrote the following blog article: With private networks just released on Exoscale, there are now more options to implement secure access to Exoscale cloud infrastructure. While we still recommend the bastion approach, as detailed in this article (https://www.exoscale.ch/syslog/2016/01/15/secure-your-cloud-computing-architecture-with-a-bastion/), there are applications or systems which do not lend themselves well to working this way. In these cases, the next best thing is building IPsec gateways. IPsec is a protocol which works directly at layer 3. It uses its configuration to determine which network flows should be sent encrypted on the wire. Once IPsec is correctly configured, selected network flows are transparently encrypted and applications do not need to modify anything to benefit from secured traffic. In addition to encryption, IPSec also authenticates the end points, so you can be sure you are exchanging packets with a trusted host For the purposes of this article we will work under the following assumptions: We want a host to network setup, providing access to cloud-hosted infrastructure from a desktop environment. Only stock tooling should be used on desktop environment, no additional VPN client should be needed. In this case, to ensure no additional software is needed on the client, we will configure an L2TP/IPsec gateway. This article will use OpenBSD as the operating system to implement the gateway. While this choice may sound surprising, OpenBSD excels at building gateways of all sorts thanks to its simple configuration formats and inclusion of all necessary software and documentation to do so in the base system. The tutorial assumes you have setup a local network between the hosts in the cloud, and walks through the configuration of an OpenBSD host as a IPsec gateway On the OpenBSD host, all necessary software is already installed. We will configure the system, as well as pf, npppd, and ipsec + Configure L2TP + Configure IPsec + Configure NAT + Enabled services: ipsec isakmpd npppd The tutorial then walks through configuring a OS X client, but other desktops will be very similar *** Running Nextcloud in a jail on FreeBSD (https://ramsdenj.com/2017/06/05/nextcloud-in-a-jail-on-freebsd.html) I recently setup Nextcloud 12 inside a FreeBSD jail in order to allow me access to files i might need while at University. I figured this would be a optimal solution for files that I might need access to unexpectedly, on computers where I am not in complete control. My Nextcloud instance is externally accessible, and yet if someone were to get inside my Jail, I could rest easy knowing they still didn't have access to the rest of my host server. I chronicled the setup process including jail setup using iocage, https with Lets Encrypt, and full setup of the web stack. Nextcloud has a variety of features such as calendar synchronization, email, collaborative editing, and even video conferencing. I haven't had time to play with all these different offerings and have only utilized the file synchronization, but even if file sync is not needed, Nextcloud has many offerings that make it worth setting up. MariaDB, PHP 7.0, and Apache 2.4 To manage my jails I'm using iocage. In terms of jail managers it's a fairly new player in the game of jail management and is being very actively developed. It just had a full rewrite in Python, and while the code in the background might be different, the actual user interface has stayed the same. Iocage makes use of ZFS clones in order to create “base jails”, which allow for sharing of one set of system packages between multiple jails, reducing the amount of resources necessary. Alternatively, jails can be completely independent from each other; however, using a base jail makes it easier to update multiple jails as well. + pkg install iocage + sysrc iocageenable=YES + iocage fetch -r 11.0-RELEASE + iocage create tag="stratus" jailzfs=on vnet=off boot=on ip4_addr="sge0|172.20.0.100/32" -r 11.0-RELEASE + iocage start stratus + iocage console stratus I have chosen to provide storage to the Nextcloud Jail by mounting a dataset over NFS on my host box. This means my server can focus on serving Nextcloud and my storage box can focus on housing the data. The Nextcloud Jail is not even aware of this since the NFS Mount is simply mounted by the host server into the jail. The other benefit of this is the Nextcloud jail doesn't need to be able to see my storage server, nor the ability to mount the NFS share itself. Using a separate server for storage isn't necessary and if the storage for my Nextcloud server was being stored on the same server I would have created a ZFS dataset on the host and mounted it into the jail. Next I set up a dataset for the database and delegated it into the jail. Using a separate dataset allows me to specify certain properties that are better for a database, it also makes migration easier in case I ever need to move or backup the database. With most of the requirements in place it was time to start setting up Nextcloud. The requirements for Nextcloud include your basic web stack of a web server, database, and PHP. Also covers the setup of acme.sh for LetsEncrypt. This is now available as a package, and doesn't need to be manually fetched Install a few more packages, and do a bit of configuration, and you have a NextCloud server *** Historical: My first OpenBSD Hackathon (http://bad.network/historical-my-first-openbsd-hackathon.html) This is a blog post by our friend, and OpenBSD developer: Peter Hessler This is a story about encouragement. Every time I use the word "I", you should think "I as in me, not I as in the author". In 2003, I was invited to my first OpenBSD Hackathon. Way before I was into networking, I was porting software to my favourite OS. Specifically, I was porting games. On the first night most of the hackathon attendees end up at the bar for food and beer, and I'm sitting next to Theo de Raadt, the founder of OpenBSD. At some point during the evening, he's telling me about all of these "crazy" ideas he has about randomizing libraries, and protections that can be done in ld.so. (ld.so is the part of the OS that loads the libraries your program needs. It's, uh, kinda important.) Theo is encouraging me to help implement some of these ideas! At some point I tell Theo "I'm just a porter, I don't know C." Theo responds with "It isn't hard, I'll have Dale (Rahn) show you how ld.so works, and you can do it." I was hoping that all of this would be forgotten by the next day, but sure enough Dale comes by. "Hey, are you Peter? Theo wanted me to show you how ld.so works" Dale spends an hour or two showing me how it works, the code structure, and how to recover in case of failure. At first I had lots of failures. Then more failures. And even more failures. Once, I broke my machine so badly I had to reinstall it. I learned a lot about how an OS works during this. But, I eventually started doing changes without it breaking. And some even did what I wanted! By the end of the hackathon I had came up with a useful patch, that was committed as part of a larger change. I was a nobody. With some encouragement, enough liquid courage to override my imposter syndrome, and a few hours of mentoring, I'm now doing big projects. The next time you're sitting at a table with someone new to your field, ask yourself: how can you encourage them? You just might make the world better. Thank you Dale. And thank you Theo. Everyone has to start somewhere. One of the things that sets the BSDs apart from certain other open source operating systems, is the welcoming community, and the tradition of mentorship. Sure, someone else in the OpenBSD project could have done the bits that Peter did, likely a lot more quickly, but then OpenBSD wouldn't have gained a new committer. So, if you are interested in working on one of the BSDs, reach out, and we'll try to help you find a mentor. What part of the system do you want to work on? *** Interview - Dan McDonald - allcoms@gmail.com (mailto:allcoms@gmail.com) (danboid) News Roundup FreeBSD 11.1-RC1 Available (https://lists.freebsd.org/pipermail/freebsd-stable/2017-July/087340.html) 11.1-RC1 Installation images are available for: amd64, i386 powerpc, powerpc64 sparc64 armv6 BANANAPI, BEAGLEBONE, CUBIEBOARD, CUBIEBOARD2, CUBOX-HUMMINGBOARD, GUMSTIX, RPI-B, RPI2, PANDABOARD, WANDBOARD aarch64 (aka arm64), including the RPI3, Pine64, OverDrive 1000, and Cavium Server A summary of changes since BETA3 includes: Several build toolchain related fixes. A use-after-free in RPC client code has been corrected. The ntpd(8) leap-seconds file has been updated. Various VM subsystem fixes. The '_' character is now allowed in newfs(8) labels. A potential sleep while holding a mutex has been corrected in the sa(4) driver. A memory leak in an ioctl handler has been fixed in the ses(4) driver. Virtual Machine Disk Images are available for the amd64 and i386 architectures. Amazon EC2 AMI Images of FreeBSD/amd64 EC2 AMIs are available The freebsd-update(8) utility supports binary upgrades of amd64 and i386 systems running earlier FreeBSD releases. Systems running earlier FreeBSD releases can upgrade as follows: freebsd-update upgrade -r 11.1-RC1 During this process, freebsd-update(8) may ask the user to help by merging some configuration files or by confirming that the automatically performed merging was done correctly. freebsd-update install The system must be rebooted with the newly installed kernel before continuing. shutdown -r now After rebooting, freebsd-update needs to be run again to install the new userland components: freebsd-update install It is recommended to rebuild and install all applications if possible, especially if upgrading from an earlier FreeBSD release, for example, FreeBSD 10.x. Alternatively, the user can install misc/compat10x and other compatibility libraries, afterwards the system must be rebooted into the new userland: shutdown -r now Finally, after rebooting, freebsd-update needs to be run again to remove stale files: freebsd-update install Oil changes, safety recalls, and software patches (http://www.daemonology.net/blog/2017-06-14-oil-changes-safety-recalls-software-patches.html) Every few months I get an email from my local mechanic reminding me that it's time to get my car's oil changed. I generally ignore these emails; it costs time and money to get this done (I'm sure I could do it myself, but the time it would cost is worth more than the money it would save) and I drive little enough — about 2000 km/year — that I'm not too worried about the consequences of going for a bit longer than nominally advised between oil changes. I do get oil changes done... but typically once every 8-12 months, rather than the recommended 4-6 months. From what I've seen, I don't think I'm alone in taking a somewhat lackadaisical approach to routine oil changes. On the other hand, there's another type of notification which elicits more prompt attention: Safety recalls. There are two good reasons for this: First, whether for vehicles, food, or other products, the risk of ignoring a safety recall is not merely that the product will break, but rather that the product will be actively unsafe; and second, when there's a safety recall you don't have to pay for the replacement or fix — the cost is covered by the manufacturer. I started thinking about this distinction — and more specifically the difference in user behaviour — in the aftermath of the "WannaCry" malware. While WannaCry attracted widespread attention for its "ransomware" nature, the more concerning aspect of this incident is how it propagated: By exploiting a vulnerability in SMB for which Microsoft issued patches two months earlier. As someone who works in computer security, I find this horrifying — and I was particularly concerned when I heard that the NHS was postponing surgeries because they couldn't access patient records. Think about it: If the NHS couldn't access patient records due to WannaCry, it suggests WannaCry infiltrated systems used to access patient records — meaning that someone else exploiting the same vulnerabilities could have accessed those records. The SMB subsystem in Windows was not merely broken; until patches were applied, it was actively unsafe. I imagine that most people in my industry would agree that security patches should be treated in the same vein as safety recalls — unless you're certain that you're not affected, take care of them as a matter of urgency — but it seems that far more users instead treat security patches more like oil changes: something to be taken care of when convenient... or not at all, if not convenient. It's easy to say that such users are wrong; but as an industry it's time that we think about why they are wrong rather than merely blaming them for their problems. There are a few factors which I think are major contributors to this problem. First, the number of updates: When critical patches occur frequently enough to become routine, alarm fatigue sets in and people cease to give the attention updates deserve, even if on a conscious level they still recognize the importance of applying updates. Colin also talks about his time as the FreeBSD Security Officer, and the problems in ensuring the patches are correct and do not break the system when installed He also points out the problem of systems like Windows Update, the combines optional updates, and things like its license checking tool, in the same interface that delivers important updates. Or my recent machines, that gets constant popups about how some security updates will not be delivered because my processor is too new. My bank sends me special offers in the mail but phones if my credit card usage trips fraud alarms; this is the sort of distinction in intrusiveness we should see for different types of software updates Finally, I think there is a problem with the mental model most people have of computer security. Movies portray attackers as geniuses who can break into any system in minutes; journalists routinely warn people that "nobody is safe"; and insurance companies offer insurance against "cyberattacks" in much the same way as they offer insurance against tornados. Faced with this wall of misinformation, it's not surprising that people get confused between 400 pound hackers sitting on beds and actual advanced persistent threats. Yes, if the NSA wants to break into your computer, they can probably do it — but most attackers are not the NSA, just like most burglars are not Ethan Hunt. You lock your front door, not because you think it will protect you from the most determined thieves, but because it's an easy step which dramatically reduces your risk from opportunistic attack; but users don't see applying security updates as the equivalent of locking their front door when they leave home. SKIP grep, use AWK (http://blog.jpalardy.com/posts/skip-grep-use-awk/) This is a tip from Jonathan Palardy in a series of blog posts about awk. It is especially helpful for people who write a lot of shell scripts or are using a lot of pipes with awk and grep. Over the years, I've seen many people use this pattern (filter-map): $ [data is generated] | grep something | awk '{print $2}' but it can be shortened to: $ [data is generated] | awk '/something/ {print $2}' AWK can take a regular expression (the part between the slashes) and matches that to the input. Anything that matches is being passed to the print $2 action (to print the second column). Why would I do this? I can think of 4 reasons: *it's shorter to type *it spawns one less process *awk uses modern (read “Perl”) regular expressions, by default – like grep -E *it's ready to “augment” with more awk How about matching the inverse (search for patterns that do NOT match)? But “grep -v” is OK… Many people have pointed out that “grep -v” can be done more concisely with: $ [data is generated] | awk '! /something/' See if you have such combinations of grep piped to awk and fix those in your shell scripts. It saves you one process and makes your scripts much more readable. Also, check out the other intro links on the blog if you are new to awk. *** vim Adventures (https://vim-adventures.com) This website, created by Doron Linder, will playfully teach you how to use vim. Hit any key to get started and follow the instructions on the playing field by moving the cursor around. There is also a menu in the bottom left corner to save your game. Try it out, increase your vim-fu, and learn how to use a powerful text editor more efficiently. *** Beastie Bits Slides from PkgSrcCon (http://pkgsrc.org/pkgsrcCon/2017/talks.html) OpenBSD's doas adds systemd compat shim (http://marc.info/?l=openbsd-tech&m=149902196520920&w=2) Deadlock Empire -- “Each challenge below is a computer program of two or more threads. You take the role of the Scheduler - and a cunning one! Your objective is to exploit flaws in the programs to make them crash or otherwise malfunction.” (https://deadlockempire.github.io/) EuroBSDcon 2017 Travel Grant Application Now Open (https://www.freebsdfoundation.org/blog/eurobsdcon-2017-travel-grant-application-now-open/) Registration for vBSDCon is open (http://www.vbsdcon.com/) - Registration is only $100 if you register before July 31. Discount hotel rooms arranged at the Hyatt for only $100/night while supplies last. BSD Taiwan call for papers opens, closes July 31st (https://bsdtw.org/)Windows Application Versand *** Feedback/Questions Joseph - Server Monitoring (http://dpaste.com/2AM6C2H#wrap) Paulo - Updating Jails (http://dpaste.com/1Z4FBE2#wrap) Kevin - openvpn server (http://dpaste.com/2MNM9GJ#wrap) Todd - several questions (http://dpaste.com/17BVBJ3#wrap) ***

This week on BSDNow, reports from AsiaBSDcon, TrueOS and FreeBSD news, Optimizing IllumOS Kernel, your questions and more. This episode was brought to you by Headlines AsiaBSDcon Reports and Reviews () AsiaBSDcon schedule (https://2017.asiabsdcon.org/program.html.en) Schedule and slides from the 4th bhyvecon (http://bhyvecon.org/) Michael Dexter's trip report on the iXsystems blog (https://www.ixsystems.com/blog/ixsystems-attends-asiabsdcon-2017) NetBSD AsiaBSDcon booth report (http://mail-index.netbsd.org/netbsd-advocacy/2017/03/13/msg000729.html) *** TrueOS Community Guidelines are here! (https://www.trueos.org/blog/trueos-community-guidelines/) TrueOS has published its new Community Guidelines The TrueOS Project has existed for over ten years. Until now, there was no formally defined process for interested individuals in the TrueOS community to earn contributor status as an active committer to this long-standing project. The current core TrueOS developers (Kris Moore, Ken Moore, and Joe Maloney) want to provide the community more opportunities to directly impact the TrueOS Project, and wish to formalize the process for interested people to gain full commit access to the TrueOS repositories. These describe what is expected of community members and committers They also describe the process of getting commit access to the TrueOS repo: Previously, Kris directly handed out commit bits. Now, the Core developers have provided a small list of requirements for gaining a TrueOS commit bit: Create five or more pull requests in a TrueOS Project repository within a single six month period. Stay active in the TrueOS community through at least one of the available community channels (Gitter, Discourse, IRC, etc.). Request commit access from the core developers via core@trueos.org OR Core developers contact you concerning commit access. Pull requests can be any contribution to the project, from minor documentation tweaks to creating full utilities. At the end of every month, the core developers review the commit logs, removing elements that break the Project or deviate too far from its intended purpose. Additionally, outstanding pull requests with no active dissension are immediately merged, if possible. For example, a user submits a pull request which adds a little-used OpenRC script. No one from the community comments on the request or otherwise argues against its inclusion, resulting in an automatic merge at the end of the month. In this manner, solid contributions are routinely added to the project and never left in a state of “limbo”. The page also describes the perks of being a TrueOS committer: Contributors to the TrueOS Project enjoy a number of benefits, including: A personal TrueOS email alias: @trueos.org Full access for managing TrueOS issues on GitHub. Regular meetings with the core developers and other contributors. Access to private chat channels with the core developers. Recognition as part of an online Who's Who of TrueOS developers. The eternal gratitude of the core developers of TrueOS. A warm, fuzzy feeling. Intel Donates 250.000 $ to the FreeBSD Foundation (https://www.freebsdfoundation.org/news-and-events/latest-news/new-uranium-level-donation-and-collaborative-partnership-with-intel/) More details about the deal: Systems Thinking: Intel and the FreeBSD Project (https://www.freebsdfoundation.org/blog/systems-thinking-intel-and-the-freebsd-project/) Intel will be more actively engaging with the FreeBSD Foundation and the FreeBSD Project to deliver more timely support for Intel products and technologies in FreeBSD. Intel has contributed code to FreeBSD for individual device drivers (i.e. NICs) in the past, but is now seeking a more holistic “systems thinking” approach. Intel Blog Post (https://01.org/blogs/imad/2017/intel-increases-support-freebsd-project) We will work closely with the FreeBSD Foundation to ensure the drivers, tools, and applications needed on Intel® SSD-based storage appliances are available to the community. This collaboration will also provide timely support for future Intel® 3D XPoint™ products. Thank you very much, Intel! *** Applied FreeBSD: Basic iSCSI (https://globalengineer.wordpress.com/2017/03/05/applied-freebsd-basic-iscsi/) iSCSI is often touted as a low-cost replacement for fibre-channel (FC) Storage Area Networks (SANs). Instead of having to setup a separate fibre-channel network for the SAN, or invest in the infrastructure to run Fibre-Channel over Ethernet (FCoE), iSCSI runs on top of standard TCP/IP. This means that the same network equipment used for routing user data on a network could be utilized for the storage as well. This article will cover a very basic setup where a FreeBSD server is configured as an iSCSI Target, and another FreeBSD server is configured as the iSCSI Initiator. The iSCSI Target will export a single disk drive, and the initiator will create a filesystem on this disk and mount it locally. Advanced topics, such as multipath, ZFS storage pools, failover controllers, etc. are not covered. The real magic is the /etc/ctl.conf file, which contains all of the information necessary for ctld to share disk drives on the network. Check out the man page for /etc/ctl.conf for more details; below is the configuration file that I created for this test setup. Note that on a system that has never had iSCSI configured, there will be no existing configuration file, so go ahead and create it. Then, enable ctld and start it: sysrc ctld_enable=”YES” service ctld start You can use the ctladm command to see what is going on: root@bsdtarget:/dev # ctladm lunlist (7:0:0/0): Fixed Direct Access SPC-4 SCSI device (7:0:1/1): Fixed Direct Access SPC-4 SCSI device root@bsdtarget:/dev # ctladm devlist LUN Backend Size (Blocks) BS Serial Number Device ID 0 block 10485760 512 MYSERIAL 0 MYDEVID 0 1 block 10485760 512 MYSERIAL 1 MYDEVID 1 Now, let's configure the client side: In order for a FreeBSD host to become an iSCSI Initiator, the iscsd daemon needs to be started. sysrc iscsid_enable=”YES” service iscsid start Next, the iSCSI Initiator can manually connect to the iSCSI target using the iscsictl tool. While setting up a new iSCSI session, this is probably the best option. Once you are sure the configuration is correct, add the configuration to the /etc/iscsi.conf file (see man page for this file). For iscsictl, pass the IP address of the target as well as the iSCSI IQN for the session: + iscsictl -A -p 192.168.22.128 -t iqn.2017-02.lab.testing:basictarget You should now have a new device (check dmesg), in this case, da1 The guide them walks through partitioning the disk, and laying down a UFS file system, and mounting it This it walks through how to disconnect iscsi, incase you don't want it anymore This all looked nice and easy, and it works very well. Now lets see what happens when you try to mount the iSCSI from Windows Ok, that wasn't so bad. Now, instead of sharing an entire space disk on the host via iSCSI, share a zvol. Now your windows machine can be backed by ZFS. All of your problems are solved. Interview - Philipp Buehler - pbuehler@sysfive.com (mailto:pbuehler@sysfive.com) Technical Lead at SysFive, and Former OpenBSD Committer News Roundup Half a dozen new features in mandoc -T html (http://undeadly.org/cgi?action=article&sid=20170316080827) mandoc (http://man.openbsd.org/mandoc.1)'s HTML output mode got some new features Even though mdoc(7) is a semantic markup language, traditionally none of the semantic annotations were communicated to the reader. [...] Now, at least in -T html output mode, you can see the semantic function of marked-up words by hovering your mouse over them. In terminal output modes, we have the ctags(1)-like internal search facility built around the less(1) tag jump (:t) feature for quite some time now. We now have a similar feature in -T html output mode. To jump to (almost) the same places in the text, go to the address bar of the browser, type a hash mark ('#') after the URI, then the name of the option, command, variable, error code etc. you want to jump to, and hit enter. Check out the full report by Ingo Schwarze (schwarze@) and try out these new features *** Optimizing IllumOS Kernel Crypto (http://zfs-create.blogspot.com/2014/05/optimizing-illumos-kernel-crypto.html) Sašo Kiselkov, of ZFS fame, looked into the performance of the OpenSolaris kernel crypto framework and found it lacking. The article also spends a few minutes on the different modes and how they work. Recently I've had some motivation to look into the KCF on Illumos and discovered that, unbeknownst to me, we already had an AES-NI implementation that was automatically enabled when running on Intel and AMD CPUs with AES-NI support. This work was done back in 2010 by Dan Anderson.This was great news, so I set out to test the performance in Illumos in a VM on my Mac with a Core i5 3210M (2.5GHz normal, 3.1GHz turbo). The initial tests of “what the hardware can do” were done in OpenSSL So now comes the test for the KCF. I wrote a quick'n'dirty crypto test module that just performed a bunch of encryption operations and timed the results. KCF got around 100 MB/s for each algorithm, except half that for AES-GCM OpenSSL had done over 3000 MB/s for CTR mode, 500 MB/s for CBC, and 1000 MB/s for GCM What the hell is that?! This is just plain unacceptable. Obviously we must have hit some nasty performance snag somewhere, because this is comical. And sure enough, we did. When looking around in the AES-NI implementation I came across this bit in aes_intel.s that performed the CLTS instruction. This is a problem: 3.1.2 Instructions That Cause VM Exits ConditionallyCLTS. The CLTS instruction causes a VM exit if the bits in position 3 (corresponding to CR0.TS) are set in both the CR0 guest/host mask and the CR0 read shadow. The CLTS instruction signals to the CPU that we're about to use FPU registers (which is needed for AES-NI), which in VMware causes an exit into the hypervisor. And we've been doing it for every single AES block! Needless to say, performing the equivalent of a very expensive context switch every 16 bytes is going to hurt encryption performance a bit. The reason why the kernel is issuing CLTS is because for performance reasons, the kernel doesn't save and restore FPU register state on kernel thread context switches. So whenever we need to use FPU registers inside the kernel, we must disable kernel thread preemption via a call to kpreemptdisable() and kpreemptenable() and save and restore FPU register state manually. During this time, we cannot be descheduled (because if we were, some other thread might clobber our FPU registers), so if a thread does this for too long, it can lead to unexpected latency bubbles The solution was to restructure the AES and KCF block crypto implementations in such a way that we execute encryption in meaningfully small chunks. I opted for 32k bytes, for reasons which I'll explain below. Unfortunately, doing this restructuring work was a bit more complicated than one would imagine, since in the KCF the implementation of the AES encryption algorithm and the block cipher modes is separated into two separate modules that interact through an internal API, which wasn't really conducive to high performance (we'll get to that later). Anyway, having fixed the issue here and running the code at near native speed, this is what I get: AES-128/CTR: 439 MB/s AES-128/CBC: 483 MB/s AES-128/GCM: 252 MB/s Not disastrous anymore, but still, very, very bad. Of course, you've got keep in mind, the thing we're comparing it to, OpenSSL, is no slouch. It's got hand-written highly optimized inline assembly implementations of most of these encryption functions and their specific modes, for lots of platforms. That's a ton of code to maintain and optimize, but I'll be damned if I let this kind of performance gap persist. Fixing this, however, is not so trivial anymore. It pertains to how the KCF's block cipher mode API interacts with the cipher algorithms. It is beautifully designed and implemented in a fashion that creates minimum code duplication, but this also means that it's inherently inefficient. ECB, CBC and CTR gained the ability to pass an algorithm-specific "fastpath" implementation of the block cipher mode, because these functions benefit greatly from pipelining multiple cipher calls into a single place. ECB, CTR and CBC decryption benefit enormously from being able to exploit the wide XMM register file on Intel to perform encryption/decryption operations on 8 blocks at the same time in a non-interlocking manner. The performance gains here are on the order of 5-8x.CBC encryption benefits from not having to copy the previously encrypted ciphertext blocks into memory and back into registers to XOR them with the subsequent plaintext blocks, though here the gains are more modest, around 1.3-1.5x. After all of this work, this is how the results now look on Illumos, even inside of a VM: Algorithm/Mode 128k ops AES-128/CTR: 3121 MB/s AES-128/CBC: 691 MB/s AES-128/GCM: 1053 MB/s So the CTR and GCM speeds have actually caught up to OpenSSL, and CBC is actually faster than OpenSSL. On the decryption side of things, CBC decryption also jumped from 627 MB/s to 3011 MB/s. Seeing these performance numbers, you can see why I chose 32k for the operation size in between kernel preemption barriers. Even on the slowest hardware with AES-NI, we can expect at least 300-400 MB/s/core of throughput, so even in the worst case, we'll be hogging the CPU for at most ~0.1ms per run. Overall, we're even a little bit faster than OpenSSL in some tests, though that's probably down to us encrypting 128k blocks vs 8k in the "openssl speed" utility. Anyway, having fixed this monstrous atrocity of a performance bug, I can now finally get some sleep. To made these tests repeatable, and to ensure that the changes didn't break the crypto algorithms, Saso created a crypto_test kernel module. I have recently created a FreeBSD version of crypto_test.ko, for much the same purposes Initial performance on FreeBSD is not as bad, if you have the aesni.ko module loaded, but it is not up to speed with OpenSSL. You cannot directly compare to the benchmarks Saso did, because the CPUs are vastly different. Performance results (https://wiki.freebsd.org/OpenCryptoPerformance) I hope to do some more tests on a range of different sized CPUs in order to determine how the algorithms scale across different clock speeds. I also want to look at, or get help and have someone else look at, implementing some of the same optimizations that Saso did. It currently seems like there isn't a way to perform addition crypto operations in the same session without regenerating the key table. Processing additional buffers in an existing session might offer a number of optimizations for bulk operations, although in many cases, each block is encrypted with a different key and/or IV, so it might not be very useful. *** Brendan Gregg's special freeware tools for sysadmins (http://www.brendangregg.com/specials.html) These tools need to be in every (not so) serious sysadmins toolbox. Triple ROT13 encryption algorithm (beware: export restrictions may apply) /usr/bin/maybe, in case true and false don't provide too little choice... The bottom command lists you all the processes using the least CPU cycles. Check out the rest of the tools. You wrote similar tools and want us to cover them in the show? Send us an email to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv) *** A look at 2038 (http://www.lieberbiber.de/2017/03/14/a-look-at-the-year-20362038-problems-and-time-proofness-in-various-systems/) I remember the Y2K problem quite vividly. The world was going crazy for years, paying insane amounts of money to experts to fix critical legacy systems, and there was a neverending stream of predictions from the media on how it's all going to fail. Most didn't even understand what the problem was, and I remember one magazine writing something like the following: Most systems store the current year as a two-digit value to save space. When the value rolls over on New Year's Eve 1999, those two digits will be “00”, and “00” means “halt operation” in the machine language of many central processing units. If you're in an elevator at this time, it will stop working and you may fall to your death. I still don't know why they thought a computer would suddenly interpret data as code, but people believed them. We could see a nearby hydropower plant from my parents' house, and we expected it to go up in flames as soon as the clock passed midnight, while at least two airplanes crashed in our garden at the same time. Then nothing happened. I think one of the most “severe” problems was the police not being able to open their car garages the next day because their RFID tokens had both a start and end date for validity, and the system clock had actually rolled over to 1900, so the tokens were “not yet valid”. That was 17 years ago. One of the reasons why Y2K wasn't as bad as it could have been is that many systems had never used the “two-digit-year” representation internally, but use some form of “timestamp” relative to a fixed date (the “epoch”). The actual problem with time and dates rolling over is that systems calculate timestamp differences all day. Since a timestamp derived from the system clock seemingly only increases with each query, it is very common to just calculate diff = now - before and never care about the fact that now could suddenly be lower than before because the system clock has rolled over. In this case diff is suddenly negative, and if other parts of the code make further use of the suddenly negative value, things can go horribly wrong. A good example was a bug in the generator control units (GCUs) aboard Boeing 787 “Dreamliner” aircrafts, discovered in 2015. An internal timestamp counter would overflow roughly 248 days after the system had been powered on, triggering a shut down to “safe mode”. The aircraft has four generator units, but if all were powered up at the same time, they would all fail at the same time. This sounds like an overflow caused by a signed 32-bit counter counting the number of centiseconds since boot, overflowing after 248.55 days, and luckily no airline had been using their Boing 787 models for such a long time between maintenance intervals. The “obvious” solution is to simply switch to 64-Bit values and call it day, which would push overflow dates far into the future (as long as you don't do it like the IBM S/370 mentioned before). But as we've learned from the Y2K problem, you have to assume that computer systems, computer software and stored data (which often contains timestamps in some form) will stay with us for much longer than we might think. The years 2036 and 2038 might be far in the future, but we have to assume that many of the things we make and sell today are going to be used and supported for more than just 19 years. Also many systems have to store dates which are far in the future. A 30 year mortgage taken out in 2008 could have already triggered the bug, and for some banks it supposedly did. sysgettimeofday() is one of the most used system calls on a generic Linux system and returns the current time in form of an UNIX timestamp (timet data type) plus fraction (susecondst data type). Many applications have to know the current time and date to do things, e.g. displaying it, using it in game timing loops, invalidating caches after their lifetime ends, perform an action after a specific moment has passed, etc. In a 32-Bit UNIX system, timet is usually defined as a signed 32-Bit Integer. When kernel, libraries and applications are compiled, the compiler will turn this assumption machine code and all components later have to match each other. So a 32-Bit Linux application or library still expects the kernel to return a 32-Bit value even if the kernel is running on a 64-Bit architecture and has 32-Bit compatibility. The same holds true for applications calling into libraries. This is a major problem, because there will be a lot of legacy software running in 2038. Systems which used an unsigned 32-Bit Integer for timet push the problem back to 2106, but I don't know about many of those. The developers of the GNU C library (glibc), the default standard C library for many GNU/Linux systems, have come up with a design for year 2038 proofness for their library. Besides the timet data type itself, a number of other data structures have fields based on timet or the combined struct timespec and struct timeval types. Many methods beside those intended for setting and querying the current time use timestamps 32-Bit Windows applications, or Windows applications defining _USE32BITTIMET, can be hit by the year 2038 problem too if they use the timet data type. The _time64t data type had been available since Visual C 7.1, but only Visual C 8 (default with Visual Studio 2015) expanded timet to 64 bits by default. The change will only be effective after a recompilation, legacy applications will continue to be affected. If you live in a 64-Bit world and use a 64-Bit kernel with 64-Bit only applications, you might think you can just ignore the problem. In such a constellation all instances of the standard time_t data type for system calls, libraries and applications are signed 64-Bit Integers which will overflow in around 292 billion years. But many data formats, file systems and network protocols still specify 32-Bit time fields, and you might have to read/write this data or talk to legacy systems after 2038. So solving the problem on your side alone is not enough. Then the article goes on to describe how all of this will break your file systems. Not to mention your databases and other file formats. Also see Theo De Raadt's EuroBSDCon 2013 Presentation (https://www.openbsd.org/papers/eurobsdcon_2013_time_t/mgp00001.html) *** Beastie Bits Michael Lucas: Get your name in “Absolute FreeBSD 3rd Edition” (https://blather.michaelwlucas.com/archives/2895) ZFS compressed ARC stats to top (https://svnweb.freebsd.org/base?view=revision&revision=r315435) Matthew Dillon discovered HAMMER was repeating itself when writing to disk. Fixing that issue doubled write speeds (https://www.dragonflydigest.com/2017/03/14/19452.html) TedU on Meaningful Short Names (http://www.tedunangst.com/flak/post/shrt-nms-fr-clrty) vBSDcon and EuroBSDcon Call for Papers are open (https://www.freebsdfoundation.org/blog/submit-your-work-vbsdcon-and-eurobsdcon-cfps-now-open/) Feedback/Questions Craig asks about BSD server management (http://pastebin.com/NMshpZ7n) Michael asks about jails as a router between networks (http://pastebin.com/UqRwMcRk) Todd asks about connecting jails (http://pastebin.com/i1ZD6eXN) Dave writes in with an interesting link (http://pastebin.com/QzW5c9wV) > applications crash more often due to errors than corruptions. In the case of corruption, a few applications (e.g., Log-Cabin, ZooKeeper) can use checksums and redundancy to recover, leading to a correct behavior; however, when the corruption is transformed into an error, these applications crash, resulting in reduced availability. ***

This week on BSDNow, we have an interview with Richard Yao, who will be telling us about the experience and challenges of porting ZFS to Linux. That plus the latest news and feedback is coming your way, on your place This episode was brought to you by Headlines Registration for MeetBSD 2016 is now Open (https://www.meetbsd.com/) “Beastie's coming home!” This year, MeetBSD will be held at UC Berkeley's Clark Kerr Campus November 11th and 12th, preceded by a two day FreeBSD Vendor/Dev Summit (Nov 9th and 10th) MeetBSD can be traced back to its humble roots as a local workshop for BSD developers and users, hosted annually in Poland since 2004. Since then, MeetBSD's popularity has spread, and it's now widely recognized as its own conference with participants from all over the world. The US version runs every two years in California since 2008, and now trades off with the east coast vBSDCon which runs on the odd years. “MeetBSD 2016 uses a mixed unConference format featuring both scheduled talks and community-driven events such as birds-of-a-feather meetings, lightning talks, hackable presentations, stump the chumps, and speed geeking sessions. Speakers are to be determined – stay tuned for more information!” Register before September 30th, and get $30 off Kris and I will be there, along with lots of other FreeBSD Developers, Vendors, and Users. MeetBSD's unconference style does a very good job of mingling users with developers and is one of my favourite conferences. *** Dual Booting FreeBSD and Windows UEFI (http://kev009.com/wp/2016/07/freebsd-uefi-root-on-zfs-and-windows-dual-boot/) Looking to install FreeBSD alongside Windows 10? What happens if that that system is pre-installed and UEFI? Well you could run TrueOS, but if that isn't your bag and you want vanilla FreeBSD we have you covered this week! Over on Kevin Bowling's blog, we have a detailed article showing exactly how to do that. First up, as prep you'll need to go into the Windows disk manager and shrink your existing NTFS partition. You'll need to next boot FreeBSD 11 or later. From there the walkthrough takes us through disk partitioning using gpart, and setup of ZFS into a boot-environment friendly layout. Once you get through the typical FreeBSD setup / extraction, the tutorial gives us a nice bonus, showing how to setup “rEFInd” for a graphical boot-menu. A great walkthrough, and hopefully it encourages others to try out dual-booting “EFI-style”. *** ZFS High-Availability NAS (https://github.com/ewwhite/zfs-ha/wiki) Interested in a DiY HA ZFS NAS? Edmund White (ewwhite on github) has posted a very detailed look at how he has custom-rolled his own Linux + ZFS + HA setup. Most of the concepts are already ones used in various other HA products, but it is interesting and informative to see a public detailed look at how ZFS and HA works. In particular this setup require some very specific hardware, such as dual-port SAS drives, so you will have to pre-plan according. The only bummer is this is a ZFS on Linux setup. Maybe this can serve as the guide / inspiration for somebody in our community to do their own FreeBSD + HA + ZFS setup and blog about it in similar detail. *** First public release of chyves - version 0.1.0 (http://chyves.org/) As bhyve continues to mature we are seeing tooling evolve around it. Enter ‘chyves' which started life as a fork of iohyve. We are looking to do an interview with the author in the near future, but we still want to bring you some of the new features / changes in this evolution of bhyve management. First up, nearly every function from iohyve has either been re-written in part or full. Among the new features, a full logging system (master and per-vm logs), multiple pool configurations, properties stored outside of ZFS (for speed) and self-upgrading. (Will that work with pkg'd version?) In addition to the above features, the website has a large chart showing the original ‘iohyve' commands, and how that usage has changed moving to chyves. Give it a spin, let the author know of issues! *** Interview - Richard Yao - ryao@gentoo.org (mailto:ryao@gentoo.org) Sr. Kernel Engineer at ClusterHQ - Major Contributor to ZFS on Linux News Roundup ZFS Deadlock: 'Directory of Death' (http://lists.freebsd.org/pipermail/freebsd-hackers/2016-July/049740.html) A user reports that when they try to install npm (the Node.js package manager), their system deadlocks It turns out, this was also hitting the FreeBSD package building machines PR 209158 (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209158) The problem was a race condition in the way renames are handled in the FreeBSD VFS vs how ZFS does them internally This bug has existed since the original import of ZFS, but some other change caused it to happen much more frequently “ZFS POSIX Layer is originally written for Solaris VFS which is very different from FreeBSD VFS. Most importantly many things that FreeBSD VFS manages on behalf of all filesystems are implemented in ZPL in a different Way. Thus, ZPL contains code that is redundant on FreeBSD or duplicates VFS functionality or, in the worst cases, badly interacts / interferes with VFS.” “The most prominent problem is a deadlock caused by the lock order reversal of vnode locks that may happen with concurrent zfsrename() and lookup(). The deadlock is a result of zfsrename() not observing the vnode locking contract expected by VFS.” The fixes have been merged to the 10.x and 11.x branches *** New BSD Magazine out (2016-07) (https://bsdmag.org/download/implementing-memory-cache-beast-architecture/) Articles include: Implementing in-memory cache in the BeaST architecture, Docker Cleanup, FreeNAS Getting Started Guide, and starting at the very beginning with open source The August issue is also out (https://bsdmag.org/download/minix-3-free-open-source-operating-system-highly-reliable-flexible-secure/) This issue features two articles about MINIX 3, continues the FreeNAS getting started guide, Optimizes the in-memory cache for the BeaST architecture, and talks about fixing failed ports for Hardened and LibreBSD We hope to have an interview with the creator of the BeaST architecture in the coming weeks *** DragonflyBSD and UEFI (http://lists.dragonflybsd.org/pipermail/users/2016-July/270796.html) We've featured a few stories and walkthroughs about using UEFI to dual-boot BSD, and now its Dragonfly BSD's turn. Dave McFarlane writes into the DF mailing lists, telling us about the specific steps taken to get UEFI installed and boot-strapped on his system. If you've done a FreeBSD manual UEFI install, the process looks very similar, but you will end up manually running ‘gpt' to create partitions, installing dist files, and eventually installing boot1.efi into the FAT EFI partition. Dave also ran into an issue with resulted in no /etc/fstab being present, and helpfully includes what his system needed to fully boot hammer properly. Somebody should document this fully for DFLY, since I would expect to become more commonplace as commodity hardware is shipped with UEFI on by default. *** Netflix and Fill (http://techblog.netflix.com/2016/08/netflix-and-fill.html) The Netflix team has produced a technical blog post describing how their OpenConnect appliances work First the content is received from the content provider, and the Netflix content team makes it ready for deployment, by transcoding the various bitrates, packaging the subtitles, etc. The finished files are then pushed to Amazon S3 storage “We deploy the majority of our updates proactively during configured fill windows. An important difference between our OpenConnect CDN and other commercial CDNs is the concept of proactive caching. Because we can predict with high accuracy what our members will watch and what time of day they will watch it, we can make use of non-peak bandwidth to download most of the content updates to the OCAs in our network during these configurable time windows. By reducing disk reads (content serving) while we are performing disk writes (adding new content to the OCAs), we are able to optimize our disk efficiency by avoiding read/write contention. The predictability of off-peak traffic patterns helps with this optimization, but we still only have a finite amount of time every day to get our content pre-positioned to where it needs to be before our traffic starts to ramp up and we want to make all of the OCA capacity available for content serving.” The OCA may actually contain more than one copy of the same video, because each disk in the OCA is independent, storing the same video on two different disks will provide twice the available read bandwidth Normally the filesystem cache would obviate the need for this, but the Netflix OCA has so much storage, and not a lot of memory, and the requests from users are offset enough that the cache is useless “OCAs communicate at regular intervals with the control plane services, requesting (among other things) a manifest file that contains the list of titles they should be storing and serving to members. If there is a delta between the list of titles in the manifest and what they are currently storing, each OCA will send a request, during its configured fill window, that includes a list of the new or updated titles that it needs. The response from the control plane in AWS is a ranked list of potential download locations, aka fill sources, for each title.” “It would be inefficient, in terms of both time and cost, to distribute a title directly from S3 to all of our OCAs, so we use a tiered approach. The goal is to ensure that the title is passed from one part of our network to another using the most efficient route possible.” The article then goes on to explain how they calculate the least cost filling source “Now that Netflix operates in 190 countries and we have thousands of appliances embedded within many ISP networks around the world, we are even more obsessed with making sure that our OCAs get the latest content as quickly as possible while continuing to minimize bandwidth cost to our ISP partners.” *** Beastie Bits: Cover reveal for “PAM Mastery” (http://blather.michaelwlucas.com/archives/2734) LibertyBSD 5.9 is out - looking for mirrors (http://libertybsd.net/download.html) Unix for Poets (https://web.stanford.edu/class/cs124/lec/124-UnixForPoets.pdf) Feedback/Questions Chuck / Ingo - Get Involved (http://pastebin.com/ksq0rfph) Oskar - Thanks (http://pastebin.com/YqzcHEMg) Alex - SMF (http://pastebin.com/WvdVZbYc) Raymond - RPI3 (http://pastebin.com/JPWgzSGv) ***

A recording from vBSDCon 2015 of the talk titled "Supporting a BSD Project" with Ed Maste and George Neville-Neil.File Info: 65Min, 31MB.Ogg Link: https://archive.org/download/bsdtalk259/bsdtalk259.ogg

Oct 2015 is the 20th anniversary of the OpenBSD source tree!This episode is brought to you by the id utility, which returns the user identity.  id appeared in 4.4 BSD.An interview from vBSDCon 2015 with Chris Henschen from fP Technologies.  They recently ported their filePro Plus product to FreeBSD.File Info: 6Min, 3MBOgg Link: https://archive.org/download/bsdtalk258/bsdtalk258.ogg

An interview with NetBSD developer Christos Zoulas at vBSDCon 2015.File Info: 15Min, 7MB.Ogg Link: https://archive.org/download/bsdtalk257/bsdtalk257.ogg

An interview with Allan Jude at vBSDCon in Reston, Virginia.  We talk about the book FreeBSD Mastery: ZFS that Michael W. Lucas and he co-authored.  You can find more information about the book at www.zfsbook.comFile Info: 16Min, 8MB.Ogg Link: https://archive.org/download/bsdtalk256/bsdtalk256.ogg

A recording of Baptiste Daroussin speaking at vBSDCon in October 2013.  He is a FreeBSD source committer and project developer for PkgNG.  PkgNG is a package management tool for FreeBSD. It is the replacement for the current pkg_info/pkg_create/pkg_add tools.File Info: 55Min, 26MB.Ogg Link: https://archive.org/download/bsdtalk239/bsdtalk239.ogg

Interview with Verisign CTO Burt Kaliski about vBSDcon.More information at http://www.vbsdcon.com/File Info: 12Min, 6MB.Ogg Link: https://archive.org/download/bsdtalk230/bsdtalk230.ogg

Recording of the vBSDCon 2013 talk "Migrating from GCC to LLVM/CLANG" with David Chisnall.File Info: 1hour, 31MB.Ogg Link: https://archive.org/download/bsdtalk233/bsdtalk233.ogg

An interview from vBSDCon with Henning Brauer.  We talk about his recent work with the pf firewall and the queuing system.File Info: 30Min, 14MB.Ogg Link: https://archive.org/download/bsdtalk234/bsdtalk234.ogg

This episode has been brought to you by the id utility, which returns a user identity.  The id utility first appeared in 4.4 BSD.Interview with Allan Jude at vBSDCon.  We talk about ScaleEngine, Jupiter Broadcasting, Tech Snap, and the BSD Now podcast.  Catch new episodes of these and many other fine podcasts at www.jupiterbroadcasting.com.File Info: 26Min, 13Mb.Ogg Link: https://archive.org/download/bsdtalk235/bsdtalk235.ogg

A talk from vBSDCon in 2013 titled Verisign and FreeBSD: Internet Scale Services at 10 Gigabits per Server presented by Mike Bentkofsky, Marc de la Gueronniere, Julien CharbonFile info: 47Min, 22MBOgg link: https://archive.org/download/bsdtalk251/bsdtalk251.ogg

My apologies for not having any recent interviews. More to come when I attend vBSDCon.  School firewalls switched over to PFSense.  Urging calm discussions around NextBSD.File Info: 6Min, 3MB.Ogg Link: https://archive.org/download/bsdtalk255/bsdtalk255.ogg

It's already our two-year anniversary! This time on the show, we'll be chatting with Scott Courtney, vice president of infrastructure engineering at Verisign, about this year's vBSDCon. What's it have to offer in an already-crowded BSD conference space? We'll find out. This episode was brought to you by Headlines OpenBSD hypervisor coming soon (https://www.marc.info/?l=openbsd-tech&m=144104398132541&w=2) Our buddy Mike Larkin never rests, and he posted some very tight-lipped console output (http://pastebin.com/raw.php?i=F2Qbgdde) on Twitter recently From what little he revealed at the time (https://twitter.com/mlarkin2012/status/638265767864070144), it appeared to be a new hypervisor (https://en.wikipedia.org/wiki/Hypervisor) (that is, X86 hardware virtualization) running on OpenBSD -current, tentatively titled "vmm" Later on, he provided a much longer explanation on the mailing list, detailing a bit about what the overall plan for the code is Originally started around the time of the Australia hackathon, the work has since picked up more steam, and has gotten a funding boost from the OpenBSD foundation One thing to note: this isn't just a port of something like Xen or Bhyve; it's all-new code, and Mike explains why he chose to go that route He also answered some basic questions about the requirements, when it'll be available, what OSes it can run, what's left to do, how to get involved and so on *** Why FreeBSD should not adopt launchd (http://blog.darknedgy.net/technology/2015/08/26/0/) Last week (http://www.bsdnow.tv/episodes/2015_08_26-beverly_hills_25519) we mentioned a talk Jordan Hubbard gave about integrating various parts of Mac OS X into FreeBSD One of the changes, perhaps the most controversial item on the list, was the adoption of launchd to replace the init system (replacing init systems seems to cause backlash, we've learned) In this article, the author talks about why he thinks this is a bad idea He doesn't oppose the integration into FreeBSD-derived projects, like FreeNAS and PC-BSD, only vanilla FreeBSD itself - this is also explained in more detail The post includes both high-level descriptions and low-level technical details, and provides an interesting outlook on the situation and possibilities Reddit had quite a bit (https://www.reddit.com/r/BSD/comments/3ilhpk) to say (https://www.reddit.com/r/freebsd/comments/3ilj4i) about this one, some in agreement and some not *** DragonFly graphics improvements (http://lists.dragonflybsd.org/pipermail/commits/2015-August/458108.html) The DragonFlyBSD guys are at it again, merging newer support and fixes into their i915 (Intel) graphics stack This latest update brings them in sync with Linux 3.17, and includes Haswell fixes, DisplayPort fixes, improvements for Broadwell and even Cherryview GPUs You should also see some power management improvements, longer battery life and various other bug fixes If you're running DragonFly, especially on a laptop, you'll want to get this stuff on your machine quick - big improvements all around *** OpenBSD tames the userland (https://www.marc.info/?l=openbsd-tech&m=144070638327053&w=2) Last week we mentioned OpenBSD's tame framework getting support for file whitelists, and said that the userland integration was next - well, now here we are Theo posted a mega diff of nearly 100 smaller diffs, adding tame support to many areas of the userland tools It's still a work-in-progress version; there's still more to be added (including the file path whitelist stuff) Some classic utilities are even being reworked to make taming them easier - the "w" command (https://www.marc.info/?l=openbsd-cvs&m=144103945031253&w=2), for example The diff provides some good insight on exactly how to restrict different types of utilities, as well as how easy it is to actually do so (and en masse) More discussion can be found on HN (https://news.ycombinator.com/item?id=10135901), as one might expect If you're a software developer, and especially if your software is in ports already, consider adding some more fine-grained tame support in your next release *** Interview - Scott Courtney - vbsdcon@verisign.com (mailto:vbsdcon@verisign.com) / @verisign (https://twitter.com/verisign) vBSDCon (http://vbsdcon.com/) 2015 News Roundup OPNsense, beyond the fork (https://opnsense.org/opnsense-beyond-the-fork) We first heard about (http://www.bsdnow.tv/episodes/2015_01_14-common_sense_approach) OPNsense back in January, and they've since released nearly 40 versions, spanning over 5,000 commits This is their first big status update, covering some of the things that've happened since the project was born There's been a lot of community growth and participation, mass bug fixing, new features added, experimental builds with ASLR and much more - the report touches on a little of everything *** LibreSSL nukes SSLv3 (http://undeadly.org/cgi?action=article&sid=20150827112006) With their latest release, LibreSSL began to turn off SSLv3 (http://disablessl3.com) support, starting with the "openssl" command At the time, SSLv3 wasn't disabled entirely because of some things in the OpenBSD ports tree requiring it (apache being one odd example) They've now flipped the switch, and the process of complete removal has started From the Undeadly summary, "This is an important step for the security of the LibreSSL library and, by extension, the ports tree. It does, however, require lots of testing of the resulting packages, as some of the fallout may be at runtime (so not detected during the build). That is part of why this is committed at this point during the release cycle: it gives the community more time to test packages and report issues so that these can be fixed. When these fixes are then pushed upstream, the entire software ecosystem will benefit. In short: you know what to do!" With this change and a few more to follow shortly, LibreSSL won't actually support SSL anymore - time to rename it "LibreTLS" *** FreeBSD MPTCP updated (http://caia.swin.edu.au/urp/newtcp/mptcp/tools/v05/mptcp-readme-v0.5.txt) For anyone unaware, Multipath TCP (https://en.wikipedia.org/wiki/Multipath_TCP) is "an ongoing effort of the Internet Engineering Task Force's (IETF) Multipath TCP working group, that aims at allowing a Transmission Control Protocol (TCP) connection to use multiple paths to maximize resource usage and increase redundancy." There's been work out of an Australian university to add support for it to the FreeBSD kernel, and the patchset was recently updated Including in this latest version is an overview of the protocol, how to get it compiled in, current features and limitations and some info about the routing requirements Some big performance gains can be had with MPTCP, but only if both the client and server systems support it - getting it into the FreeBSD kernel would be a good start *** UEFI and GPT in OpenBSD (https://www.marc.info/?l=openbsd-cvs&m=144092912907778&w=2) There hasn't been much fanfare about it yet, but some initial UEFI and GPT-related commits have been creeping into OpenBSD recently Some support (https://github.com/yasuoka/openbsd-uefi) for UEFI booting has landed in the kernel, and more bits are being slowly enabled after review This comes along with a number (https://www.marc.info/?l=openbsd-cvs&m=143732984925140&w=2) of (https://www.marc.info/?l=openbsd-cvs&m=144088136200753&w=2) other (https://www.marc.info/?l=openbsd-cvs&m=144046793225230&w=2) commits (https://www.marc.info/?l=openbsd-cvs&m=144045760723039&w=2) related to GPT, much of which is being refactored and slowly reintroduced Currently, you have to do some disklabel wizardry to bypass the MBR limit and access more than 2TB of space on a single drive, but it should "just work" with GPT (once everything's in) The UEFI bootloader support has been committed (https://www.marc.info/?l=openbsd-cvs&m=144115942223734&w=2), so stay tuned for more updates (http://undeadly.org/cgi?action=article&sid=20150902074526&mode=flat) as further (https://twitter.com/kotatsu_mi/status/638909417761562624) progress (https://twitter.com/yojiro/status/638189353601097728) is made *** Feedback/Questions John writes in (http://slexy.org/view/s2sIWfb3Qh) Mason writes in (http://slexy.org/view/s2Ybrx00KI) Earl writes in (http://slexy.org/view/s20FpmR7ZW) ***

This week on the show, we'll be talking to Mike Larkin about various memory protections in OpenBSD. We'll cover recent W^X improvements, SSP, ASLR, PIE and all kinds of acronyms! We've also got a bunch of news and answers to your questions, coming up on BSD Now - the place to B.. SD. This episode was brought to you by Headlines OpenSMTPD for the whole family (http://homing-on-code.blogspot.com/2015/05/accept-from-any-for-any-relay-via.html) Setting up a BSD mail server is something a lot of us are probably familiar with doing, at least for our own accounts This article talks about configuring a home mail server too, but even for the other people you live with After convincing his wife to use their BSD-based Owncloud server for backups, the author talks about moving her over to his brand new OpenSMTPD server too If you've ever run a mail server and had to deal with greylisting, you'll appreciate the struggle he went through In the end, BGP-based list distribution saved the day, and his family is being served well by a BSD box *** NetBSD on the Edgerouter Lite (https://blog.netbsd.org/tnf/entry/hands_on_experience_with_edgerouter) We've talked a lot about building your own BSD-based router on the show, but not many of the devices we mention are in the same price range as consumer devices The EdgeRouter Lite, a small MIPS-powered machine, is starting to become popular (and is a bit cheaper) A NetBSD developer has been hacking on it, and documents the steps to get a working install in this blog post The process is fairly simple, and you can cross-compile (http://www.bsdnow.tv/tutorials/current-nbsd) your own installation image on any CPU architecture (even from another BSD!) OpenBSD and FreeBSD also have some (http://www.openbsd.org/octeon.html) support (http://rtfm.net/FreeBSD/ERL/) for these devices *** Bitrig at NYC*BUG (https://www.youtube.com/watch?v=h4FhgBdYSUU) The New York City BSD users group has semi-regular meetings with presentations, and this time the speaker was John Vernaleo John discussed Bitrig (http://www.bsdnow.tv/episodes/2014_12_10-must_be_rigged), an OpenBSD fork that we've talked about a couple times on the show He talks about what they've been up to lately, why they're doing what they're doing, difference in supported platforms Ports and packages between the two projects are almost exactly the same, but he covers the differences in the base systems, how (some) patches get shared between the two and finally some development model differences *** OPNsense, meet HardenedBSD (https://hardenedbsd.org/article/shawn-webb/2015-05-08/hardenedbsd-teams-opnsense) Speaking of forks, two FreeBSD-based forked projects we've mentioned on the show, HardenedBSD (http://www.bsdnow.tv/episodes/2014_08_27-reverse_takeover) and OPNsense (http://www.bsdnow.tv/episodes/2015_01_14-common_sense_approach), have decided to join forces Backporting their changes to the 10-STABLE branch, HardenedBSD hopes to introduce some of their security additions to the OPNsense codebase Paired up with LibreSSL, this combination should offer a good solution for anyone wanting a BSD-based firewall with an easy web interface We'll cover more news on the collaboration as it comes out *** Interview - Mike Larkin - mlarkin@openbsd.org (mailto:mlarkin@openbsd.org) / @mlarkin2012 (https://twitter.com/mlarkin2012) Memory protections in OpenBSD: W^X (https://en.wikipedia.org/wiki/W%5EX), ASLR (https://en.wikipedia.org/wiki/Address_space_layout_randomization), PIE (https://en.wikipedia.org/wiki/Position-independent_code), SSP (https://en.wikipedia.org/wiki/Buffer_overflow_protection) News Roundup A closer look at FreeBSD (http://www.techopedia.com/2/31035/software/a-closer-look-at-freebsd) The week wouldn't be complete without at least one BSD article making it to a mainstream tech site This time, it's a high-level overview of FreeBSD, some of its features and where it's used Being that it's an overview article on a more mainstream site, you won't find anything too technical - it covers some BSD history, stability, ZFS, LLVM and Clang, ports and packages, jails and the licensing If you have any BSD-curious Linux friends, this might be a good one to send to them *** Linksys NSLU2 and NetBSD (http://ramblingfoo.blogspot.com/2015/05/linksys-nslu2-adventures-into-netbsd.html) The Linksys NSLU2 is a proprietary network-attached storage device introduced back in 2004 "About 2 months ago I set a goal to run some kind of BSD on the spare Linksys NSLU2 I had. This was driven mostly by curiosity, after listening to a few BSDNow episodes and becoming a regular listener [...]" After doing some research, the author of this post discovered that he could cross-compile NetBSD for the device straight from his Linux box If you've got one of these old devices kicking around, check out this write-up and get some BSD action on there *** OpenBSD disklabel templates (http://blog.jeffreyforman.net/2015/05/09/from-0-to-an-openbsd-install-with-no-hands-and-a-custom-disk-layou) We've covered OpenBSD's "autoinstall" feature for unattended installations in the past, but one area where it didn't offer a lot of customization was with the disk layout With a few recent changes (http://undeadly.org/cgi?action=article&sid=20150505123418), there are now a series of templates you can use for a completely customized partition scheme This article takes you through the process of configuring an autoinstall answer file and adding the new section for disklabel Combine this new feature with our -stable iso tutorial (http://www.bsdnow.tv/tutorials/stable-iso), and you could deploy completely patched and customized images en masse pretty easily *** FreeBSD native ARM builds (https://svnweb.freebsd.org/base?view=revision&revision=282693) FreeBSD -CURRENT builds for the ARM CPU architecture can now be built natively, without utilities that aren't part of base Some of the older board-specific kernel configuration files have been replaced, and now the "IMC6" target is used This goes along with what we read in the most recent quarterly status report - ARM is starting to get treated as a first class citizen *** Feedback/Questions Sean writes in (http://slexy.org/view/s2088U2OjO) Ron writes in (http://slexy.org/view/s29ZKhQKOz) Charles writes in (http://slexy.org/view/s2NCVHEKt1) Bostjan writes in (http://slexy.org/view/s2mGRoKo5G) ***

This week on the show, we'll be talking with Paul Schenkeveld, chairman of the EuroBSDCon foundation. He tells us about his experiences running BSD conferences and how regular users can get involved too. We've also got answers to all your emails and the latest news, coming up on BSD Now - the place to B.. SD. This episode was brought to you by Headlines More BSD presentation videos (https://www.meetbsd.com/) The MeetBSD video uploading spree continues with a few more talks, maybe this'll be the last batch Corey Vixie, Web Apps in Embedded BSD (https://www.youtube.com/watch?v=Pbks12Mqpp8) Allan Jude, UCL config (https://www.youtube.com/watch?v=TjP86iWsEzQ) Kip Macy, iflib (https://www.youtube.com/watch?v=P4FRPKj7F80) While we're on the topic of conferences, AsiaBSDCon's CFP was extended (https://twitter.com/asiabsdcon/status/538352055245492226) by one week This year's ruBSD (https://events.yandex.ru/events/yagosti/rubsd14/) will be on December 13th in Moscow Also, the BSDCan call for papers (http://lists.bsdcan.org/pipermail/bsdcan-announce/2014-December/000135.html) is out, and the event will be in June next year Lastly, according to Rick Miller, "A potential vBSDcon 2015 event is being explored though a decision has yet to be made." *** BSD-powered digital library in Africa (http://peercorpsglobal.org/nzegas-digital-library-becomes-a-reality/) You probably haven't heard much about Nzega, Tanzania, but it's an East African country without much internet access With physical schoolbooks being a rarity there, a few companies helped out to bring some BSD-powered reading material to a local school They now have a pair of FreeNAS Minis at the center of their local network, with over 80,000 books and accompanying video content stored on them (~5TB of data currently) The school's workstations also got wiped and reloaded with FreeBSD, and everyone there seems to really enjoy using it *** pfSense 2.2 status update (https://blog.pfsense.org/?p=1486) With lots of people asking when the 2.2 release will be done, some pfSense developers decided to provide a status update 2.2 will have a lot of changes: being based on FreeBSD 10.1, Unbound instead of BIND, updating PHP to something recent, including the new(ish) IPSEC stack updates, etc All these things have taken more time than previously expected The post also has some interesting graphs showing the ratio of opened and close bugs for the upcoming release *** Recommended hardware threads (https://www.reddit.com/r/BSD/comments/2n8wrg/bsd_on_mini_itx/) A few threads on caught our attention this week, all about hardware recommendations for BSD setups In the first one, the OP asks about mini-ITX hardware to run a FreeBSD server and NAS Everyone gave some good recommendations for low power, Atom-based systems The second thread (https://www.marc.info/?t=141694918800006&r=1&w=2) started off asking about which CPU architecture is best for PF on an OpenBSD router, but ended up being another hardware thread For a router, the ALIX, APU and Soekris boards still seem to be the most popular choices, with the third (https://www.reddit.com/r/homelab/comments/24m6tj/) and fourth (https://www.reddit.com/r/PFSENSE/comments/2nblgp/) threads confirming this If you're thinking about building your first BSD box - server, router, NAS, whatever - these might be some good links to read *** Interview - Paul Schenkeveld - freebsd@psconsult.nl (mailto:freebsd@psconsult.nl) Running a BSD conference News Roundup From Linux to FreeBSD - for reals (https://www.reddit.com/r/freebsd/comments/2nqa60/) Another Linux user is ready to switch to BSD, and takes to Reddit for some community encouragement (seems to be a common thing now) After being a Linux guy for 20(!) years, he's ready to switch his systems over, and is looking for some helpful guides to transition In the comments, a lot of new switchers offer some advice and reading material If any of the listeners have some things that were helpful along your switching journey, maybe send 'em this guy's way *** Running FreeBSD as a Xen Dom0 (http://wiki.xenproject.org/wiki/FreeBSD_Dom0) Continuing progress has been made to allow FreeBSD to be a host for the Xen hypervisor This wiki article explains how to run the Xen branch of FreeBSD and host virtual machines on it Xen on FreeBSD currently supports PV guests (modified kernels) and HVM (unmodified kernels, uses hardware virtualization features) The wiki provides instructions for running Debian (PV) and FreeBSD (HVM), and discusses the features that are not finished yet *** HardenedBSD updates and changes (http://hardenedbsd.org/article/shawn-webb/2014-11-18/aout-and-null-mapping-support-removal) a.out is the old executable format for Unix The name stands for assembler output, and was coined by Ken Thompson as the fixed name for output of his PDP-7 assembler in 1968 FreeBSD, on which HardenedBSD is based, switched away from a.out in version 3.0 A restriction against NULL mapping was introduced in FreeBSD 7 (https://www.freebsd.org/security/advisories/FreeBSD-EN-09:05.null.asc) and enabled by default in FreeBSD 8 However, for reasons of compatibility, it could be switched off, allowing buggy applications to continue to run, at the risk of allowing a kernel bug to be exploited HardenedBSD has removed the sysctl, making it impossible to run in ‘insecure mode' Package building update: more consistent repo, no more i386 packages (http://hardenedbsd.org/article/shawn-webb/2014-11-30/package-building-infrastructure-maintenance) *** Feedback/Questions Boris writes in (http://slexy.org/view/s2kVPKICqj) Alex writes in (http://slexy.org/view/s21Fic4dZC) (edit: adding "tinker panic 0" to the ntp.conf will disable the sanity check) Chris writes in (http://slexy.org/view/s2zk1Tvfe9) Robert writes in (http://slexy.org/view/s22alvJ4mu) Jake writes in (http://slexy.org/view/s203YMc2zL) *** Mailing List Gold Real world authpf use (https://www.marc.info/?t=141711266800001&r=1&w=2) The (https://svnweb.freebsd.org/ports/head/UPDATING?r1=373564&r2=373563&pathrev=373564) great (https://lists.freebsd.org/pipermail/freebsd-ports/2014-November/096788.html) perl (https://lists.freebsd.org/pipermail/freebsd-ports/2014-November/096799.html) event (https://lists.freebsd.org/pipermail/freebsd-perl/2014-November/010146.html) of (https://lists.freebsd.org/pipermail/freebsd-perl/2014-November/010149.html) 2014 (https://lists.freebsd.org/pipermail/freebsd-perl/2014-November/010167.html) ***

This week on the show, we sit down for an interview with Jordan Hubbard, one of the founders of the FreeBSD project - and the one who invented ports! Later in the show, we'll be showing you some new updates to the OpenBSD router tutorial from a couple weeks ago. We've also got news, your questions and even our first viewer-submitted video, right here on BSD Now.. the place to B.. SD. Headlines Getting to know your portmgr (http://blogs.freebsdish.org/portmgr/2013/11/18/getting-to-know-your-portmgr-erwin-lansing/) In this interview they talk to one of the "Annoying Reminder Guys" - Erwin Lansing, the second longest serving member of FreeBSD's portmgr (also vice-president of the FreeBSD Foundation) He actually maintains the .dk ccTLD Describes FreeBSD as "the best well-hidden success story in operating systems, by now in the hands of more people than one can count and used by even more people, and not one of them knows it! It's not only the best operating system currently around, but also the most supportive and inspiring community." In the next one (http://blogs.freebsdish.org/portmgr/2013/11/25/getting-to-know-your-portmgr-martin-wilke/) they speak with Martin Wilke (miwi@) The usual, "what inspires you about FreeBSD" "how did you get into it" etc. *** vBSDCon wrap-up compilation (http://blog.hostileadmin.com/2013/11/20/vbsdcon-wrap-ups/) Lots of write-ups about vBSDCon gathered in one place Some from OpenBSD guys (http://undeadly.org/cgi?action=article&sid=20131121050402) Some from FreeBSD guys (http://freebsdfoundation.blogspot.com/2013/11/vbsdcon-trip-report-john-mark-gurney.html) Some from RootBSD (http://www.rootbsd.net/vbsdcon-2013-wrap-up/) Some from iXsystems (http://www.ixsystems.com/resources/ix/blog/vbsdcon-2013.html) Some from Verisign (http://blogs.verisigninc.com/blog/entry/builders_and_archaeologists) And of course our own wrap-up chat in BSD Now Episode 009 (http://www.bsdnow.tv/episodes/2013_10_30-current_events) *** Faces of FreeBSD (http://freebsdfoundation.blogspot.com/2013/11/faces-of-freebsd-each-week-we-are-going.html) This week they talk to Gábor Páli from Hungary Talks about his past as a game programmer and how it got involved with FreeBSD "I met János Háber, who admired the technical merits of FreeBSD and recommended it over the popular GNU/Linux distributions. I downloaded FreeBSD 4.3-RELEASE, found it reliable, consistent, easy to install, update and use." He's been contributing since 2008 and does lots of work with Haskell in ports He also organizes EuroBSDCon and is secretary of the FreeBSD Core Team *** Dragonfly 3.6 released (http://www.dragonflybsd.org/release36/) dports now default instead of pkgsrc Big SMP scaling improvements Experimental i915 and KMS support See our interview (http://www.bsdnow.tv/episodes/2013_11_13-the_gateway_drug) with Justin Sherrill if you want to hear (a lot) more about it - nearly an hour long *** Interview - Jordan Hubbard - jkh@freebsd.org (mailto:jkh@freebsd.org) / @omgjkh (https://twitter.com/omgjkh) FreeBSD's founding and future Tutorial Building an OpenBSD router, part 2 (http://www.bsdnow.tv/tutorials/openbsd-router) Note: there was a mistake in the video version of the tutorial, please consult the written version for the proper instructions. *** News Roundup pfSense 2.1 on AWS EC2 (http://blog.pfsense.org/?p=1132) We now have pfSense 2.1 available on Amazon's Elastic Compute Cloud (EC2) In keeping with the community spirit, they're also offering a free "public" AMI Check the FAQ and User Guide on their site for additional details Interesting possibilities with pfSense in the cloud *** Puffy on the desktop (http://distrowatch.com/weekly.php?issue=20131118#feature) Distrowatch, a primarily Linux-focused site, features an OpenBSD 5.4 review They talk about using it on the desktop, how to set it up Very long write-up, curious Linux users should give it a read Ends with "Most people will still see OpenBSD as an operating system for servers and firewalls, but OpenBSD can also be used in desktop environments if the user doesn't mind a little manual work. The payoff is a very light, responsive system that is unlikely to ever misbehave" *** Two-factor authentication with SSH (http://cmacr.ae/openbsd/security/networking/2013/11/25/ssh-yubi.html) Blog post about using a yubikey with SSH public keys Uses a combination of a OTP, BSDAuth and OpenBSD's login.conf, but it can be used with PAM on other systems as well Allows for two-factor authentication (a la gmail) in case your private key is compromised Anyone interested in an extra-hardened SSH server should give it a read *** PCBSD weekly digest (http://blog.pcbsd.org/2013/11/weekly-feature-digest-112313/) 10.0 has approximately 400 PBIs for public consumption They will be merging the GNOME3, MATE and Cinnamon desktops into the 10.0 ports tree - please help test them, this is pretty big news in and of itself! PCDM is coming along nicely, more bugs are getting fixed Added ZFS dataset options to PCBSD's new text installer front-end *** Feedback/Questions Ben writes in (http://slexy.org/view/s2ag1fA7Ug) Florian writes in (http://slexy.org/view/s2TSIvZzVO) Zach writes in (http://slexy.org/view/s20Po4soFF) Addison writes in (http://slexy.org/view/s20ntzqi9c) Adam writes in (http://slexy.org/view/s2EYJjVKBk) Adam (https://twitter.com/redshirtlinux)'s BSD Router Project tutorial can be downloaded here (http://bsdnow.cdn.scaleengine.net/bsdrouterproject.m4v). ***

After returning from a successful EuroBSDCon in Malta, we're back to get you caught up on all the latest news! We've got stories, interviews and a special treat for OpenBSD fans later in the show. All that and more on this week's BSD Now, the place to B.. SD. Headlines FreeBSD 9.2 released (https://www.freebsd.org/releases/9.2R/relnotes.html) FreeBSD 9.2-RELEASE is finally out Highlights include ZFS TRIM and LZ4 support, virtio drivers, dtrace and OpenSSH updates as well as lots of driver improvements Will be supported until 2014-09-30 Get out there and freebsd-update or buildworld! *** Four new NetBSD releases (https://blog.netbsd.org/tnf/entry/netbsd_5_2_1_and) NetBSD 5.2 and 5.1 branches get security and bugfix updates The 6.1 and 6.0 branches were updated soon after (https://blog.netbsd.org/tnf/entry/netbsd_6_1_2_and), also with security updates and bug fixes Check the show notes for the full changelog *** BIND being replaced by unbound in FreeBSD (http://freshbsd.org/commit/freebsd/r255597) Most FreeBSD users are familiar with BIND from the security notifications It has has many vulnerabilities over the years, and we'll finally be rid of it (http://blog.des.no/2013/09/dns-in-freebsd-10/) Being replaced with unbound and ldns, everyone rejoices (http://blog.des.no/2013/09/dns-again-a-clarification/) As of September 24th (https://svnweb.freebsd.org/base?view=revision&revision=255850), BIND is no longer built by default As of September 30th (http://freshbsd.org/commit/freebsd/r255949), BIND was completely removed Includes an easy to use script (http://freshbsd.org/commit/freebsd/r255809) for local DNS OpenBSD also has unbound in base (http://marc.info/?l=openbsd-cvs&m=137984954228414&w=2), but it's not built by default yet *** DragonflyBSD future plans (http://lists.dragonflybsd.org/pipermail/kernel/2013-September/062975.html) An announcement was posted that details some possible plans for Dragonfly dports (their version of FreeBSD ports) will be switching to GCC 4.7 i915 support is probably going to be in version 3.6 Work is being done on HAMMER 2, but it won't make it to 3.6 3.6 is also likely going to ditch pkgsrc as the default in favor of dports, due to a hugely positive reaction from the community *** FreeBSD ports get Stack Protector support (https://lists.freebsd.org/pipermail/freebsd-ports-announce/2013-September/000066.html) Some portsnap users noticed a massive sweep of every port being updated Shortly after, stack protector (https://en.wikipedia.org/wiki/Buffer_overflow_protection) support was announced by Bryan Drewery Only works on i386 and AMD64 on FreeBSD 10 and AMD64 on 9 Hopefully will become the default, but needs to go through some testing and exp-runs *** EuroBSDCon 2013 wrap-up chat BSD Now is back from EuroBSDCon with lots of stories We picked up an OpenBSD 5.4 CD set at EuroBSDCon, before the official release We'll give a little showcase of what's inside, they put a lot of effort into it Comes with the OS, source code, stickers, music, cool other stuff Consider supporting the OpenBSD project (http://www.openbsd.org/orders.html) *** Interview - Marshall Kirk McKusick - mckusick@freebsd.org (mailto:mckusick@freebsd.org) Various topics Tutorial Faster recompiles with ccache and tmpfs (http://www.bsdnow.tv/tutorials/ccache) News Roundup List of vBSDCon speakers posted (http://blog.hostileadmin.com/2013/09/09/reminder-vbsdcon-registrations-are-open/) Registration will be open until October 23rd Presentations covering FreeBSD, OpenBSD, FreeNAS and others *** Xen PVHVM added to GENERIC (https://svnweb.freebsd.org/base?view=revision&revision=255744) It's now possible to run FreeBSD 10 under Xen with the GENERIC kernel freebsd-update will work now With FreeBSD 10 ALPHA 4 (https://lists.freebsd.org/pipermail/freebsd-snapshots/2013-September/000045.html) just being released, should be interesting We should call the new kernel "XENERIC" *** Dragonfly AMD KMS port (http://lists.dragonflybsd.org/pipermail/kernel/2013-September/062993.html) A Dragonfly user has started porting the new FreeBSD AMD KMS driver Still a work in progress, asking for help from the community *** NetBSD gets an nVidia driver (http://mail-index.netbsd.org/source-changes/2013/09/18/msg047712.html) NetBSD gets a preliminary nVidia driver So far only supports the GeForce 2MX, so not a lot of use just yet No acceleration yet, but it's a start *** FreeBSD cracks the top 10 on DistroWatch (http://distrowatch.com/dwres.php?resource=popularity) Over the last year FreeBSD has steadily moved up the rankings from #18 to #10 Increasing from an average of 570 to 779 hits per day Surpassed CentOS, Puppy Linux and Slackware *** Feedback/Questions Charlie writes in (http://slexy.org/view/s21jRKf7lp) Kjell-Aleksander writes in (http://slexy.org/view/s2M0OKmxMK) Stefen writes in (http://slexy.org/view/s2YlVuhhUa) Sichendra writes in (http://slexy.org/view/s2P7KtE5x2) ***

We kick off the first episode with the latest BSD news, show you how to avoid intrusion detection systems and talk to Peter Hessler about BGP spam blacklists! Headlines Radeon KMS commited (https://lists.freebsd.org/pipermail/svn-src-head/2013-August/050931.html) Committed by Jean-Sebastien Pedron Brings kernel mode setting to -CURRENT, will be in 10.0-RELEASE (ETA 12/2013) 10-STABLE is expected to be branched in October, to begin the process of stabilizing development Initial testing shows it works well May be merged to 9.X, but due to changes to the VM subsystem this will require a lot of work, and is currently not a priority for the Radeon KMS developer Still suffers from the syscons / KMS switcher issues, same as Intel video More info: https://wiki.freebsd.org/AMD_GPU *** VeriSign Embraces FreeBSD (http://www.eweek.com/enterprise-apps/verisign-embraces-open-source-freebsd-for-diversity/) "BSD is quite literally at the very core foundation of what makes the Internet work" Using BSD and Linux together provides reliability and diversity Verisign gives back to the community, runs vBSDCon "You get comfortable with something because it works well for your particular purposes and can find a good community that you can interact with. That all rang true for us with FreeBSD." *** fetch/libfetch get a makeover (http://freshbsd.org/commit/freebsd/r253680) Adds support for SSL certificate verification Requires root ca bundle (security/rootcanss) Still missing TLS SNI support (Server Name Indication, allows name based virtual hosts over SSL) *** FreeBSD Foundation Semi-Annual Newsletter (http://www.freebsdfoundation.org/press/2013Jul-newsletter) The FreeBSD Foundation took the 20th anniversary of FreeBSD as an opportunity to look at where the project is, and where it might want to go The foundation sets out some basic goals that the project should strive towards: Unify User Experience “ensure that knowledge gained mastering one task translates to the next” “if we do pay attention to consistency, not only will FreeBSD be easier to use, it will be easier to learn” Design for Human and Programmatic Use 200 machines used to be considered a large deployment, with high density servers, blades, virtualization and the cloud, that is not so anymore “the tools we provide for status reporting, configuration, and control of FreeBSD just do not scale or fail to provide the desired user experience” “The FreeBSD of tomorrow needs to give programmability and human interaction equal weighting as requirements” Embrace New Ways to Document FreeBSD More ‘Getting Started' sections in documentation Link to external How-Tos and other documentation “upgrade the cross-referencing and search tools built into FreeBSD, so FreeBSD, not an Internet search engine, is the best place to learn about FreeBSD” Spring Fundraising Campaign, April 17 - May 31, raised a total of $219,806 from 12 organizations and 365 individual donors. In the same period last year we raised a total of $23,422 from 2 organizations and 53 individuals Funds donated to the FreeBSD Foundation have been used on these projects recently: Capsicum security-component framework Transparent superpages support of the FreeBSD/ARM architecture Expanded and faster IPv6 Native in-kernel iSCSI stack Five New TCP Congestion Control Algorithms Direct mapped I/O to avoid extra memory copies Unified Extensible Firmware Interface (UEFI) boot environment Porting FreeBSD to the Genesi Efika MX SmartBook laptop (ARM-based) NAND Flash filesystem and storage stack Funds were also used to sponsor a number of BSD focused conferences: BSDCan, EuroBSDCon, AsiaBSDCon, BSDDay, NYCBSDCon, vBSDCon, plus Vendor summits and Developer summits It is important that the foundation receive donations from individuals, to maintain their tax exempt status in the USA. Even a donation of $5 helps make it clear that the FreeBSD Foundation is backed by a large community, not only a few vendors Donate Today (http://www.freebsdfoundation.org/donate) *** The place to B...SD Ohio Linuxfest, Sept. 13-15, 2013 (http://ohiolinux.org/schedule) Very BSD friendly Kirk McKusick giving the keynote BSD Certification on the 15th, all other stuff on the 14th Multiple BSD talks *** LinuxCon, Sept. 16-18, 2013 (http://events.linuxfoundation.org/events/linuxcon-north-america) Dru Lavigne and Kris Moore will be manning a FreeBSD booth Number of talks of interest to BSD users, including ZFS coop (http://linuxconcloudopenna2013.sched.org/event/b50b23f3ed3bd728fa0052b54021a2cc?iframe=yes&w=900&sidebar=yes&bg=no) EuroBSDCon, Sept. 26-29, 2013 (http://2013.eurobsdcon.org/eurobsdcon-2013/talks/) Tutorials on the 26 & 27th (plus private FreeBSD DevSummit) 43 talks spread over 3 tracks on the 28 & 29th Keynote by Theo de Raadt Hosted in the picturesque St. Julians Area, Malta (Hilton Conference Centre) *** Interview - Peter Hessler - phessler@openbsd.org (mailto:phessler@openbsd.org) / @phessler (https://twitter.com/phessler) Using BGP to distribute spam blacklists and whitelists Tutorial Using stunnel to hide your traffic from Deep Packet Inspection (http://www.bsdnow.tv/tutorials/stunnel) News Roundup NetBSD 6.1.1 released (https://blog.netbsd.org/tnf/entry/netbsd_6_1_1_released) First security/bug fix update of the NetBSD 6.1 release branch Fixes 4 security vulnerabilities Adds 4 new sysctls to avoid IPv6 DoS attacks Misc. other updates *** Sudo Mastery (http://blather.michaelwlucas.com/archives/1792) MWL is a well-known author of many BSD books Also does SSH, networking, DNSSEC, etc. Next book is about sudo, which comes from OpenBSD (did you know that?) Available for preorder now at a discounted price *** Documentation Infrastructure Enhancements (http://freebsdfoundation.blogspot.com/2013/08/new-funded-project-documentation.html) Gábor Kövesdán has completed a funded project to improve the infrastructure behind the documentation project Will upgrade documentation from DocBook 4.2 to DocBook 4.5 and at the same time migrate to proper XML tools. DSSSL is an old and dead standard, which will not evolve any more. DocBook 5.0 tree added *** FreeBSD FIBs get new features (https://svnweb.freebsd.org/base?view=revision&revision=254943) FIBs (as discussed earlier in the interview) are Forward Information Bases (technical term for a routing table) The FreeBSD kernel can be compiled to allow you to maintain multiple FIBs, creating separate routing tables for different processes or jails In r254943 ps(1) is extended to support a new column ‘fib', to display which routing table a process is using *** FreeNAS 9.1.0 and 9.1.1 released (http://www.ixsystems.com/resources/ix/news/ixsystems-announces-revolutionary-freenas-910-release.html) Many improvements in nearly all areas, big upgrade Based on FreeBSD 9-STABLE, lots of new ZFS features Cherry picked some features from 10-CURRENT New volume manager and easy to use plugin management system 9.1.1 released shortly thereafter to fix a few UI and plugin bugs *** BSD licensed "patch" becomes default (http://freshbsd.org/commit/freebsd/r253689) bsdpatch has become mature, does what GNU patch can do, but has a much better license Approved by portmgr@ for use in ports Added WITHGNUPATCH build option for people who still need it ***