Hacker Public Radio

This show has been flagged as Explicit by the host. Freedom 0–3 (verbatim reference) The four freedoms, GNU/FSF: https://www.gnu.org/philosophy/free-sw.html Richard Stallman: https://www.stallman.org/ Named FOSS figures (freedom 3 list) Linus Torvalds (Linux): https://en.wikipedia.org/wiki/Linus_Torvalds Dries Buytaert (Drupal): https://dri.es Guido van Rossum (Python): https://gvanrossum.github.io Ian Murdock (Debian): https://en.wikipedia.org/wiki/Ian_Murdock Brian Behlendorf (Apache): https://en.wikipedia.org/wiki/Brian_Behlendorf Miguel de Icaza (GNOME/Mono): https://en.wikipedia.org/wiki/Miguel_de_Icaza Infrastructure block (post beer 1) curl / Daniel Stenberg: https://curl.se and https://daniel.haxx.se xkcd 2347 (dependency comic): https://xkcd.com/2347/ xz backdoor, CVE-2024-3094: https://en.wikipedia.org/wiki/XZ_Utils_backdoor Andres Freund's original disclosure: https://www.openwall.com/lists/oss-security/2024/03/29/4 Licensing block (beer 2–3) GPL: https://www.gnu.org/licenses/gpl-3.0.html MIT License: https://opensource.org/license/mit FreeBSD: https://www.freebsd.org macOS/Darwin BSD lineage: https://en.wikipedia.org/wiki/Darwin_(operating_system) PlayStation using FreeBSD (Orbis OS): https://en.wikipedia.org/wiki/Orbis_OS Industry acquisitions (beer 4) Microsoft acquires GitHub, 2018, $7.5B: https://news.microsoft.com/2018/06/04/microsoft-to-acquire-github-for-7-5-billion/ FOSDEM field report (beer 4–5) FOSDEM: https://fosdem.org Ladybird browser / Andreas Kling: https://ladybird.org Godot Engine: https://godotengine.org Home Assistant: https://www.home-assistant.io Redis → Valkey fork: https://valkey.io Terraform → OpenTofu fork: https://opentofu.org Homework block (beer 5) Ollama: https://ollama.com Codeberg: https://codeberg.org LibreOffice: https://www.libreoffice.org Firefox: https://www.mozilla.org/firefox Provide feedback on this episode.

This show has been flagged as Clean by the host. In the seventeenth episode of the HPR Beer Garden, the guys look at a beer style (or label) that has recently appeared from across the pond: DDH IPA. Kevie tries out Smug DDH IPA by Northern Monk , Paul samples Hazy Jaipur by the Thornbrige Brewery and Dave enjoys Pinball by Brewdog . Connect with the guys on Untappd : Dave Paul Kevie The intro sounds for the show are used from: https://freesound.org/people/mixtus/sounds/329806/ https://freesound.org/people/j1987/sounds/123003/ https://freesound.org/people/greatsoundstube/sounds/628437/ The next 3 beer styles to be reviewed: Amber Ale Lager Mild Provide feedback on this episode.

This show has been flagged as Clean by the host. I tend to prefer Ebook and Audiobooks to physical books, and these are some of the apps I use, and some of the sources I go to to find Ebooks or Audiobooks. These can be inexpensive, or in the case of Public Domain, free. Ebooks and Audiobooks I no longer wish to devote space to accumulating and storing physical books. As a former college professor I had more of them than most people, and tended to fill the house with bookcases and shelves. That made a certain sense in the pre-Internet years since I would often want to go back to them to look up things. But now I have more information at my fingertips than I ever had in those days, and I find it more convenient to just look up anything I need to know. And with Ebooks I can take my library with me anywhere I go. This means if I am sitting in a waiting room for a Doctor's appointment (and at my age that happens often enough) I can just pull out my phone and open up a book. I always have at least one. Right now on my phone I have apps for Kindle , Google Play , Barnes & Noble Nook , Kobo Cantook (successor to Aldiko), Bookshop.org , and Calibre . As for devices other than my phone, I own a kindle, I once owned a Nook, and I will buy a Kobo reader when Trump is gone. Kobo is Canadian, and last time I looked Trump had put on a tariff that was twice what the device costs. And finally I have the Bookshop.org app and Calibre. I started collecting Ebooks some years back, and at first the only option seemed to be buying them from large corporations. That has changed now, but I still have collections On Kindle, Google Play, Nook, and Kobo. I have all four because it seemed prudent to not put all my eggs in one basket. But now there are other options. One I like to use when possible is Bookshop.org. It is group of independent bookstores and I would rather support them than some giant corporation. And they seem to have all of the popular titles. Another similar site is Books-a-Million , which lets you order online and pick up at a local store if you need a physical book. Cantook and Calibre are both good for holding any miscellaneous Ebooks you have, and you can give them your Ebooks from the other sellers, though I have found that the native apps tend to work better. These days I rarely purchase books on Kindle, etc. But sometimes they have a special deals. If you want to build your library, there are some good places to try. One is BookBub which will send you emails with deals on books. On any one e-mail you may not see anything you want, but I found that once or twice a month I did see books I wanted and at good prices. And Books-a-Million has 99 cent deals to check out. But if you want to get free books, you have several options. If you want a chance to read a popular Ebook but don't need to own it, see what your local Library offers. Mine offers Hoopla which is good for Ebooks, Audiobooks, and streaming music. What you can find depends on what your library has in its collection, but my wife uses it almost daily for audiobooks. Then there are the works that have become part of the Public Domain. This varies somewhat by your country, but you might be surprised to learn how much is available. For Ebooks one of the biggest sites is Project Gutenberg . They follow US Copyright law for their collection, and they are a non-profit that can always use some support, so if you use them, give them a few Dollars or Euros. They tend to be the source for many or most other sites for Public Domain Ebooks They have over 75,000 free Ebooks. A newer site is Faded Page which is much smaller but focused on Canadian works. Standard Ebooks is a site that pulls from Project Gutenberg but does some additional formatting and is very easy to read online in your browser. For Audiobooks a good source is Open Culture , which advertises it has 1,000 free Audiobooks. We should also mention LibriVox , which also pulls from Project Gutenberg for its texts, but then turns them into Public Domain Audiobooks. They are always looking for volunteers who have a microphone and a computer to record Public Domain audiobooks. And last but not least the Internet Archive has tons of Ebooks and Audiobooks that are free. Links https://www.amazon.com/kindle/ https://play.google.com/store/books?hl=en_US&pli=1 https://www.barnesandnoble.com/b/ebooks-nook/_/N-8qa https://www.kobo.com/us/en?srsltid=AfmBOoquJmiNH_aVTUx22AIRlBH8CF4iir4H8qpTKjQ4dZ-fSMCDAaD7 https://market.cantook.com/ https://bookshop.org/ebooks https://calibre-ebook.com/ https://www.booksamillion.com/ https://www.bookbub.com/ https://www.hoopladigital.com/ https://www.gutenberg.org/ https://www.fadedpage.com/ https://standardebooks.org/ https://www.openculture.com/freeaudiobooks https://librivox.org/ https://archive.org/ https://www.palain.com/ebooks-and-audiobooks/ Provide feedback on this episode.

This show has been flagged as Clean by the host. MakeACopy MakeACopy is an open-source document scanner app for Android that allows you to digitize paper documents with OCR functionality. The app is designed to be privacy-friendly, working completely offline without any cloud connection or tracking. Features Camera Scanning : Capture documents using the device camera Edge Detection : Automatic document edge detection using OpenCV, enhanced with an ONNX model (DocAligner) Perspective Correction : Adjust and crop documents with manual or automatic perspective correction Image Enhancement : Apply filters (grayscale, contrast, sharpening) OCR : Offline text recognition with Tesseract OCR Review : Interactive review and correction of OCR results with word-level editing, dictionary-based suggestions, confidence highlighting, and re-OCR for individual words PDF Export : Save as searchable PDF with recognized text JPEG Export : Export scans as high-quality JPEG images (configurable quality, color/BW) Multi-page Scanning : Combine multiple pages into one document; reorder and manage pages before export Last Scans : Quickly access and reuse your most recent scans Share & Save : Export locally or share with other apps Dark Mode : Material 3 theme with day/night support Privacy-Focused : 100% offline functionality, no internet connection required Provide feedback on this episode.

This show has been flagged as Explicit by the host. Overview: Elroy sits down with Elsbeth to unpack her perspective on AI ethics — where the technology helps, where it risks harm, and what responsible use actually looks like in practice. Topics covered (In no particular order, as the rabid squirrels of Elsbeth's AuDHD brain don't do "order"): Elsbeth's path into AI evaluation and content moderation, and how that shaped her thinking on ethics in practice What "ethical use" of AI means day-to-day, not just in the abstract Respecting intellectual property — the case for protecting artists and creators as AI tools become more capable and widespread Finding the balance: supporting AI's genuine benefits without letting them come at creators' expense AI as an accessibility tool — its particular promise for neurodivergent people, and where it can lower barriers others don't face Where the conversation is heading next, and what listeners should watch for Closing thoughts: Elsbeth's take is one of cautious optimism — a call to hold both truths at once: AI can do real good, and that good has to be built on respect for the people whose work made it possible. Further Reading & Resources On AI, Ethics, and Intellectual Property AI and Intellectual Property in 2026: why transparency and court rulings will define the next phase of governance — The AI Journal https://aijourn.com/ai-and-intellectual-property-in-2026-why-transparency-and-court-rulings-will-define-the-next-phase-of-governance/ Global laws governing intellectual property rights for AI-generated works — Discover Artificial Intelligence, Springer Nature Link https://link.springer.com/article/10.1007/s44163-026-01165-8 AI and IP laws 2026: Understanding Authorship and Ownership legalities — AnalystIP https://analystip.com/ai-and-ip-laws-2026-authorship-ownership-explained/ AI Art in 2026: The Ethics, Law, and What Artists Should Know — UrduPure https://www.urdupure.com/blog/ai-art-in-2026-the-ethics-law-and-what-artists-should-know On AI and Neurodivergent Accessibility Navigating Neurodivergence with AI Chatbots: Benefits, Tensions, and Implications for HCI — CHI 2026 Conference Proceedings https://dl.acm.org/doi/10.1145/3772318.3791334 Is AI making work more accessible for neurodiverse people? — Everway https://www.everway.com/en-gb/blog/is-ai-making-work-more-accessible-for-neurodiverse-people/ Autoethnographic Insights from Neurodivergent GAI "Power Users" — NCBI/PMC https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12645485/ A scoping review of inclusive and adaptive human–AI interaction design for neurodivergent users — Disability and Rehabilitation: Assistive Technology https://www.tandfonline.com/doi/full/10.1080/17483107.2025.2579822 Show rated *explicit* for possible course language and strong opinions. Provide feedback on this episode.

This show has been flagged as Clean by the host. Heinlein wrote a series of books under contract to Scribners that were aumed at younger readers, pre-teen and teen. Today we would call them Young Adult, but back then they were called Juveniles. But even an adult reader can enjoy many of these books. Because they were aimed at younger readers, he could not always be as explicit as he might have liked, but if you pay attention you just might noticed he slipped in something subversive. Heinlein: The Juveniles As a boy I read voraciously. I remember my mother organizing weekly trips to the town library, where I would load up on books, and I quickly became focused on Science Fiction, along with the usual stuff kids read, like the Hardy Boys, Tom Swift, and the Walter Farley horse books. But the idea of going into space grabbed me very early. And one of the first authors I read was Heinlein. He had taken a leave from publishing during World War II, when he was doing research at the Philadelphia Navy Yard. But when the war was over, he set out to move beyond the “pulps” and expand the market for his stories. And one big market for him was what were called “juveniles” at the time, and would today be called “Young Adult”. I was reading adult fiction by the time I was 11 or 12, but before that (and even after that, in fact) I read these Heinlein novels with great relish. Heinlein's target audience for these novels was teenage boys. He did a few stories aimed at girls, but mostly he wrote for boys. And these were mostly “coming of age” stories where the teenage protagonist has adventures, and as a result grows and develops. They are very loosely related via some internal references, but should really be thought of as stand-alone stories. They also in some cases have references to his other stories, including the Future history stories. He wrote 13 of these novels, one per year, from 1947 to 1959. The series was published by Scribners until the last one was rejected by them. Heinlein then published it with a different publisher, and stopped writing these novels altogether in favor of more adult fiction. The novels roughly form a progression telling the story of space exploration. It starts with a trip to the Moon, then Venus, Mars, Jupiter's moon, and so on until we reach the Lesser Magellanic Cloud. Rocket Ship Galileo (1947) – While readable, this initial effort was not up to Heinlein's later standards. The plot concerns three teenagers who assist an uncle to build a rocket ship and go to the Moon. When they get there they discover Nazis have already arrived. The Nazis try to kill the group, but they succeed in turning the tables, stealing the Nazi ship, and returning to Earth as heroes. This novel became the basis (loosely) for the movie Destination Moon (1950), and I personally would consider the movie to be superior to the novel. It is also worth noting that the theory brought up in Blowups Happen reappears in this novel. They find evidence of an ancient Lunar civilization that was destroyed, and theorize that the craters on the moon were caused by the explosion of nuclear reactors that caused the extinction of the civilization. Space Cadet (1948) – This was in part the inspiration for the Tom Corbett franchise, which licensed the name Space Cadet from Heinlein. It is about a young man who is accepted to the Academy for the Space Patrol. It follows him through his education in the Academy, and then into his first mission after graduating. This holds up better than the previous novel. And it is tied back to the story The Long Watch from the Future History. Part of the story takes place on an inhabited Venus that is cloudy and swampy. Red Planet (1949) – This is set on Mars, as it is also portrayed in Stranger In A Strange Land (1961). Mars is inhabited by native Martians, but also by human colonists. It has the canals, and with seasonal changes the colonists migrate from north to south and back. A pair of teenage boys get caught up in a revolution when the evil corporation that controls the colony pushes the colonists too far. In the end you wish there really were canals and Martians. Farmer In The Sky (1950) – This is one of the best, as seen by the Retro Hugo this novel won in 2000. Jupiter's moon Ganymede is being terraformed because Earth is overcrowded and food is rationed. A teenage boy and his family emigrate to Ganymede and try to make a life there, which they eventually succeed in doing. The Green Hills of Earth is mentioned here, tying this into the future History. There are frequent references to the Boy Scouts, due to the fact that the story ran as a serial in Boy's Life magazine. Between Planets (1951) – A teenage boy is caught up in interplanetary intrigue. Venus and Mars have colonies, but Earth is trying to control them too much. So revolution is on the menu. This is clearly patterned in the colonial wars of the 18th and 19th centuries, such as the American Revolution against England. By this point the so-called “Juveniles” are really having more adult content, and reviewers re starting to rate them in comparison with adult science fiction. This novel was also first serialized in Boy's Life magazine. The Rolling Stones (1952) – Here we have teenage twin boys, Castor and Pollux, as the protagonists. They and their family live on the Moon, but decide to travel, so this novel is a kind of travelogue as they go to Mars, then to the Asteroid belt. The grandmother, Hazel Stone, appears in Heinlein's later works as well. One interesting episode in this story involves “martian flat cats”, which are furry, lovable, and reproduce like mad with the right conditions. If this sound like Star Trek's Tribbles, that is also what the Star Trek producers thought, so they got permission from Heinlein to use the idea. Starman Jones (1953) – We are now further into the future, and the human race is exploring the stars. Unfortunately, very restrictive guilds closely control who can participate in this. A teenage boy named Max, who happens to have an eidetic memory, has memorized the Astrogation tables from his uncle's books, and wants to join the Guild, but is turned down. He lies his way onboard a ship, and through a series of events becomes the only one who can guide the ship home. The Star Beast (1954) – A teenage boy has an alien “pet” his great-grandfather had brought back that has grown very large, and is considered a nuisance. A court decides the beast must be killed, but that proves easier to say than to do. It appears that the beast isn't even aware that people are trying to kill it. Meanwhile, a powerful and hitherto unknown alien species demand the return of one of their own, or they will destroy the Earth. Of course, it is this beast, who is actually royalty to the alien species. One interesting point is that government officials in this story are portrayed sympathetically as intelligent and dedicated. Tunnel In The Sky (1955) – In the future humanity is colonizing other planets, and a group of teenagers are taking their final survival test. They are sent to a planet and told that they have to survive for 10 days, But more than 10 days go by with no pickup, and they know something went wrong. So they have to establish their own little society to keep surviving. An interesting note is that the protagonist, Rod Walker, is black. It was never explicitly stated in the text, but Heinlein was firm in stating this. The clue is when the others expect Rod to end up with Caroline, who is explicitly stated to be black. This was Heinlein being subtly subversive. To have a black protagonist for a boy's story in 1955 in America would be impossible. but Heinlein was completely anti-racist, among other things. Time For The Stars (1956) – This is a novel that takes Relativity seriously, which was not common in the 1950s. Researchers have discovered that some twins and triplets can communicate telepathically (and instantaneously), and so when a group of ships is sent out to explore other star systems, one twin is on the ship and the other remains on Earth to provide communication. The twin on Earth ages must faster than the one in space, of course, and eventually the Earth twin dies, But they discover that the connection sometimes passes down through the family, so the protagonist, Tom Bartlett, becomes connected first to his niece, then his grandniece, and finally his great-grandniece. Citizen of the Galaxy (1957) – This book is about a future slave trade, which Heinlein strongly hated. The protagonist is a boy who is bought at a slave auction by an old beggar, but the beggar is more than he seems. He is actually spying and gathering data regarding the slave trade. When discovered, he commits suicide, but he had prepared the young boy, Thorby, who then contacts the Free Traders who spirit him away. He has to adapt to this new society, but then is delivered to the Hegemonic Guard. It turns out his “father” (i.e. the man who bought him) was an officer in this organization. And when they run the background checks, they discover that he is the heir to a large conglomerate, and that conglomerate may be implicated in the slave trade. Have Space Suit—Will Travel (1958) – Clifford “Kip” Russell dreams of going to the Moon, and enters a contest with that as the top prize. Unfortunately, he wins the somewhat lesser prize of a used spacesuit. The first few chapters focus on him doing repairs and maintenance to make it functional again, and this displays Heinlein's engineering background. It is more engaging than you might think. Then while wearing the spacesuit he receives a radio message, and he is kidnapped along with an alien called “The Mother Thing” and a young girl who is a genius. The kidnappers are a group of aliens who consider anyone not of their race to be animals. The trio first try to escape on the Moon, but are recaptured, then taken to Pluto, where they succeed in killing the alien kidnappers. Then they are taken to the Lesser Magellanic Cloud to be put on trial to determine if the human race should be allowed to live. Starship Troopers (1959) – This is the novel that Scribners rejected, and which brought the Juvenile series to an end. And it bears absolutely no resemblance at all to the movie, to the point that for Heinlein fans the word Verhoeven is considered an obscenity. A young man, Juan “Johnny” Rico joins the military, where he has to grow up and then take part in a war against an insectoid race, but that is all background really. The book is primarily a glorification of military service, which is not surprising given Heinlein's background. And it focuses on a series of discussions under the heading of “History and Moral Philosophy”, which lets Heinlein expound on his values and beliefs. The novel won a Hugo, but it is an add one given that the plot is secondary to the philosophizing. One of the most controversial ideas is that in this society the right to vote is limited to people who have been in Federal Service. Heinlein said this didn't have to be military, but the only ones we see are in fact military veterans. So, these are the Heinlein Juveniles. In my opinion, many of them are quite good reading for adults. The thing that separates them from adult novels in Heinlein's body of work is the lack of any sex element. That would become prominent in Heinlein's later adult novels, but it was not something you could put in a book aimed at teenagers, certainly not in the 1950s, and arguably the case today as well. Links https://en.wikipedia.org/wiki/Rocket_Ship_Galileo https://en.wikipedia.org/wiki/Destination_Moon_(film) https://en.wikipedia.org/wiki/Space_Cadet https://en.wikipedia.org/wiki/Tom_Corbett,_Space_Cadet_(TV_series) https://en.wikipedia.org/wiki/Red_Planet_(novel) https://en.wikipedia.org/wiki/Stranger_in_a_Strange_Land https://en.wikipedia.org/wiki/Farmer_in_the_Sky https://en.wikipedia.org/wiki/Scout_Life https://en.wikipedia.org/wiki/Between_Planets https://en.wikipedia.org/wiki/The_Rolling_Stones_(novel) https://en.wikipedia.org/wiki/Starman_Jones https://en.wikipedia.org/wiki/The_Star_Beast_(novel) https://en.wikipedia.org/wiki/Tunnel_in_the_Sky https://en.wikipedia.org/wiki/Time_for_the_Stars https://en.wikipedia.org/wiki/Citizen_of_the_Galaxy https://en.wikipedia.org/wiki/Have_Space_Suit%E2%80%94Will_Travel https://en.wikipedia.org/wiki/Starship_Troopers https://www.palain.com/science-fiction/the-golden-age/robert-a-heinlein/heinlein-the-juveniles/ Provide feedback on this episode.

This show has been flagged as Clean by the host. sunshine / xbox contlers / plex / steam / UWPHook / Bluetoothctl restart / HDMI dongle https://www.youtube.com/watch?v=FM4FbA4-W_c UGREEN USB C Hub 5 in 1 Multiport Adapter Revodok 105 4K HDMI, 100W Power Delivery, 3 USB-A Data Ports, USB C Dongle for MacBook Pro/Air, iPad Pro, iMac, iPhone 16 Pro/Pro Max, XPS, Thinkpad https://www.amazon.com/dp/B0BR3M8XHK Detached Command: @ECHO OFF cd "C:backupgamestream_launchpad" c:windowsSystem32HdrSwitcher.exe disable start /MIN gamestream_launchpad.exe 1920 1080 gamestream_playnite.ini .bindkeys rc cat .xbindkeysrc /bin/bash /home/plex/.local/bin/Plex.sh /bin/bash /home/plex/.local/bin/Steam.sh /home/plex/.local/bin/kasa --host 192.168.1.239 --port 9999 --type dimmer on; /home/plex/.local/bin/kasa --host 192.168.1.239 --port 9999 --type dimmer brightness 100;xgamma -gamma 1.3 /home/plex/.local/bin/kasa --host 192.168.1.239 --port 9999 --type dimmer on; /home/plex/.local/bin/kasa --host 192.168.1.239 --port 9999 --type dimmer brightness 40;xgamma -gamma 1.3 /home/plex/.local/bin/kasa --host 192.168.1.239 --port 9999 --type dimmer on; /home/plex/.local/bin/kasa --host 192.168.1.239 --port 9999 --type dimmer brightness 12;xgamma -gamma 1.3 /home/plex/.local/bin/kasa --host 192.168.1.239 --port 9999 --type dimmer off;xgamma -gamma 1.3" cat /home/plex/.local/bin/Steam.sh # requires wmctrl -l and xbindkeys # # xdotool windowactivate ```xdotool search --name 'window name'``` # 0 StartSteamLink(){ echo "restarting bluetooth" sudo systemctl restart bluetooth echo "Waking Game PC" sudo /usr/sbin/etherwake -i enp1s0 -D "d8:bb:c1:a2:2c:0b" echo "Starting steamlink" killall -9 steamlink killall moonlight sleep 20 # /snap/bin/moonlight /home/plex/.local/bin/Moonlight-6.1.0-x86_64.AppImage } StartSteamLink SUMMARY: Speaker discusses technical challenges with Sun Shine, Moonlight, and Play Night setups. IDEAS: Sun Shine and Moonlight enable remote game streaming. Play Night acts as a multimodal launcher for Steam. Resolution changes cause display issues during gameplay. Proper exit sequence is critical to revert settings. Ultra-wide resolutions are unusable in some configurations. Multiple launchers require separate logins for different games. Environment variables can adjust client dimensions. Background commands improve system stability. Cat named Mojo is mentioned during the discussion. 5G networks support slower-paced gaming. Technical glitches include frame drops and latency. Steam accounts are tied to specific game libraries. Detached commands run in the background without user interaction. GameStream Launchpad manages resolution and launcher paths. Magical system behavior simplifies remote gaming. Star Wars and Baldur’s Gate are cited as example games. 22nd Millie thing refers to network latency. Multiple Steam accounts complicate game access. Task killing reverts system settings automatically. Cross-platform streaming solutions are highlighted. RECOMMENDATIONS: Use environment variables for client resolution adjustments. Exit applications properly to revert settings. Set up detached background commands for stability. Monitor ultra-wide resolution compatibility. Utilize Play Night for Steam account management. Avoid multiple logins for different game libraries. Opt for cross-platform streaming solutions. Check network latency for smooth gameplay. Use bat files for automated task execution. Configure GameStream Launchpad for resolution changes. Prioritize proper task termination to prevent glitches. Test 5G networks for slower-paced gaming. Leverage environment variables for client settings. Ensure all applications exit before shutting down. Use detached commands for background processes. Verify resolution compatibility with ultra-wide displays. Combine Sun Shine and Moonlight for reliable streaming. Log in once for multiple Steam accounts. Address frame drops in low-latency networks. Stream games universally across platforms. Provide feedback on this episode.

This show has been flagged as Clean by the host. -------------------- 01 Introduction In a response to a post on a previous episode, I said that I would take a look at ID3 tags. ID3 tags are text information that is added to an MP3 audio file, such as the author, date, name, and other information. 02 While I am at it I will also look at Vorbis comments, which perform a similar function for vorbis, the container format for "ogg" audio files. As example audio files, I will use a recent HPR episode, hpr4678 in both mp3 and vorbis formats plus also one from another podcast as well. 03 There is Free Software which you can use to view, edit, or remove both types of tags or comments, and I will describe how to use it in this episode. 04 I will cover how to view tags and extract the text information, as well as how to strip the tags from a file and why you may wish to do so under certain very specific circumstances. I won't cover how to add to or edit tags in an MP3 or OGG file, as that is a more involved subject that I don't have much experience with. 05 I will mainly talk about ID3 MP3 tags rather than vorbis comments for the simple reason that the situation with MP3 files is an utter mess while vorbis comments are very straightforward and so there isn't as much to say about them. 06 As often happens when researching a subject to write a podcast script, I have learned quite a bit that I didn't know previously, and discovered that things that I thought I did know were wrong. I hope that you may learn a few things from this episode that you didn't know previously either. -------------------- 07 Background I won't go over the detailed history of ID3 tags, as I don't have a reliable source for this. Briefly however, so far as I can determine, there is no official independent standard for ID3 tags or vorbis comments. Both seem to be more in the nature of a convention that was created by an unofficial group of contributors rather than something issued by a standards body. 08 However, I don't see the lack of a conventional standards body as necessarily a problem with respect to the use of ID3 tags or vorbis comments. I just am not familiar enough with the industry to know who to regard as being an authoritative source when it comes to the history and development of them Since I am unsure as who to credit with what developments, I will avoid that sort of detail. 09 ID3 Tags There is a web site with the URL of ID3.org that seems to have the best reference material on the topic of ID3 tags. According to this site, the term "ID3" means "IDentify an MP3". 10 Vorbis Comments For vorbis, the reference site seems to be xiph.org. Although the implementation details may differ from ID3, from our perspective as podcast listeners, they can be seen as more or less equivalent in terms of what I am going to address here. -------------------- 11 ID3 Details Versions One of the problems with ID3 is that there are multiple incompatible versions, particularly from the very early days. ID3v1 tags are located at the end of the audio file in the last 128 bytes. This was done for compatibility reasons to allow early MP3 players to simply ignore the tags if they didn't know how to deal with them. The tags would simply appear as a short burst of static from the perspective of these early players. 12 ID3v2 moved the tags to the start of the MP3 file to allow players to scan the tags for information such as titles without having to read to the end of the file to find them. There are far fewer limits on the amount of information that can be placed in ID3v2 tags. 13 ID3v1 is obsolete and only very old players will require it. However, it is still used by some publishers for backward compatibility reasons. ID3v1 included numerical musical "genre" category codes which apparently turned out to be a very bad idea in practice. 14 Furthermore, all genre categories above 70 were defined by an audio software company called Nullsoft who created software such as Winamp. These codes were never actually part of the ID3 standard, although there was never really a standard to begin with. 15 The current ID3 version is 2.3. There is a version 2.4, but apparently it is not actually generally accepted and may be a developmental dead end. 16 The id3.org web site contains a copy of the ID3v2.3 standard, but I am not going to address the technical details here. This would only be of interest to someone who was creating software to read and write ID3 tags. 17 ID3v2 Frames The ID3v2 information is encoded into what are called "frames". The text information is contained in text information frames. 18 Text information frames start with a set of four character identifiers, all starting with the capital letter 'T'. Examples "TALB" is the "Album/Movie/Show title" frame. "TIT2" is the "Title/Songname/Content description" frame. "TYER" is the "Year" frame. There are many more, but I won't go into more detail here. 19 HPR ID3 Tags HPR makes an interesting case study because they use both ID3v1 and ID3v2 in the same file. This can cause some interesting problems with software that tries to read those tags. This is because most software appears to expect one or the other, but not both. However, so long as this does not cause problems with anything that actually plays the files, this is not a problem so far as people who just want to listen to podcasts are concerned. It does mean though that we have more to talk about than we would otherwise would have had. -------------------- 20 Vorbis Comment Fields The situation with Vorbis comments is much simpler, as there seems to be just one standard that was adhered to from the start rather than a succession of hacks. Information is stored in "fields", which xiph describes as being like Unix environment variables. These consist of a field name followed by an equal sign and then the information intended for that field. Field names are case insensitive. 21 The field names are not firmly defined at this stage, but there is a list of recommended names. Examples are "TITLE" is the Track or Work name. "ARTIST" is the person responsible for the work. "LICENSE" is the license information. 22 You can see the complete list for yourself on the comment field and header specification page at xiph.org -------------------- 23 Software for Listing, Adding, and Modifying Tags and Fields There are three software packages that I will now describe which allow you to list, add, modify, and remove tags and fields. There are other packages which can do the same, including some which offer a GUI interface. However, I will limit myself to describing these three. The principles should be the same for others. 24 ffprobe ffprobe is part of the ffmpeg package. If you have listened to my previous episodes on audio, you will have heard me talk about ffmpeg. ffprobe is used to display information about media files rather than for modifying them. ffprobe is licensed under the GPLv2 or later. ffprobe can be used to display information about both MP3 and OGG Vorbis files. 25 ID3v2 The next is the rather aptly named id3v2 and works with MP3 files. On Linux systems, this should be provided by the id3v2 package. On Debian derivatives this can be installed as follows sudo apt install id3v2 26 This also installs a man page which provides a brief list of the options. According to the README file in the source tarball, this is published under the LGPL ID3v2 is particularly useful for displaying ID3v1 tags. 27 vorbiscomment The third is "vorbiscomment" and works with OGG files. On Linux systems this should be provided by the "vorbis-tools" package. On Debian derivatives this can be installed as follows sudo apt install vorbis-tools 28 The vorbiscomment program is used to list or edit comments in Ogg Vorbis files. This also installs a man page which provides a brief list of the options. According to the license.lgpl file in the source tarball, this is published under the LGPL version2. 29 The display format for vorbiscomment follows the data definition format in the vorbis standard, whereas ffprobe re-formats it to match its own preferred appearance. -------------------- 30 Examples from HPR With the background out of the way, I will now give several examples of how to list the tags or fields. 31 Listing ID3v1 Tags As previously mentioned HPR uses both ID3v1 and ID3v2 tags in the same file. However, so far as I can tell, most software seems to look for ID3v2 tags first, and don't display the ID3v1 tags if both are present. 32 The ID3v2 program however seems to do the opposite, at least with HPR podcasts. However, the number of samples that I have which have both are rather limited, so I can't be sure if this is always the case or if this is a side effect of something else. I did mention that ID3 tags were a mess, didn't I? 33 Let's go on to our example however. If we want to see the ID3v1 tags, then using hpr4678 as an example, if we type id3v2 -l hpr4678.mp3 = we get the following result. 34 id3v1 tag info for hpr4678.mp3: Title : High Resolution Elapsed Time i Artist: Whiskeyjack Album : Hacker Public Radio Year: 2026, Genre: Unknown (186) Comment: https://hackerpublicradio.or Track: 0 hpr4678.mp3: No ID3v2 tag 35 The first line tells us that this information is ID3v1 tags. The last line tells us that there are no ID3v2 tags. This last line is incorrect, but we will come back to that later. 36 In between are three lines of text. The first line contains the title and the artist. The title is the name of the HPR episode, or at least part of it. The artist is the HPR contributor who made that episode. 37 The second line contains the album name, the year, and the genre. The album name is this case is simply Hacker Public Radio, as the concept of an album doesn't really fit a podcast. The year is the year that the episode was recorded, or at least the year in which the MP3 file was assembled with the HPR intro and the tags added. 38 The genre is listed as "unknown 186". Recall that with ID3v1 tags there is a one byte numeric genre code, but that no genres above 70 were ever officially assigned. It seems to be a general convention though to use a code 186 for podcasts. 39 The third line contain a comment and track number. The comment in this case is the HPR web site URL. The track is zero. "Track" would appear to serve no useful purpose in this instance. However, it may be there for reasons of compatibility that I am unaware of, so I would be very reluctant to remove that without very good reason. 40 If we look at the above information in detail we can see that two of the tags appear to have cut their text information off short. The title is cut off in mid word after the 30th character. The final "g" in "hackerpublicradio.org" is cut off in the comment. 41 Alternative Method for ID3v1 We can confirm whether the text being cut short is due to a problem with the id3v2 program, or whether it really represents the data in the file by using a rather simple check. 42 Recall that ID3v1 tags are simply the last 128 bytes of the MP3 file. All we need to do is to extract the last 128 bytes of the file. We can do this using the standard tail command. tail -c128 hpr4678.mp3 | tr '' ' ' | tr -c '[:print:]' 'x' 43 The -c128 option used with tail tells it to extract the last 128 bytes of the file. We then pass the result through the "tr" command and tell it to replace null bytes with new line characters. Then we replace any remaining non-printable characters with an 'x'. When we do that we get the following 44 TAGHigh Resolution Elapsed Time iWhiskeyjack Hacker Public Radio 2026https://hackerpublicradio.or x 45 The first three characters are capital TAG. This is a flag which indicates that what follows are ID3 tags. 46 Next, we have 30 characters which specify the title. The next 30 characters specify the artist. The next 30 characters are the album, or in this case just "Hacker Public Radio". The next 4 characters are the year. The next 30 characters are a comment, or in this case the HPR URL, except for the final "g". The last character is the genre code, which we have replaced with an "x" because it is otherwise non-printable. 47 Taken together, these add up to 128 bytes. We can see that the field lengths are of fixed length with pre-defined meanings based on position. -------------------- 48 ID3v2 Using ffprobe Now lets move on to ID3v2 tags, which is probably more useful for most people. For this, we will switch to using ffprobe. The command for this is ffprobe -hide_banner hpr4678.mp3 The -hide_banner option suppresses extra data about the codecs which doesn't interest us much and leaves mainly the tag information plus a few other things. 49 The output gives us the full data that is associated with the podcast episode from the HPR web site. This includes the episode number, year, full title, author, license, and full summary text. You can see a full copy of this in the show notes. Input #0, mp3, from 'hpr4678.mp3': Metadata: track : 4678 year : 2026 title : High Resolution Elapsed Time in Shell Scripts author : Whiskeyjack copyright : CC-BY-SA artist : Whiskeyjack album : Hacker Public Radio comment : https://hackerpublicradio.org Clean; Surprises encountered when measuring elapsed time in shell scripts The license is CC-BY-SA genre : Podcast encoder : Lavf61.7.103 date : 2026 Duration: 00:30:10.18, start: 0.023021, bitrate: 64 kb/s Stream #0:0: Audio: mp3, 48000 Hz, mono, fltp, 64 kb/s 50 The ID3v2 tag version contains all of the information which was provided by the author, including the full title and description without the 30 character limit of ID3V1. If you want any of this information for some reason you should be able to extract it from the MP3 file using a combination of ffprobe, grep, and cut rather than trying to scrape the HPR web site and matching it to the MP3 later. -------------------- 51 ID3v1 Using ffprobe I previously mentioned that HPR MP3 files have both ID3v1 and ID3v2 tags in the same file, and that ffprobe will default to using ID3v2 tags if present. However, what happens if we remove the ID3v2 tags and leave the ID3v1 tags? I will describe how to strip tags later on in this episode, but let's just assume for now that I have done this. If we then use ffprobe to read the ID3 tags using the same command as before, we get the following output. 52 We get a series of lines in the same format as with ID3v2, but with each data element limited to at most 30 bytes. These include title, artist, album, date, comment, and genre. You can see the full output in the show notes. Input #0, mp3, from 'test.mp3': Metadata: title : High Resolution Elapsed Time i artist : Whiskeyjack album : Hacker Public Radio date : 2026 comment : https://hackerpublicradio.or genre : Podcast Duration: 00:30:10.18, start: 0.023021, bitrate: 64 kb/s Stream #0:0: Audio: mp3, 48000 Hz, mono, fltp, 64 kb/s 53 The information is the same as when read by the id3v2 program, but formatted for display in the manner that ffprobe uses. This shows that ffprobe can indeed read ID3v1 tags if they are the only ones present. -------------------- 54 Reading Vorbis Comments Now let us turn our attention to vorbis comments, which are the equivalent to tags for ogg files. We will look at this using two methods. 55 Using vorbiscomment The first method we will look at is using the vorbiscomment package. The command is vorbiscomment -l hpr4678.ogg 56 The output of this is a series of lines with key value pairs separated by equal signs. The output is also exactly the same as the MP3 file, except that there is an additional "language" field, "track" becomes "TRACKNUMBER", and there is no "date" field. You can see the full output in the show notes. encoder=Lavc61.19.101 libvorbis TRACKNUMBER=4678 year=2026 language=English title=High Resolution Elapsed Time in Shell Scripts author=Whiskeyjack copyright=CC-BY-SA artist=Whiskeyjack album=Hacker Public Radio DESCRIPTION=https://hackerpublicradio.org Clean; Surprises encountered when measuring elapsed time in shell scripts The license is CC-BY-SA genre=Podcast 57 Using ffprobe Now lets do the same again using ffprobe. The command for this is ffprobe -hide_banner hpr4678.ogg 58 The output content is the same of course, but the field names have all been forced to lower case, and instead of an equal sign as a separator between the key and value, this has been replaced by a colon and white space has been added to make the output look a bit nicer. You can see the full output in the show notes. Input #0, ogg, from 'hpr4678.ogg': Duration: 00:30:10.14, start: 0.000000, bitrate: 86 kb/s Stream #0:0(English): Audio: vorbis, 192000 Hz, mono, fltp, 4294967 kb/s Metadata: encoder : Lavc61.19.101 libvorbis track : 4678 year : 2026 title : High Resolution Elapsed Time in Shell Scripts author : Whiskeyjack copyright : CC-BY-SA artist : Whiskeyjack album : Hacker Public Radio comment : https://hackerpublicradio.org Clean; Surprises encountered when measuring elapsed time in shell scripts The license is CC-BY-SA genre : Podcast -------------------- 59 Another Example from Another Podcast The above is interesting, but it's a sample of one podcast. Let's look at another different one altogether. For this test I used two episodes of the Linux Matters podcast, episodes 68 and 82. As to why I am using two different episodes I will explain in a moment. 60 Episode 68 We will start with examining episode 68 Using id3v2 The command using id3v2 is id3v2 -l LMP68.mp3 61 This provides output as a series of lines containing the official 4 character identifiers from the standard, a description of the identifiers, and the text provided by the authors. The identifiers include TIT2 indicating title, TALB indicating show title, TRCK indicating track number, and a number of others. You can see the full output in the show notes. id3v2 tag info for LMP68.mp3: TIT2 (Title/songname/content description): 68: Frameworks, Filesystems and Fixes TPE1 (Lead performer(s)/Soloist(s)): Linux Matters TALB (Album/Movie/Show title): Linux Matters TYER (Year): 2025 TRCK (Track number/Position in set): 68 COMM (Comments): ()[]: https://linuxmatters.sh/ APIC (Attached picture): (LMP-3000-moon.jpg)[, 3]: image/jpeg, 554576 bytes LMP68.mp3: No ID3v1 tag 62 From this we can see what the id3v2 program would normally do with ID3v2 tags. Note that it outputs the actual 4 character identifiers, plus a description of what they mean, and then the actual data. This helps when trying to understand the actual encoding of the data. 63 Using ffprobe Now let's try that with ffprobe. The command is ffprobe -hide_banner LMP68.mp3 64 This provides the same publisher provided data as before. However it does not display the 4 character identifiers but instead uses its own format for display. Also note in both cases that there is a picture embedded in the MP3 file which is used to generate an icon for display in your file manager. With ffprobe this results in there being two keys called "title" and two keys called "comment". This makes grepping for the metadata more complicated, but it should still be possible. You can see the full output in the show notes. Input #0, mp3, from 'LMP68.mp3': Metadata: title : 68: Frameworks, Filesystems and Fixes artist : Linux Matters album : Linux Matters comment : https://linuxmatters.sh/ track : 68 date : 2025 Duration: 00:28:12.45, start: 0.025056, bitrate: 114 kb/s Stream #0:0: Audio: mp3, 44100 Hz, mono, fltp, 112 kb/s Metadata: encoder : LAME3.100 Stream #0:1: Video: mjpeg (Progressive), yuvj444p(pc, bt470bg/unknown/unknown), 4166x4166 [SAR 72:72 DAR 1:1], 90k tbr, 90k tbn (attached pic) Metadata: title : LMP-3000-moon.jpg comment : Cover (front) 65 Episode 82 Now lets try that again with a different episode, 82. Using id3v2 The command for id3v2 is id3v2 -l LMP82.mp3 This results in id3v2 saying LMP82.mp3: No ID3 tag What happened here? 66 Using ffprobe Let's try that again with ffprobe. The command ffprobe -hide_banner LMP82.mp3 provides the expected output. Input #0, mp3, from 'LMP82.mp3': Metadata: date : 2026-05 title : 82: Ditching Grammarly for Open Sauce album : Linux Matters track : 82 artist : Linux Matters comment : https://linuxmatters.sh Duration: 00:33:37.83, start: 0.025056, bitrate: 113 kb/s Stream #0:0: Audio: mp3, 44100 Hz, mono, fltp, 112 kb/s Stream #0:1: Video: png, rgb24(pc, gbr/unknown/unknown), 3000x3000, 90k tbr, 90k tbn (attached pic) Metadata: title : Linux Matters Logo comment : Cover (front) 67 This results in a few minor changes from episode 68, but otherwise it looks the same. So there are ID3 tags, but for some reason id3v2 couldn't recognize them. A bit more research and experimentation shows that this change appears to have happened right after episode 68, when they changed hosting and processing arrangements. I suspect that something changed with respect to the ID3v2 tag formatting somewhere along the way in the change over, and this in turn has affected the ability of the id3v2 program to recognize the tags. I will come back to the implications of this later in my conclusions. -------------------- 68 Stripping ID3 Tags Now let's turn to a differen topic. Stripping ID3 tags refers to removing the ID3 tags from the audio file. I will start off by emphasizing that normally, you don't want to do this. You should only be doing it if you suspect the ID3 tags are causing a problem with the playback or ordering of the files. 69 In my case I do it when I have problems with my MP3 player when playing certain podcasts. This MP3 player orders files according to ID3 tags rather than by file name. This can result in the podcasts being played in an unpredictable order which I find undesirable. This is particularly a problem with podcasts from certain publishers where the title data does not follow any consistent pattern, but is whatever someone felt like doing that day. 70 I also often have to normalize the files from the same publishers to get a consistent audio loudness. See my series on Simple Podcasting for information on how to use ffmpeg to normalize the audio loudness. The solution to the inconsistent tag formats in these cases is to simply strip the ID3 tags altogether. The player then falls back on using the file names, and I can readily rename the files to a consistent format. I have never had these sorts of problems with HPR podcasts. If you are not having any problems of this nature, then as I said, don't bother stripping the tags. 71 To strip the ID3 tags from an MP3 file use the following. id3v2 -D hpr4678.mp3 id3v2 will strip the tags and overwrite the existing file. If you wish to keep a copy with the tags, be sure to keep a backup before you try things. 72 When it comes to stripping tags, the options are "-s" deletes ID3v1 tags. "-d" deletes ID3v2 tags. "-D" deletes both v1 and v2 tags. -------------------- 73 Conclusions In this episode we took a very brief look at ID3 tags and vorbis comments and ways of reading them. 74 ID3 tags and Vorbis comments provide a means of allowing information about an MP3 or OGG Vorbis file to be embedded in the file itself. Podcast publishers very often use this to label the file with information such as title, publisher, and author. We can read this information using Fee Software tools such as ffprobe, ID3v2, and vorbiscomment. 75 ID3 tags seem to be a mess with more than one incompatible versions, and difficulties in reading them even within the same version. They are difficult to test for because there is so much hardware out there of varying ages, much of which you will never have heard of let alone had access to. If you are recording episodes for HPR you do not have to worry about this, as HPR will do this behind the scenes for you. However, if you are responsible for producing a podcast or other similar audio and you have a setup that works, it is probably best not to change anything without good reason. 76 Vorbis comments seem to be much less of a problem. However, there are far fewer devices which can play OGG files compared to MP3, so simply switching to OGG may not be a realistic solution to the ID3 tag problem. 77 If you wish to have one tool that can read ID3 tags of all sorts and vorbis comments, then ffprobe is your obvious choice. 78 The ID3v2 program will provide more information about the ID3 tags, including the actual identifiers used. However, it does not work in all cases. 79 The vorbiscomment program will read Vorbis comments from OGG files in a manner which is closer to the actual vorbis format than ffprobe does, which uses its own display format. 80 HPR includes both ID3v1 and ID3v2 tags in its MP3 files. ffprobe can be used to read the ID3v2 tags, and the id3v2 program can be used to read the ID3v1 tags. You can also read the ID3v1 tags using the tail command. 81 I have barely scratched the surface of this subject and have not talked at all about creating tags or comments. If anyone else would like to take up the challenge of providing more detail, or of correcting any mistakes that have made, please send in a podcast episode on the subject. If you have any comments you would like to make, leave them in the comment section below this episode on the HPR web site. 82 I hope to see you all again in future in another episode of HPR. -------------------- References https://id3.org/Introduction https://www.xiph.org/ https://wiki.xiph.org/VorbisComment -------------------- Provide feedback on this episode.

This show has been flagged as Clean by the host. I recently had an experience where UNIX tools proved very useful. A relative had an old mobile phone running Android that stopped connecting to the carrier's network and bought a new one to replace it. I took on the job of trying to copy their files (consisting of just photos and videos) off of the old phone. Google's software was desperate to convince me to upload everything to the cloud, but I wasn't interested. It offered the option of copying the files over to an SD card, but failed on repeated attempts to do that. The option I tried next was to transfer them to another device via Bluetooth—that one did actually work, although it was slow and would only handle sending about 100 files at a time. They came over to my laptop OK, but the problem with that method was that all of the file times were set to the time when they were transferred. I'm not super familiar with how mobile apps manage metadata, but would presume that they look to file times for organizing photos by date. Fortunately, the names of each of the files included the date and time they were created. I recognized that I could write a bit of shell script to parse the filenames and set the file times accordingly. While there were over 800 files, the good news is that there were only three different categories of filenames, so the logic to extract the information needed was relatively simple. Each file had eight numerical digits representing the date and six digits representing the time. It would definitely be an option to come up with a more sophisticated parser that could handle a wide variety of filenames, but I went the lazy way and just handled those three cases. Another nice aspect was that none of the filenames contained spaces, which allowed me to be a bit less careful when using them in command lines. I didn't need to worry about time zones because my laptop was set to the same time zone as the phone—also, if a time was off a by a few hours it wouldn't make a practical difference. Examples of the three different types of filenames I had to deal with, labeled with the relevant values: YYYY=year, MM=month, DD=day, hh=hour, mm=minute, and SS=second. 00001IMG_00001_BURST20250525140124.jpg YYYYMMDDhhmmSS IMG_20220223_124023.jpg VID_20221017_095024.mp4 YYYYMMDD hhmmSS 20191224_195939.jpg 20161021_122620-1.jpg 20191130_134317_Burst01.jpg 20200129_223612_010.jpg YYYYMMDD hhmmSS I considered awk as an option (see Whiskeyjack's comment on HPR episode 4657 ), but realized it has no built-in way to change file times, so I set it aside. Don't worry, I will come back to that later. My approach was to use an if-then shell construct to choose how to treat the three categories of filenames. For the if condition, I fed the filename into the grep -q command with an appropriate regular expression to test whether it matches. The -q option to grep causes it not to output anything—it returns a zero exit status if there's a match and a status greater than zero if there isn't. Then, there is an elif statement with another grep -q test for the second category of filenames. Finally, an else statement is followed by the command to run for all other filenames. The whole thing is wrapped in a for loop that runs over all the files in the current directory. The touch command , when used with the -t option, can be given a string consisting of the year, month, day, hour, minute, and second. These are all numerals that are run together, except that a period sits between the minute and second. So we need a way to extract these numbers and to insert the period. That's where the cut utility comes in. It can be given a set of characters to select, and I specified a different set representing the appropriate ones depending on which category a filename fit into. To insert the period, I used sed to replace the last two characters with a period followed by those characters. The first script was to test out that I was getting the correct results. for fn in * do if echo "$fn" | grep -q BURST then printf "$fn " echo $fn | cut -c '21-34' | sed 's/..$/.&/' elif echo "$fn" | grep -q -E '^(IMG_|VID_)' then printf "$fn " echo $fn | cut -c '5-12,14-19' | sed 's/..$/.&/' else printf "$fn " echo $fn | cut -c '1-8,10-15' | sed 's/..$/.&/' fi done This one actually sets the file times. The -c option to touch prevents it from creating a file if one with that name doesn't already exist. for fn in * do if echo "$fn" | grep -q BURST then touch -c -t "$(echo $fn | cut -c '21-34' | sed 's/..$/.&/')" "$fn" elif echo "$fn" | grep -q -E '^(IMG_|VID_)' then touch -c -t "$(echo $fn | cut -c '5-12,14-19' | sed 's/..$/.&/')" "$fn" else touch -c -t "$(echo $fn | cut -c '1-8,10-15' | sed 's/..$/.&/')" "$fn" fi done The script ran over all the files in less than 15 seconds and correctly set the file time on each. Job done, right? Well, after I did this, it struck me that there was room for improvement. The script would probably run more quickly if I used a case construct instead of an if construct that called grep multiple times. While the pattern-matching notation used with case is not as flexible and can handle fewer situations than the regular expression syntax available with grep , in this case (see what I did there?) it is sufficient. Testing it out, using case reduced the running time by 45%. Replacing if with case —the commands to be executed for each category of filename can remain exactly the same. for fn in * do case "$fn" in *BURST*) printf "$fn " echo $fn | cut -c '21-34' | sed 's/..$/.&/' ;; IMG_*|VID_*) printf "$fn " echo $fn | cut -c '5-12,14-19' | sed 's/..$/.&/' ;; *) printf "$fn " echo $fn | cut -c '1-8,10-15' | sed 's/..$/.&/' esac done for fn in * do case "$fn" in *BURST*) touch -c -t "$(echo $fn | cut -c '21-34' | sed 's/..$/.&/')" "$fn" ;; IMG_*|VID_*) touch -c -t "$(echo $fn | cut -c '5-12,14-19' | sed 's/..$/.&/')" "$fn" ;; *) touch -c -t "$(echo $fn | cut -c '1-8,10-15' | sed 's/..$/.&/')" "$fn" esac done I couldn't completely put awk out of my mind, though, and I eventually came up with an awk script for the same purpose. This is far faster, probably because everything can be done within awk except actually modifying the file times, which is possible using the system() function to call touch . I was able to knock 90% off the running time, which for 800 files isn't a big deal but might make a difference if you have hundreds of thousands of files. The awk counterparts to both scripts above. Unlike those, ls is used to feed it with the list of filenames. We have the full power of extended regular expressions available to use for matching against the filenames. The next statement causes awk to skip any remaining pattern-action pairs and go to the next line of input. ls | awk '/BURST/ { print $0, substr($0, 21, 12) "." substr($0, 33, 2) next } /^(IMG_|VID_)/ { print $0, substr($0, 5, 8) substr($0, 14, 4) "." substr($0, 18, 2) next } { print $0, substr($0, 1, 8) substr($0, 10, 4) "." substr($0, 14, 2) }' ls | awk '/BURST/ { system("touch -c -t " substr($0, 21, 12) "." substr($0, 33, 2) " " $0) next } /^(IMG_|VID_)/ { system("touch -c -t " substr($0, 5, 8) substr($0, 14, 4) "." substr($0, 18, 2) " " $0) next } { system("touch -c -t " substr($0, 1, 8) substr($0, 10, 4) "." substr($0, 14, 2) " " $0) }' A further optimization that came to me later was to not call system() from within awk , but to instead just have awk print out a set of command lines. These can then be piped to sh to actually be executed. This cut the running time down by 95% compared to my original script. The fastest version I was able to come up with. If you run it without the | sh on the end, you can check that it's outputting the right information before actually modifying anything. The backslash on the end of a couple lines causes the subsequent line to be treated as a continuation of the existing line. Normally I would just keep everything on one line even if it runs longer than 80 columns, but for display purposes this looks nicer. ls | awk '/BURST/ { print "touch -c -t " substr($0, 21, 12) "." substr($0, 33, 2) " " $0 next } /^(IMG_|VID_)/ { print "touch -c -t " substr($0, 5, 8) substr($0, 14, 4) "." substr($0, 18, 2) " " $0 next } { print "touch -c -t " substr($0, 1, 8) substr($0, 10, 4) "." substr($0, 14, 2) " " $0 }' | sh It is probably true that this could have been carried out just as easily on Windows using Microsoft's PowerShell. I'm not very familiar with it, but would imagine (or hope) that it includes commands for managing these basic things like text manipulation and modifying file times. If you are stuck in an environment where you don't have a UNIX-like system available, investigate how to accomplish a task with the tools you do have. While I had the necessary information in the filenames to use, that might not be the case in all situations. You could look for other sources of dates—most digital cameras will add EXIF tags to a JPEG file giving the date and time it was created. (Hopefully, the clock in the camera will be set accurately.) While there is no standard UNIX utility to read those tags, free and open source software tools are widely available for that purpose. I found one called exiftags that included the utility exiftime , which specifically outputs EXIF data relating to time. The output format was a little trickier to handle, but awk was able to manage it with a little coaxing. Example of output produced by exiftime . Note that the first line with the filename is only printed if more than one filename is given as an argument. Also, for amusing-sign.jpg , apparently I edited that photo after taking it and the editing software updated the "created" tag but left the others intact. Not all images will necessarily have created, generated, and digitized tags; we will just take whichever ones exist. I redirected standard error to /dev/null to get rid of error messages for files that don't have EXIF tags; we'll handle those below. $ exiftime *.jpg 2>/dev/null 20260508_154743.jpg: Image Created: 2026:05:08 15:47:43 Image Generated: 2026:05:08 15:47:43 Image Digitized: 2026:05:08 15:47:43 20260508_155044.jpg: Image Created: 2026:05:08 15:50:44 Image Generated: 2026:05:08 15:50:44 Image Digitized: 2026:05:08 15:50:44 3704a78e771c2a25a894ef2f0b5a2a629f1eba80.jpg: amusing-sign.jpg: Image Created: 2017:01:24 23:14:04 Image Generated: 2017:01:24 21:18:07 Image Digitized: 2017:01:24 21:18:07 dscf3011.jpg: Image Created: 2015:01:01 00:02:19 Image Generated: 2015:01:01 00:02:19 Image Digitized: 2015:01:01 00:02:19 window-view.jpg: $ We can take advantage of the fact that different records are separated by a blank line. In awk , when RS is set to a null string and FS is set to a newline character, each set of non-blank lines is treated as a record and each line within those sets is treated as a field. One or more blank lines separate each record. For the output of exiftime , this means that $1 will contain the filename and $2 will contain the first line after the filename. For those files without an EXIF date tag, $2 will be a null string, which is treated by awk as FALSE, so the pattern will not match, the action will not be taken, and nothing will be printed. If a file has multiple tags, I will just use the first one reported by exiftime (contained in $2 ). The sub() function call removes the colon that exiftime prints after the filename, and the gsub() function call removes all non-numeric characters from the date and time in the tag. (After a comma within a print statement, a backslash is not necessary to continue a line.) Also, this time I bothered to print quotation marks around the filename in case it contains spaces. $ exiftime *.jpg 2>/dev/null | awk 'BEGIN { FS = "n" ; RS = "" } $2 { sub(":$", "", $1) gsub("[^0-9]", "", $2) print "touch -c -t", substr($2, 1, 12) "." substr($2, 13, 2), """ $1 """ }' touch -c -t 202605081547.43 "20260508_154743.jpg" touch -c -t 202605081550.44 "20260508_155044.jpg" touch -c -t 201701242314.04 "amusing-sign.jpg" touch -c -t 201501010002.19 "dscf3011.jpg" $ I would imagine that there's some photo management program out there that I could have used to accomplish this. But then I would have had to locate it, verify that it wasn't some malware-loaded garbage, download, and install it. And chances are it would want to take over all the photos on my laptop. Instead, with standard UNIX tools and shell capabilities like if , case , process substitution, and pipelines, I was able to complete the task without having to install anything. The techniques I described can be used in different circumstances and with the output of different utilities. My intention was not just to explain how to solve this specific problem, but to hopefully teach you some things that you can apply in many situations. Perhaps if you use them to tackle a challenge of your own, you'll record an episode for HPR to share what you know. Provide feedback on this episode.

This show has been flagged as Explicit by the host. New hosts There were no new hosts this month. Last Month's Shows Id Day Date Title Host 4673 Wed 2026-07-01 First contact conversation Archer72 4674 Thu 2026-07-02 Audiobooks Ahuka 4675 Fri 2026-07-03 Yard Inflatables operat0r 4676 Mon 2026-07-06 HPR Community News for June 2026 HPR Volunteers 4677 Tue 2026-07-07 UNIX Curio #10 - Checksums and Hashes Vance 4678 Wed 2026-07-08 High Resolution Elapsed Time in Shell Scripts Whiskeyjack 4679 Thu 2026-07-09 HPR Beer Garden 15 - Double IPA Kevie 4680 Fri 2026-07-10 Robert A. Heinlein: The Future History, Part 2 Ahuka 4681 Mon 2026-07-13 My Disabilities Antoine 4682 Tue 2026-07-14 Behind the Keyboard: A Cybersecurity Operator's Real-World Workflow operat0r 4683 Wed 2026-07-15 Recording the hallway track Ken Fallon 4684 Thu 2026-07-16 Sim Racing on the cheap! operat0r 4685 Fri 2026-07-17 Listening to SSB stations in the early 1980s Lennart Benschop 4686 Mon 2026-07-20 Debugging Security Cameras: Firmware Updates, Python Scripts and Windows Workarounds operat0r 4687 Tue 2026-07-21 UNIX Curio #11 - Merging Files Vance 4688 Wed 2026-07-22 Downloading Podcasts with a Shell Script Whiskeyjack 4689 Thu 2026-07-23 Cheap Yellow Display Project Part 8: Writing the code Trey 4690 Fri 2026-07-24 Playing Civilization V, Part 14 Ahuka 4691 Mon 2026-07-27 Viva la Coda Lee 4692 Tue 2026-07-28 Noise Music Tutorial 2: Using Audacity to Make Noise TheDUDE 4693 Wed 2026-07-29 Amateur Radio Field Days Archer72 4694 Thu 2026-07-30 HPR Beer Garden 16 - Belgian Blonde Kevie 4695 Fri 2026-07-31 Try not to buy a phone operat0r Comments this month Past shows hpr4644 (2026-05-21) "Response to comments on HPR4424: Newsboat..." by Archer72. Archer72 said: "Ken on Community Show HPR4676" (2026-07-06 13:48:39) Ken Fallon said: "hpr3962 :: It's your data" (2026-07-06 15:07:47) hpr4669 (2026-06-25) "HPR Beer Garden 14 - Super Strong Lager" by Kevie. The_Dud3 said: "Favorite Malt Liquores/Super Strong Lagers" (2026-07-22 02:07:59) hpr4672 (2026-06-30) "Hey Mum, I'm on Spotify ! " by Ken Fallon. Archer72 said: "Another great show!" (2026-07-08 09:46:44) This month's shows hpr4674 (2026-07-02) "Audiobooks" by Ahuka. The Librarian said: "Ook ?" (2026-07-02 10:50:38) hpr4677 (2026-07-07) "UNIX Curio #10 - Checksums and Hashes" by Vance. xmanmonk said: "Another great show" (2026-07-07 21:17:41) candycanearter07 said: "cool show :D" (2026-07-08 11:32:04) Vance said: "Thanks, and systemd as a future topic" (2026-07-09 01:50:23) Vance said: "Sorry, xmanmonk" (2026-07-12 03:20:02) hpr4678 (2026-07-08) "High Resolution Elapsed Time in Shell Scripts" by Whiskeyjack. candycanearter07 said: "cool ep" (2026-07-09 14:38:32) Whiskeyjack said: "Reply to candycanearter07 on HPR4678" (2026-07-09 22:48:47) hpr4681 (2026-07-13) "My Disabilities" by Antoine. Archer72 said: "Reading" (2026-07-22 13:43:46) hpr4684 (2026-07-16) "Sim Racing on the cheap!" by operat0r. Jim DeVore said: "Great episode!" (2026-07-28 03:43:23) hpr4685 (2026-07-17) "Listening to SSB stations in the early 1980s" by Lennart Benschop. Lucinda said: "Thank you" (2026-07-21 10:53:42) hpr4688 (2026-07-22) "Downloading Podcasts with a Shell Script" by Whiskeyjack. candycanearter07 said: "cool solution" (2026-07-22 23:14:22) Whiskeyjack said: "Response to candycanearter07 in HPR4688" (2026-07-23 19:14:55) Mailing List discussions Policy decisions surrounding HPR are taken by the community as a whole. This discussion takes place on the Mailing List which is open to all HPR listeners and contributors. The discussions are open and available on the HPR server under Mailman. The threaded discussions this month can be found here: https://lists.hackerpublicradio.com/pipermail/hpr/2026-July/thread.html Events Calendar With the kind permission of LWN.net we are linking to The LWN.net Community Calendar. Quoting the site: This is the LWN.net community event calendar, where we track events of interest to people using and developing Linux and free software. Clicking on individual events will take you to the appropriate web page. Provide feedback on this episode.

This show has been flagged as Clean by the host. IDEAS: Using a cheap phone plan with SMS for authentication. Mighty Text as a free SMS solution. Google VoIP number for shared accounts. Issues with SMS verification blocking VoIP numbers. Avoiding carrier-specific number blocks. Multiple SIM cards for cost-effective SMS. Rooting a phone to manage apps. Security concerns with third-party apps. Limited data usage for minimal phone plans. Challenges with app compatibility on rooted devices. Short-term phone solutions for SMS needs. Shared Google accounts for streamlined access. Avoiding premium SMS services like $15/month plans. Using Wi-Fi for data instead of cellular plans. Importance of SMS for MFA (multi-factor authentication). Transitioning from old phones to new setups. Balancing convenience and cost in phone plans. Reliance on SMS for banking and insurance access. Difficulty finding non-blocked SMS verification options. Preference for minimal, low-cost phone solutions. RECOMMENDATIONS: Use a shared Google account for SMS access. Opt for a cheap phone plan with unlimited texting. Try Mighty Text as a free SMS alternative. Avoid premium SMS services with high fees. Use Wi-Fi instead of cellular data for minimal plans. Choose carrier numbers over VoIP for critical services. Root a device to manage app settings. Test SMS compatibility with banks and providers. Consider multiple SIM cards for redundancy. Prioritize SMS for MFA over other verification methods. Monitor app updates for compatibility with rooted devices. Select phones with flexible data plans. Use downloaded content instead of streaming. Check for SMS blockages with new services. Explore low-cost phone options for minimal use. Maintain backup SMS methods for emergencies. Simplify phone setups to reduce costs. Verify SMS support before switching providers. Combine Wi-Fi and SMS for reliable connectivity. Share accounts to streamline digital access. Provide feedback on this episode.

This show has been flagged as Clean by the host. Dave and Kevie are back with another HPR Beer Garden and this time they turn their attention to Belgian Blonde Ales. Kevie samples Leffe Blonde , whilst Dave opts for La Chouffe . Connect with the guys on Untappd: Dave Kevie The intro sounds for the show are used from: https://freesound.org/people/mixtus/sounds/329806/ https://freesound.org/people/j1987/sounds/123003/ https://freesound.org/people/greatsoundstube/sounds/628437/ The next 3 beer styles to be reviewed: DDH IPA Amber Ale Lager Provide feedback on this episode.

This show has been flagged as Clean by the host. Hi, this is Archer72 for another episode of Hacker Public Radio. In this episode, the ARRL Field Days was in the past month, so I thought this would be a good time to highlight events in the US as well as around the world. Field Day (amateur radio : edited on 4 June 2026, at 13:51) Experience the power of ham radio at 2025 ARRL Field Day Harrison County Amateur Radio Club Jun 12, 2025 Updated Jun 17, 2025 Amateur Radio Club to host annual field day By Keith Clifford Harrison County Amateur Radio Club Jun 15, 2026 Harrison County Amateur Radio Club Field Day Jun 22, 2026 Preceding collage used with permission by Keith Clifford of the Harrison County Amateur Radio Club (K4HSN) Provide feedback on this episode.

This show has been flagged as Explicit by the host. Links to a more in depth way to do the musique concrete method and harsh noise wall (they don't use audacity, but it all can be done on audacity, just gives you an idea what I'm talking about) visual of generating noise on audacity (for harsh noise, you want to have a larger amplitude or any thing that makes the noise louder) https://youtu.be/CkZS1C7JUmM?si=qGe7g94lrBE-fAh6 Musique Concrete https://youtu.be/hvI0URwiCJc?si=l_7N3DASLu4Cw7qQ Harsh Noise Wall https://youtu.be/_zwgleyfBgQ?si=IgtK3ydM0queuHBX Provide feedback on this episode.

This show has been flagged as Clean by the host. C++ is a high-level, general-purpose programming language created by Danish computer scientist Bjarne Stroustrup. First released in 1985 as an extension of the C programming language, adding object-oriented (OOP) features, it has since expanded significantly over time adding more OOP and other features - from https://en.wikipedia.org/wiki/Main_Page Serenity OS - https://serenityos.org The C Programming Language, Kernighan and Ritchie - https://colorcomputerarchive.com/repo/Documents/Books/The%20C%20Programming%20Language%20%28Kernighan%20Ritchie%29.pdf C++ - https://en.wikipedia.org/wiki/C%2B%2B The Ghost of Unix SVR4 - https://github.com/macsplit/unix_docProvide feedback on this episode.

This show has been flagged as Clean by the host. In our sample game we look at playing as Austria and aiming for a Diplomatic Victory. And our focus is on puppeting City-States, but I misunderstood and instead of making a Diplomatic Victory easier, it makes it harder. I still managed to get my Diplomatic Victory, but a Science or Domination Victory would definitely have been easier in this scenario. Playing Civilization V, Part 14 Now I am at turn 340, I just puppeted another City-State, and I have one more on the way. I have sent a large number of military units, both land and sea, to the City-States that are on Japan's border. If he tries anything now he will likely regret it. Now I am starting to build things I had not bothered with before, like Granaries and Amphitheaters. I got a couple of Great Engineers, and used them to rush build Wonders. I am nearly finished puppeting all of the City-States, and my Happiness and Income are both quite healthy. I am in such good shape I started to build the World's Fair, which I initally proposed then ignored. At this point I only have one more City-State to puppet, then I will have puppeted every City-State in the game. So it is time to put my large income to use in other ways by giving gifts to the other players. I may want to get a few more votes on my side to get over the top in the World Leader vote when it happens. On turn 357 I completed the puppeting of all of the City-States in the game. Meanwhile, I am building up my Science and building Wonders as they become available. I didn't build any in the early game because I had other priorities, but now I have a strong military, high income, and no other pressing needs. My score on the Leader Board is now double that of my nearest competitor which is still Japan. And their denunciation me of just expired with them taking no action. I used a Great Engineer to rush build the Porcelain Tower, earning me another Great Scientist, which I then used to build another Academy. At this point it is mostly a slog to the finish. Since I was so dominant and so wealthy I could afford to pursue a couple of other options just in case. One was a Science victory, which looked like a possibility, though it would be close. I had to be very focused in my Science Research. The other option was a Culture victory, and I had dominated two other empires culturally without half trying, and I was generating Great Artists, Great Musicians, and Great Writers pretty rapidly. So I built Museums, then Archeologists, and grabbed every artifact I find to stuff into the Museums. But the final Victory came when I researched Globalization. This tech allows you to get an extra vote in a World Council for every Spy you send to another empire's capital and designate as an Ambassador. I immediately sent my spies out (they had previously been in my cities doing counter-espionage since I had a tech lead). And a few turns later a vote for World Leader came up and I won. This was my first time using Maria Theresa of Austria as my leader, and it was fun, but in the end puppeting all of the City-Staters actually made it harder to win a Diplomatic victory. In other games I have played, every City-State that was an ally was an extra vote in the World Council, and I assumed that making them puppets would mean I would get that many votes. But that didn't happen. Of course, if I had read more carefully in the Civilization Wiki I would have realized this, because it says there very clearly that Austria is one of the worst Empires to pick if you are going for a Diplomatic victory. So the fact that I managed it anyway is nice. I think I actually could have done a Science victory as I had my Spaceship half-built when I won the Diplomatic Victory. But I hope this analysis of how I actually played this game will be helpful to someone else. Links https://www.palain.com/gaming/civilization-v/playing-civilization-v-part-14/ Provide feedback on this episode.

This show has been flagged as Clean by the host. Hello, again. This is Trey. Welcome to part 8 in my Cheap Yellow Display (CYD) Project series. If you wish to catch up on earlier episodes, you can find them on my HPR profile page https://www.hackerpublicradio.org/correspondents/0394.html It is hard to believe that I started this project and the HPR series to document it more than a year ago. Time flies. Life happens. I spent the last 8 months so focused on work related activities that I had to set the project aside. And once I set it aside, it was difficult to get back to again. The one time I tried, I found that my son's old Windows laptop, which I had commandeered to use for the project, was once and truly dead. We live in a different world now than we did when I began this project. Today, everything is about AI – how it is changing our world, increasing efficiencies, and even displacing certain types of jobs. "Vibe coding" is transforming the way we make software, and now everyone is a developer. Within my organization, we are all being strongly encouraged to learn more about AI and apply it in our daily work. We are blessed to have access to a wide range of training and to powerful tools which support the process. Several colleagues within my organization and outside my organization have recommended Claude Code -- for development, for organization, for brainstorming, and for much more. My role is not that of a developer, and I have had no need for Claude Code at work. There are plenty of other tools for me to use. But at home, I thought... I could install Claude Code at home to experiment with and to learn. And then it hit me. I wonder if I could use Claude Code to help me with my stalled CYD project. "Hello, my name is Trey, and I am a fraud." OK. I don't think I am a fraud, but having never used such a powerful tool to help me code, I feel a little bit like a fraud, with Claude doing the work for me. Let's talk through what we did. As I mentioned, I was unable to use the laptop on which I created the original GUI code. But no worries, because It was all on GitHub, right? So, I began by trying to install Claude Code on one of my Ubuntu machines. That failed miserably, and all the instructions I found online recommended running it on Windows. So I transitioned to installing Claude Code on a freshly built Windows 10 system using some old hardware which will not support Windows 11. Surprisingly, It seems that Claude wants to run within Ubuntu on Windows Subsystem for Linux (WSL). Once I went through all the hoops for installing and setting up my basic subscription, I got to work. I created a new directory to use for all my project files, and another one for Claude to write all its output files to. Everything I have read and heard about using Claude Code for a project like this recommended beginning by using Claude to create the project plan. So my first prompt was as follows: "I am building a project using an ESP32 cheap yellow display. The final goal is to have a device with a graphical user interface which will allow me store several text messages, each assigned to their own specific button. Then, when one of the buttons is pressed, the message will be converted into morse code, and the message will be sent by blinking an onboard LED. I already have the initial design for the GUI coded in my github repository. Please construct a detailed step by step plan for building this application" I wanted to keep it simple at the beginning, so even though I plan to use an opto-isolated relay for the output, I am starting out using the built in LED. Claude asked me for the URL for my Github repository, and I provided it. After thinking for a bit, it produced a project plan. CYD Morse Sender — Step-by-Step Build Plan What You Already Have - 3 fully designed LVGL screens (Send, Config, Keyboard) - 7 send buttons with green styling on the Send Screen - Speed slider (5–30 WPM) and program slot dropdown on Config Screen - Text area + LVGL keyboard widget on Keyboard Screen - Display driver and touchscreen hardware initialized (but touch read was stubbed out) --- Phase 1: Fix Touchscreen Input Initialize the XPT2046 on VSPI, replace the hardcoded touched = false with real touch data, calibrate raw ADC coordinates to screen pixels, and verify all three nav buttons transition screens correctly. Phase 2: Persistent Message Storage Use Preferences.h / NVS to store 7 messages ("msg0" – "msg6") that survive power cycles. Load on boot and populate send button labels. Phase 3: Config Screen — Slot Selection & Speed Wire the Program dropdown to g_selectedSlot, wire the Speed slider to g_wpm with a live WPM label, and wire the Set button to clear the text area and navigate to the Keyboard Screen. Phase 4: Keyboard Screen — Capture and Save Text On keyboard OK (LV_EVENT_READY): read the text area, save to NVS, update the send button label, navigate back. On Cancel: navigate back without saving. Phase 5: Morse Code Engine Build morse.h with the full A–Z / 0–9 lookup table, a textToMorse() converter, and Paris-standard timing (unit_ms = 1200 / wpm). Phase 6: Non-Blocking LED Transmission State machine (IDLE / LED_ON / LED_OFF) driven by millis() in a morseUpdate() function called every loop — no delay(). LED on GPIO 17 (blue). Phase 7: Wire the Send Buttons Each button calls startMorse(getMessage(slot)) if not already transmitting. Disable all send buttons during TX; re-enable when the state machine reaches the end. Phase 8: Integration & Polish Stop button, WPM persistence, empty-slot guard, serial debug output, final touch calibration, end-to-end test checklist. --- Recommended order: Phase 1 → 2 → 7 (partial) → 5 → 6 → 7 (full) → 3 → 4 → 8 This was wild! Claude determined exactly how far along I was in the project, and also alerted me to the fact that the "touch read was stubbed out". Ugh. This means that the files I had uploaded to GitHub were my original GUI files (Episode 05 – HPR4532 - https://hackerpublicradio.org/eps/hpr4532/index.html ) and not the ones that I finally got working properly (Episode 07 – HPR4624 - https://hackerpublicradio.org/eps/hpr4624/index.html ). That was my own fault. Did I mention that I don't get Git? I REALLY need to learn to properly use Git! But, we have a plan, broken down by eight numbered phases. And they seem to address all the functionality I wanted with a few additional things I had not thought about. Interestingly, even though these phases are sequentially numbered, Claud recommended that we approach them in a bizarre order: Phase 1 → 2 → 7 (partial) → 5 → 6 → 7 (full) → 3 → 4 → 8 . Alright. Let's see what we can do. The first phase is to fix the touchscreen input. Claude took me through it step-by-step, asking as it needed to read specific project files. Finally, it wrote a new ui.ino code file to my speficied output directory for me to test. I copied it into the correct file location, said a quick prayer, compiled in Arduino IDE, and downloaded to the CYD. Well, that is... interesting. The display looked nothing like it was supposed to. There were vertical green bars with smaller dashed green vertical stripes in them. I will include a picture in the show notes so that you can see what it looked like and why it was so difficult to describe. I spent the next hour or so trying to explain what I was seeing to a chat bot. Claude recommended potential fixes which either did nothing or made the situation worse. I began questioning whether this was a good idea, how people actually gained efficiencies talking to a bot, and even several life choices. Then I had a thought. I prompted Claude: If I were to take a picture of the screen on the cheap yellow display and copy it into the output folder, would you be able to analyze it to better determine what is wrong and how to fix it? Shockingly, Claude answered in the affirmative, and told me to copy the picture to the output folder and let it know when to proceed. It analyzed the picture and more of the supporting files it had copied from my GitHub, asking each time if it could access that file. It determined that my original code was written for a flavor of LVGL version 8 and I was now using LVGL 9.5. It recommended changes, and then asked permission to make those changes, file by file. .h files & .c files, Finally, I just gave it permission to edit the files in the project folder without asking for permission for each file each time. Claude was still explaining each change, showing me exactly what would be changed, and asking for permission, so that I could review all of the changes. But now it was not asking additional permission to write to each of the impacted files. Next, Code compiled and downloaded. Different screen, but not right. Again, I took a picture and gave it to Claude to analyze. So, Claude paused and altered the code to generate a specific test pattern overtop of the GUI. The test pattern was supposed to cover the entire rectangular screen. But parts of the pattern were in a square on the screen and parts were not. Another photograph and analysis, told Claude that there were some rotation/screensize issues. We repeated this several times. Some resulted in improvement, and others did not. This is the point where I noticed something interesting. Not about Claude, specifically, or about the app. But I noticed something interesting about myself and about the process. Previously, when I was working through some of these challenges without Claud, I found myself becoming more and more stressed, frustrated, and angry, until I found a solution. Then another problem would repeat the cycle. Success in the end was great, but the emotional extremes during the process were not always pleasant. Now, I was effectively managing the project, and relaying information to the resource responsible for fixing the problems -- a very different experience. But I also ran into another issue. Claude became absolutely certain that the problem revolved around the device not accurately knowing where the 4 corners of the screen were. But in reality, the output of the test pattern was rotated 90 degrees from the actual screen. It took several iterations of me insisting that the problem had to do with screen orientation and not corner coordinates. It was interesting to experience the tool doubling down on an obvious mistake, but we finally resolved that. Again, while it was frustrating, it was much less stressful. We proceeded to Phase 2: Persistent Message Storage where we ensured that the button labels on the send screen were stored in the devices persistent storage, so that, when they are edited to contain the message they should send, that information would survive a reboot. Next, we combined elements of Phase 5: Morse Code Engine , Phase 6: Non-Blocking LED Transmission , and Phase 7: Wire the Send Buttons together. Building the morse code engine was an area I had been thinking about for a while. I already had working parts of something similar in the Arduino practice oscillator I have referenced a few times in this series. The code for the practice oscillator may be found on my GitHub, but it was all based on original code from jmharvey1, with my only contribution being making pin assignments variables so that the code could easily be ported to different devices. So, I was happy that we were building the morse code engine directly. The code for it may be found in morse.h, which uses a constant character lookup table to define each character. Without any specific direction from me, Claude used the PARIS timing methods I have already described within Episode 6 of this series. It defines timing for DOT, DASH, LETTER_GAP, and WORD_GAP, and all are based on a simple calculation of 1200 ms / the number of words per minute (WPM) we wish to transmit. Along the way, we discovered that, if we tried to use the delay() function, it would crash the program due to a conflict with the LVGL timer used for touchscreen inputs. Claude altered all the delays accordingly. Then, Phase 3: Config Screen — Slot Selection & Speed allowed us to configure the WPM we wished to use in addition to selecting a specific Send button to reconfigure. This forced us to work on Phase 4: Keyboard Screen — Capture and Save Text which is used to type the entries for each Send button. At this point, I also decided that we would want to also use the Keyboard Screen to send ad hoc morse as we typed it. During this phase we discovered several bugs which seemed to cause random freezes. Careful troubleshooting with messages output to the Arduino IDE's serial console helped us narrow down the causes and remedy them. Finally all the tests worked and I am able to merrily pre-configure macro buttons with custom messages and use the CYD to send the morse code for those messages to the on-board LED at whichever rate I specify. I have noticed in my presentation of this narrative that I repeatedly slip into the first person plural terms "we" and "us" instead of the first person singular terms "I" and "me". I have unconsciously personified Claud and recognized it as an integral part of my (formerly one person) development team. I finally configured Claude to connect to my GitHub repo and upload all the files and documentation. We additionally created a CYD-Narrative.md file which describes in more detail all the work which was done on the project. I still do not 100% get git, but we are successfully using it. You can find all these files in my GitHub repo ( https://github.com/jttrey3/CYD_MorseSender ) where they are shared under a GPL 3.0 license. There are still several additional steps I plan to complete in the next few months. 1. I will be integrating an opto-isolated relay which will allow me to plug the device into the straight key input on any amateur radio. This will require a battery power source, charge controller, and more hardware. I... make that "We" (Claude & I) will be modifying the code to support an audio side tone through an attached speaker when sending code We will add an output selection switch to the config page to choose any combination of speaker, relay, or LED as output. We will develop a downloadable firmware which I hope to share with the Cheap Yellow Display community. If you can think of any additional features you would like to see integrated, please drop me an email using the address in my HPR profile. I may also work with a friend to attempt to 3d print a case for the entire contraption, and I will be sure to record additional episodes sharing the process. I have learned so much throughout this project, about the CYD, ESP32, GUIs, Claude Code, GitHub, and most of all, about myself. Does using AI to develop this code make me a fraud? It still feels like it in some ways. Does it make me more productive? ABSOLUTELY! I made consistent forward progress when I only had 30-60 minutes each day to work on it, and everything discussed in this episode was completed in less than a week. If I had been able to work on it for a few hours uninterrupted, it may have only taken me 3-5 hours. Does it empower and inspire me to do more projects like this? 100% I feel like I had support working with me the whole way. I was less stressed overall, and it had less of an impact on the amount of and quality of time I spent with my family. I will be wrapping up this series soon, without any more 6 month gaps, I hope. Until next time... Provide feedback on this episode.

This show has been flagged as Clean by the host. 01 Introduction In this episode I will describe techniques for downloading podcasts using basic shell commands such as wget. I will illustrate this using a bash script that can be used to download HPR podcasts. Even if you do not have any interest in downloading your podcasts using this method, you may find some of the methods useful or interesting. It is the principles that are discussed here that are important, rather than the implementation. 02 I realize that there are already a number of different podcast download programs available, including at least one written in bash. However, you may feel that none of these suit how you wish to do things and want to create your own system tailored to your specific needs. If so, then I hope the following is of some use to you. If not, then you may still find some of the things discussed here to still be of interest. Some of the subjects I cover include wget to a user defined file name. parsing xml with xmllint. using inotifywait to trigger an action when a file is created or modified. using notify-send to send a message to the notification area. and a way of allowing a cron job to send a message to the user interface. 03 Background There has been an ongoing discussion in comments to some HPR episodes about problems downloading HPR podcast episodes. Apparently some people have been experiencing problems with the way the episode URLs are structured. 04 I am afraid that I don't fully understand the nature of these problems, so I won't be addressing that problem directly. Instead, I will present a bash script that I have written which can be used to download HPR podcasts. This bash script can be run using cron to automatically fetch new HPR podcasts and save them to a designated directory. This is a simplified version of a script that I have used for years to download HPR and other podcasts. 05 I won't try to read the full bash script out in this podcast, as that would be a bit dull to listen to. I will instead describe what each section does and why I chose to do things that way. Perhaps other people can offer suggestions of better ways to do things. I will post the full bash script in the show notes. 06 Fetching Podcasts The standard way of distributing podcasts is to publish an RSS feed containing URL links to the audio files. RSS is a very long established and widely supported mechanism for this and other purposes. An RSS feed is basically an XML document which can be accessed over HTTP. These URLs contained in the RSS XML document can then be used to download the actual audio files, such as MP3 or OGG files. 07 Basically what we need to do is the following • Download the RSS XML document. • Extract the URL links to the audio files. • Compare the list of these links to a previously saved list to see which ones are new and which ones are ones that we previously downloaded. 08 • Make a list of the new URLs. • Go through this list of new URLs and download each of the new audio files. • Check to see that we actually received the new audio file. • Add the URLs of the files we successfully downloaded to our saved list of podcast URLs 09 In addition to this, we would like to have the above happen automatically in the background without our having to take any action on our own. We may wish to receive a notification of when a new podcast has arrived however. We would probably also wish to receive notification of any errors or failures. 10 Fetching Podcasts - The Preliminaries Our desire to be able to run the script automatically imposes some requirements on our solution. To schedule the script we will use cron. Cron is a Linux facility to run scripts on a schedule. 11 One of the side effects of using cron however is that we need to specify the full path to the locations where we intend to keep any data files, plus also the full path to where we intend to put the downloaded podcasts. 12 So the first thing we need to do in our script is to specify a number of different values for things like file location, the URL for the HPR RSS feed, and several other things as well. I will skip over the details of these, although I may make reference to them later. 13 Get the RSS Data The first thing of real substance to do is to fetch the current RSS feed data. I have put this in a bash function called getrssurldata The contents of this function are a one liner, but with a number of elements chained together through pipes. 14 Downloading the RSS XML Document • First we use wget, which is a standard command on most Linux distros. • We specify four things. • First we set a timeout. I have chosen 20 seconds. • Next we set the retry limit. I have chosen 3. 15 • Then we specify that the output of wget is sent to stdout rather than saved as a file. • This is done by using the -O option followed by a space and then a dash. • The O option is usually used to specify a file to save the output to, but when used with a dash causes output to go to stdout. • Then we specify the URL of the HPR RSS feed. 16 Contents of the XML Document This gives us the HPR RSS XML document. There are about 5,000 lines in this RSS document. Most of those lines are the show notes which are also included in the feed. 17 Extracting the Podcast Episode URLs There are only 10 lines of the document that contain information that we are interested in however. These lines are enclosed in "enclosure" XML tags. We just need to find those lines and separate out the URLs 18 Standard Command Line Tools There are two ways that we can do this. One is to use a combination of grep, sed, and cut. Grep can find the lines containing the enclosure tags. Sed and cut can extract the URL from the surrounding extraneous data. 19 However, this method does not discriminate between real enclosure tags in the data portion of the RSS feed and enclosure tags in the show notes which are included in the feed from episodes such as this one. This may be an acceptable problem in practical terms, but we can do better. 20 Using an XML Parser The other method is to actually parse the XML document. there are at least two command line XML parsers that I am aware of. These are "xmllint", and "xlmstarlet". I have used xmllint in this example. I have not used xmlstarlet, so I can't offer any comment on how easy or difficult to use it is. 21 I won't give a detailed explanation of all the things that xmllint can do. It has many features, most of which, as the name suggests, have to do with finding formatting problems with the XML itself. Describing everything it can do would be at least one episode in itself. I will instead just give the particular command used and explain each element of it. 22 In this example assume that we are piping the output of wget directly into xmllint. The complete command is xmllint --xpath "//channel/item/enclosure/@url" - | cut -d'"' -f2 23 In this example, xmllint is the name of the command. --xpath tells it to parse the document according to the string which follows. "//channel/item/enclosure/@url" tells it to find a series of tags in the hierarchy of channel, followed by item, followed by enclosure, and then extract the url attribute from the enclosure tag. The "-" which follows tells it to look for input from stdin rather than from a file. 24 The result is a string which has the url attribute name, an equal sign, and the URL that we want enclosed in quotes. To get just the URL itself, we pipe the output from xmllint into cut, using the doublequote characters as delimiters. We then save the result in a temporary file. 25 Finding the New Episodes Next we wish to find the new podcast episodes. Each HPR episode is identified by a unique URL. This means that if we save the URLs of episodes that we have already downloaded, we just have to look for the URLs that do not appear in this saved list. https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr4659/hpr4659.mp3 26 The easiest way to do this is to take our two lists of URLs, sort each into temporary files, and then compare the sorted URLs using the "comm" command. 27 This is simple, but has a drawback. Some podcasts occasionally change distributors. When they do this, the old podcasts are re-published with new URLs and you end up downloading a lot of old episodes over again. 28 With HPR we could get around this by extracting just the file name and looking for that instead of the full URL. I will however leave that problem as an exercise for the student and just accept that if the URL format changes we may end up downloading old episodes over again. Since the feed has a maximum of only 10 episodes in it however, that isn't really that big of a problem. It would be more of a problem with podcasts which have very large numbers of episodes in their feed, but the solutions to those will be feed specific. 29 Downloading the New Podcasts We should now have a list of URLs for the new podcasts we do not already have. Typically this should be only one file, but there could be several, or even as many as 10, if we have not turned on our computer in a while. Therefore, we need to iterate through the file of new podcast URLs and download each one. 30 Before we do that however, we should check to see if there is in fact anything new to download. To do this, simply use "wc -l" to count the number of lines in the list of new URLs and save the resulting number. 31 If this number is zero, there is nothing to download, we can skip the download step. As an additional check, we should see if the number of downloads exceeds some threshold value that we wish to set. This is not a major problem with HPR, but some podcasts have hundreds of files in their RSS feed rather than just the most recent ones. If we do exceed our download limit, then we need to log an error and skip downloading. 32 Assuming there are no problems so far however, the first thing we need to do is to extract the name of the audio file from the URL. We can do that using the "basename" command. We will use this to specify the name that we use when we save the audio file. 33 HPR has a very well formed file name. Some podcasts do not however, and for those you would need to construct some sort of suitable name either using information found in the URL or simply creating a name using a time stamp. 34 Next we download the audio file using wget. This is similar to how we downloaded the RSS feed, but with a few changes. One is that I have increased the timeout to 90 seconds. This may not have been necessary, but seemed like a good idea. 35 The next is that when specifying the output file name using -O, we use the file name we extracted from the URL. The third is that we specify a destination directory using the -P option. 36 After wget has finished, including any retries that it had to do, we next check that the expected new file is both present and not empty. We did this using an "if" statement with the "-s" option. If the file was found and not zero, then we add that URL to a temporary list of downloaded URLs. 37 If the file was not present, or was zero length, we output an error message to an error log. I will come back to this point later. 38 Next, if there is more that one podcast to download we sleep for 3 seconds. While not strictly necessary, it is considered to be "polite" to not hammer a server repeatedly, but rather to put a small delay between file downloads.. 39 After we have downloaded all the audio files in our list, we can add the list of URLs for the files downloaded to the permanent list. While we are at it, we should use "tail" to trim the permanent log to keep it from growing indefinitely. This limit should be several times bigger than the number of files in the RSS feed. In this case I selected 50. 40 Finally we write any errors to the permanent error log, and also write these same errors to another file used to signal errors for display to the user. We have now successfully downloaded at least one HPR podcast. 41 Notify the User of Events It would be convenient to be informed of new podcast downloads when they occur, and also be notified of any errors. One of the limitations of cron jobs is that they cannot access the user interface. This means that we cannot readily send a message directly to the notification system to inform the user of the presence of new podcasts or of errors. 42 inotifywait The solution to this is to use "inotifywait" to monitor particular files and directories for changes. The man page for inotifywait states the following - 43 inotifywait efficiently waits for changes to files using Linux's inotify(7) interface. It is suitable for waiting for changes to files from shell scripts. It can either exit once an event occurs, or continually execute and output events as they occur. End of quote. 44 In many Linux distros, inotifywait is provided by the "inotify-tools" package. I won't go over all the features of inotifywait. Instead, I will just describe how to use it for our purposes here. 45 inotifywait Modes I should point out first though that inotifywait operates in two different modes. In the normal default mode, it exits after being triggered by an event and must be re-established again in order to resume monitoring. In monitor mode, which is enabled by using the "-m" option, it runs indefinitely, responding to events. I will use the default mode here. 46 The man page for inotifywait provides a simple example that we could copy and modify for our purposes. A great many examples that you will find are based on this example. However, it doesn't quite do what we want, so we need to change a few things. 47 podfetchnotify The first shell script is one which monitors for the arrival of new podcasts and sends a notification to the user. I will call this "podfetchnotify". The complete scripts are in the show notes, I will just provide a brief description here. 48 Setting Up Event Watches Using inotifywait The script is enclosed in a while loop which run indefinitely. In the first line inside the while loop, we call inotifywait. inotifywait will then block until the event it is told to look for occurs. In short, execution of the script will wait there until an event occurs. 49 The names of the events are listed in the man file. In this case we are looking for "modify", "create", and "moved_to". Each of these does pretty much as you would expect, reacting to modifying an existing file, creating a new file, or moving a file to that directory. 50 Problems When Testing Using Text Editors I should point out that if you are testing a script which uses inotifywait, then modifying a file with a text editor may not produce the results that you may think it would. Instead it treats this as a new file with the same name, with the original file being erased. Since inotifywait attaches itself to the inode rather than the filename, it sees the file that the text editor changed as being a new file. If you wish to test this realistically, then use "echo" to overwrite the file by using I/O redirection. 51 Capturing Output In my example I capture the output from standard out into a variable, but I don't do anything with it. If you wish to for example display the name of the newly downloaded podcast file, then use the --format option along with an appropriate formatting code. There are details about this in the man page. On the next line we capture the exit code using "$?" 52 Responding to Exit Codes If the exit code was zero, then a monitored event was triggered and there should a new podcast in the directory. In this case we display a message indicating that a new podcast has arrived. I will describe how to send notifications shortly. If the exit code was not zero, then an error occurred. An example of such an error would be if the directory were not present when monitoring was started. In this case we display a message indicating that a fatal error has occurred and then exit. 53 Delay for More Podcasts Finally, we use "sleep" to wait for some arbitrary period of time to prevent notifications from being triggered multiple times if several podcasts were being downloaded in succession. In this case I chose to wait for 60 seconds. 54 We have now completed the process and can return to the top of the loop and resume waiting using inotifywait. 55 Sending Notifications to the User I mentioned above about sending notification messages to the user. In the Gnome desktop, notification messages appear from the centre of the top bar in a list. Other desktops or operating systems may have something similar. 56 To send a notification message to the notification area, you use the "notify-send" command. Simply follow notify-send with a quoted string and it will be displayed in the notification area. 57 podfetcherrornotify The second shell script is one which notifies the user of errors. I will call this "podfetcherrornotify". With this shell script we set up a watch on a file which contains any error messages from podfetch. This script is very similar to podfetchnotify. 58 The exceptions are With inotifywait we only monitor for "modify". There is no sleep command at the end of the loop. Instead we sleep for a few seconds just after getting the exit code from inotifywait. This helps prevent problems caused by race conditions. 59 Next we check the inotifywait exit code. If it was zero, then we read the error report file and send a notification message to the user containing that error message. 60 If it was not zero, then we check to make sure that the directory that should contain the error log exists. If it does not exist, then we send a notification message to that effect to the user and terminate the script. 61 If the directory exists, then we check to see if the error message file used for signalling exists. If the file does not exist, then we create it. 62 One of the reasons for an inotifywait error is that if the file that it is told to monitor does not exist, it cannot set up a watch condition. By creating the file we correct the cause of the error and allow inotifywait to operate normally. 63 Finally we increment an error counter and check to see if the limit is exceeded. If there are excessive errors, then send a notification message to the user and exit. The reason for this is to give the user an indication that the error notifications are not working for some reason and there may be a problem that needs looking into. 64 The error counter is reset every time the inotifywait exit status is ok, so occasional unexpected glitches should be something that is ignored. Of course podcast fetching errors are something that will probably happen only rarely if at all, so this final step may be seen as an unnecessary embellishment. 65 Installing the Scripts Next I will describe how to install and prepare the scripts to run. We need to perform the following steps. 66 • First, we need to create a directory to hold the scripts and their associated data files. • Next we need to create a directory to hold the downloaded podcasts. • Then we must copy the scripts to these directories and make them executable. • Then, we must edit the scripts to have the file path in the script match the locations of the new directories that we created. 67 • Then we need to install xmllint, or alternatively modify the download script to comment out the use of xmllint and enable the alternative method using grep and sed instead. • Then we need to run each script manually from the command line to check for errors. • If podfetch ran correctly, it should download the most recent 10 podcasts during this test. 68 Adding podfetch to the Crontab The above describes how to run the scripts manually. In order to fetch podcasts automatically, we need to add the podfetch script to the cron schedule. To do this, open a terminal. 69 Type "crontab -e", and then press return. A text editor should open up containing the crontab file. On Ubuntu, this editor is GNU nano. Enter the appropriate cron parameters. I will provide an example here for running it 12 minutes past the hour every three hours. 70 12 */3 * * * /home/username/pathtofiles/podfetch.sh 71 I won't explain cron in detail here. The example that I have just given should be good enough for most people. The "*/3" parameter will cause it to run every three hours. The "12" parameter will cause it to run 12 minutes past the hour when it does run. 72 Checking every three hours should be good enough for most people, but you can adjust that as you see fit. I would recommend however that you don't check more frequently than once per hour. Checking more frequently than necessary puts extra load on the distribution servers. It is very unlikely that you really do need each new episode the moment it is available. 73 I would also recommend changing the "12" parameter to some other random minute value. I would suggest avoiding on the hour or on the half hour, as a lot of other people are probably checking at those times, and it would be better to spread the load out more evenly over time. 74 The file path parameter should of course match the actual path to wherever you have located the script, including the correct user name. 75 Making the Notification Scripts Start Automatically The two notification scripts can be made to start automatically. The exact method to do this may vary according to distribution or desktop. 76 On Ubuntu this is done using the Startup Applications Preferences GUI program, which should come already installed. 77 I won't go into details on this here, it should be fairly self evident how to use it once you see it. What this program does is to create ".desktop" files in the ".config/autostart" directory in your home directory. 78 These ".desktop" files are all run automatically on start up. Once you have added the notification scripts, you will need to log out and then log back in to make them active. 79 Conclusion I this episode I explained how to write a set of simple shell scripts to automatically download each new episode of HPR as it comes out and to notify you of its arrival. 80 The download script described here is tailored specifically for use with HPR only. However, it was derived from a larger script that downloaded other podcasts as well, based on information read in from a text file. If you are feeling ambitious, you can add those features back into this to handle all of the podcasts that you listen to. 81 In a comment to another episode of HPR I had said that I would cover ID3 tags in MP3 files, but this episode is long enough now, so I will leave that subject for later. I look forward to seeing you again later on another episode of Hack Public Radio. # ====================================================================== podfetchdownloader #!/bin/bash # Fetch pending HPR podcasts listed in the HPR RSS feed. # 8-Jun-2026 # Licensed under GPLv3 or later. # ====================================================================== # Today's date and time as YYYYMMDDHHMMSS. podttimestamp=$( date +"%Y%m%d%H%M%S" ) # The absolute path to the script. This is necessary when running it # using a cron job. podpath="/home/me/Apps/hprfetch" # This is the absolute path to where to store the podcast files. podfilepath="/home/me/Music/Podcasts/HPR" # Create the full path names here for all the text files used. podcastsfetched="$podpath/podcastsfetched.txt" poderrorslog="$podpath/poderrorslog.txt" poderrorsreport="$podpath/poderrorsreport.txt" tmpoldurlssorted="$podpath/tmpoldurlssorted.txt" tmppodsnew="$podpath/tmppodsnew.txt" tmppodstodownload="$podpath/tmppodstodownload.txt" tmppodserrors="$podpath/tmppodserrors.txt" tmppodcastsfetched="$podpath/tmppodcastsfetched.txt" tmplog="$podpath/tmplog.txt" # The URL for the HPR RSS feed. PodURL="http://hackerpublicradio.org/hpr_rss.php" # Limit on number of podcasts to download. DownloadLimit=11 # Name of the podcast. PodName="Hacker Public Radio" # ====================================================================== # Check if the required paths exist. # If this path does not exist, cannot log the error. if [[ ! -d "$podpath/" ]]; then echo "$podttimestamp Error - Could not find $podfilepath." exit 1 fi # Where to store the podcast file fetched. if [[ ! -d "$podfilepath/" ]]; then echo "$podttimestamp Error - Could not find $podfilepath." >> $tmppodserrors # Copy the errors log from the temporary errors file to the permanent files. LogErrors exit 1 fi # ====================================================================== # Check if the podcast log exists. We read it before we write to it, # so it must exist or we will hang on it not being present. if [[ ! -e $podcastsfetched ]]; then touch $podcastsfetched fi # ====================================================================== # Delete the specified files if they exist. # This accepts multiple file names in a variable number of parameters. CleanupFiles () { # $@ accepts multiple parameters. for f in "$@"; do # Check if the file exists. if [ -e "$f" ]; then rm "$f" fi done } # ====================================================================== # Copy the errors log from the temporary errors file to the permanent files. LogErrors () { if [ -e $tmppodserrors ]; then # The permanent log. cat $tmppodserrors >> $poderrorslog # This file is monitored for display by other scripts. cat $tmppodserrors > $poderrorsreport fi } # ====================================================================== # Get the URL data from an RSS feed GetRSSURLData () { wget --timeout=20 --tries=3 -O - "$PodURL" | xmllint --xpath "//channel/item/enclosure/@url" - | cut -d'"' -f2 | sort > $tmppodsnew # This is an alternate method that does not use xmllint. # However, it is not as robust. If someone were to include the # first grep search pattern in their show notes, then it would # look for that as a valid tag and output the following text # as a URL. #wget --timeout=20 --tries=3 -O - "$PodURL" | grep " $poderrorsreport fi # Increment the error counter. count=$(( count + 1 )) if (( count > 3 )); then notify-send "Podfetch error: Excessive unknown errors, exiting." exit 1 fi fi done # ====================================================================== Provide feedback on this episode.

This show has been flagged as Clean by the host. ether This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems. I frequently find myself reaching for the cut utility when writing scripts to extract one piece of data from a line, or to select specific fields from a log file. While I am familiar with its counterpart, paste , I don't employ it very often because I don't typically need its functionality. This perhaps has to do with the fact that I rarely work with text files containing lists. For shorter lists, I usually end up using a spreadsheet and for larger ones, a relational database. Both are valuable tools with their own strengths and weaknesses, but it is good to also know about standard utilities for working with lists. After uploading UNIX Curio #8 ( HPR episode 4657 ), I felt like maybe I had been too dismissive of the comm utility in that episode and should talk more about tools that are useful when managing lists. I don't frequently find myself using paste 1 , but can explain how it works. Briefly, it is a rough opposite of cut —when given multiple files as arguments, it assembles the first line from each one separated by tabs, then the second line, and so on. Instead of tabs, a different delimiter can be chosen with the -d option. Another option is -s , which swaps rows and columns so that the contents of each named file would appear on one line. While paste itself doesn't qualify as a UNIX Curio in my opinion, there is one feature that does: a hyphen can be given as an argument multiple times. In this special case, the output is taken line by line from standard input, but is spread across as many columns as there are hyphens. Example of using paste to turn the output of ls into columns. Because these columns are separated by tabs, they don't necessarily line up when a filename is eight or more characters long. The -1 is not required for the second ls command since that behavior is implied when output isn't going to a terminal. The -C option to ls usually gives nicer-looking output on a terminal—also, it lists in ascending order down by column. (Most implementations default to -C when output goes to a terminal.) If you want items ascending along rows like the paste example does, try ls -x instead. $ ls -1 /proc/net anycast6 arp bnep connector dev dev_mcast dev_snmp6 fib_trie fib_triestat hci icmp icmp6 if_inet6 igmp igmp6 ip6_flowlabel ip6_mr_cache ip6_mr_vif ip_mr_cache ip_mr_vif ip_tables_matches ip_tables_names [...35 more entries not shown...] $ ls /proc/net | paste - - - - anycast6 arp bnep connector dev dev_mcast dev_snmp6 fib_trie fib_triestat hci icmp icmp6 if_inet6 igmp igmp6 ip6_flowlabel ip6_mr_cache ip6_mr_vif ip_mr_cache ip_mr_vif ip_tables_matches ip_tables_names ip_tables_targets ipv6_route l2cap mcfilter mcfilter6 netfilter netlink netstat packet protocols psched ptype raw raw6 rfcomm route rt6_stats rt_acct rt_cache sco snmp snmp6 sockstat sockstat6 softnet_stat stat tcp tcp6 udp udp6 udplite udplite6 unix wireless xfrm_stat $ ls -C /proc/net anycast6 if_inet6 l2cap rfcomm tcp arp igmp mcfilter route tcp6 bnep igmp6 mcfilter6 rt6_stats udp connector ip6_flowlabel netfilter rt_acct udp6 dev ip6_mr_cache netlink rt_cache udplite dev_mcast ip6_mr_vif netstat sco udplite6 dev_snmp6 ip_mr_cache packet snmp unix fib_trie ip_mr_vif protocols snmp6 wireless fib_triestat ip_tables_matches psched sockstat xfrm_stat hci ip_tables_names ptype sockstat6 icmp ip_tables_targets raw softnet_stat icmp6 ipv6_route raw6 stat $ ls -x /proc/net anycast6 arp bnep connector dev dev_mcast dev_snmp6 fib_trie fib_triestat hci icmp icmp6 if_inet6 igmp igmp6 ip6_flowlabel ip6_mr_cache ip6_mr_vif ip_mr_cache ip_mr_vif ip_tables_matches ip_tables_names ip_tables_targets ipv6_route l2cap mcfilter mcfilter6 netfilter netlink netstat packet protocols psched ptype raw raw6 rfcomm route rt6_stats rt_acct rt_cache sco snmp snmp6 sockstat sockstat6 softnet_stat stat tcp tcp6 udp udp6 udplite udplite6 unix wireless xfrm_stat The paste command has limitations—the files you give it must all be already arranged in the same order, and if any file is missing a value, it must have a blank line so that subsequent lines will match up correctly. The files do not necessarily have to be sorted alphabetically, but whatever order they are in has to be the same. Check out HPR episodes 962 and 4201 for some more background on the paste utility. Example of using paste with files where some values are empty. Bob works from home so doesn't have an office assigned, and the laboratory Carol works in doesn't have a phone. This relies on the fact that the same line number in every file relates to the same person/entry. $ cat names Alice Bob Carol Dave $ cat offices 203 Lab6A 117 $ cat phones +1 212-555-1234 +1 919-555-2345 +1 212-555-1278 $ paste names offices phones Alice 203 +1 212-555-1234 Bob +1 919-555-2345 Carol Lab6A Dave 117 +1 212-555-1278 Our second UNIX Curio for today is a utility called join 2 , which has a bit more sophistication. It operates on two files, which can have multiple columns, and combines them using the join field. By default, the first column/field in each file is the join field, and only entries that exist in both files are printed. The -1 and -2 options can be used to join on a different field, and -o selects specific fields to be output. To make it so lines with missing entries also appear, you need to use the -a option, but an actual empty string with separator won't be printed unless -o is also present and includes the field. The default field separator character is one or more "blanks" in the current locale—for the POSIX locale, this means a space or a horizontal tab. The -t option selects a different character and also removes the treatment of multiple occurrences as a single separator, making it possible to have an empty field in one or both of the files. By default, a single space is used to separate fields in the output. If -t is given, the same character is used for separating fields in both input and output. You would need to pipe output through another tool like tr if you wanted to have a different separator in the output. The join utility might be an improvement over paste in some cases, since the join field makes it a little easier to identify which entries match up across files. It is limited to operating only on two files (one of which can be standard input), so combining more than that requires either creating temporary intermediate files or chaining together join commands in a pipeline. Another requirement is that all files must already be sorted in the current locale. Example showing how join can be used with two tab-separated lists. The LC_ALL assignment forces join to sort using the C (POSIX) locale instead of whatever might be set in your environment. The "@" on the header line has no special meaning; it is just there to make sure it sorts before any letters or numbers (in the C locale; it might not in other locales). Note that if -t were not specified, plist would be treated as having three fields because of the space separating the country code from the rest of the phone number. $ export tab="$(printf 't')" #To more easily use tab characters below $ cat olist @Name Office Alice 203 Carol Lab6A Dave 117 $ cat plist @Name Phone Alice +1 212-555-1234 Bob +1 919-555-2345 Dave +1 212-555-1278 $ LC_ALL=C join -t "$tab" olist plist @Name Office Phone Alice 203 +1 212-555-1234 Dave 117 +1 212-555-1278 $ LC_ALL=C join -t "$tab" -a 1 -a 2 olist plist @Name Office Phone Alice 203 +1 212-555-1234 Bob +1 919-555-2345 Carol Lab6A Dave 117 +1 212-555-1278 $ #By default, join acts as if empty fields don't exist; use -o to include $ LC_ALL=C join -t "$tab" -a 1 -a 2 -o 0,1.2,2.2 olist plist @Name Office Phone Alice 203 +1 212-555-1234 Bob +1 919-555-2345 Carol Lab6A Dave 117 +1 212-555-1278 $ #The -e option sets a placeholder to use for empty fields $ LC_ALL=C join -t "$tab" -e "(none)" -a 1 -a 2 -o 0,1.2,2.2 olist plist @Name Office Phone Alice 203 +1 212-555-1234 Bob (none) +1 919-555-2345 Carol Lab6A (none) Dave 117 +1 212-555-1278 The brief description for join is "relational database operator"—I won't dispute that, but in my view it offers far fewer capabilities than people would expect from today's relational databases. I would imagine that when most people think of those they have Structured Query Language (SQL) in mind, which offers a lot more flexibility and functions to operate on data. However, I can see how join could be suitable for simple operations. Our last UNIX Curio for today relates to the sort utility 3 . While, as you might expect, it is well-known for its ability to sort data, it has another feature that is more obscure. When used with the -m option, instead of sorting the files given as arguments, it merges them together. All of the files are expected to already be sorted—once combined, the list that is output will also be sorted. The order in which the files are named does not matter; it is not required for the contents of the first file to start before the second, just that both are sorted. $ cat women Alice Carol $ cat men Bob Dave $ sort -m men women Alice Bob Carol Dave Imagine that you organize an annual event and have a separate pre-sorted list of attendees' e-mail addresses for each of the past three years. You are planning this year's event and want to send out an announcement to all of these people, as they will probably be interested. The command sort -m -u 2023list 2024list 2025list would spit out a combined list that you can use for your e-mail blast. Because it is likely that some people would have attended in more than one year, I included the -u option—it removes any duplicate entries. It is probably no surprise that the sort utility appeared early on—it was in 1971's First Edition UNIX, though it didn't gain the merging functionality until Fifth Edition 4 in 1973. What did come as a shock to me is that both cut and paste didn't show up until 1980 with System III 5 , and were actually preceded by join , which was in Seventh Edition UNIX 6 from 1979. I assumed that at least cut would have been around far earlier, given its usefulness and how firmly established it is, but I suppose it just seems to have been with us forever. As mentioned, I don't typically manage data as text files containing lists, and I probably won't start using the join utility or these features of paste and sort very much. But it is still useful to know that they exist and how they work. Hopefully this episode has taught you a bit about them. References: Paste specification https://pubs.opengroup.org/onlinepubs/9699919799/utilities/paste.html Join specification https://pubs.opengroup.org/onlinepubs/9699919799/utilities/join.html Sort specification https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sort.html A Research UNIX Reader: Fifth Edition sort manual page https://archive.org/details/a_research_unix_reader/page/n19/mode/1up System III paste manual page https://www.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/man/man1/paste.1 Seventh Edition UNIX join manual page https://man.cat-v.org/unix_7th/1/join Provide feedback on this episode.

This show has been flagged as Explicit by the host. Show Notes Episode Overview Operator kicks off the episode feeling under the weather but shares a quick tip for making perfect egg drop soup before diving into his main project: diagnosing why his front-door security camera stopped sending alerts and recording events. What follows is a live-debugging session covering network config, script logging, Windows permission hacks, NTP time drift, and firmware flashing. Key Topics & Breakdown Egg Drop Soup Hack: How to get that perfect ribbony texture by creating a boiling swirl before pouring in the eggs, plus broth-to-egg ratio tips. Camera Setup & Network Config: Using static DHCP via MAC address binding on a UniFi Dream Machine (UDM) for local domain resolution instead of hardcoding IPs. Python & Cron Automation: Running a custom Python script every 2 minutes to check for new recordings, parsing logs with grep -v , and navigating massive log files in vi . Windows Troubleshooting Tangent: Deleting the stubborn Windows.old folder using the TrustedInstaller service hack ( ExecTI.exe ) instead of taking ownership manually. Time Sync & Firmware Quirks: Discovering the camera's system clock was stuck in 2011/2026, causing missed events. Downloading firmware via a slow third-party link, renaming .bin to .zip , and extracting with 7-Zip. Pre-Flash Backup Routine: Exporting camera configuration before upgrading, storing it in Google Drive for searchable documentation, and clearing old log/trigger files to reset the event pipeline. ️ Tools & Techniques Mentioned crontab + Python scripts for automated monitoring grep -v , cat , tail , and vi (line navigation with :1000 ) Obsidian for note-taking & AI assistant integration Firefox/Playwright for headless browser testing Turbo Download Manager & Bolt Media Downloader for multi-threaded/sniffing downloads 7-Zip for archive extraction Google Drive for searchable config backups Resources & Links Python API Script: Uniview IPC3628SR Recording Checker Camera Model: IPC3628SR (Uniview Wyze ISP Warm Light Deterrent Network Camera) TrustedInstaller Run-as Tool: ExecTI TrustedInstaller Runner Quick Takeaways Always verify NTP/time sync on IoT cameras before troubleshooting missed events or alerts. Use grep -v "noise" to quickly filter out repetitive log entries when debugging automation scripts. Windows system folders can be stubborn; running commands as TrustedInstaller bypasses hidden file locks without manual ownership changes. Always export and back up device configs before flashing firmware, even if the upgrade seems straightforward. Third-party download links often use temporary tokens or .bin wrappers; renaming to .zip and verifying with 7-Zip can save headaches. Thanks for listening! Stay curious, keep your logs clean, and remember: defense in depth starts at home. Example trusted installer hack # Shhhh I can't IR ... Defender, ForcePoint, SMS Agent Host ...I just can't anymore ... sc config TrustedInstaller binPath= "Reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesSense" /v Start /t reg_dword /d 4 /f" sc start "TrustedInstaller" sc config TrustedInstaller binPath= "Reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesFppsvc" /v Start /t reg_dword /d 4 /f" sc start "TrustedInstaller" sc config TrustedInstaller binPath= "Reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesCcmExec" /v Start /t reg_dword /d 4 /f" sc start "TrustedInstaller" sc config TrustedInstaller binPath= "Reg add "HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesWinDefend" /v Start /t reg_dword /d 4 /f" sc config TrustedInstaller binPath= "C:WindowsservicingTrustedInstaller.exe" Provide feedback on this episode.

This show has been flagged as Clean by the host. Lennart tells about the shortwave radio he had in the early 1980s and what he could hear on it, for example he marine band, amateur radio, CB and of course broadcast stations. The radio did not have a BFO to demodulate SSB stations, but using a second radio close by and using its local oscillator he could still make those signals intelligible. Provide feedback on this episode.

This show has been flagged as Clean by the host. Thrustmaster TMX Force Feedback Pro Black Xbox One / Xbox Series X/S / PC https://www.ebay.com/itm/157554569422 Dayton Audio DAEX25 Audio Exciter Pair - Sound Exciter Pair Audio Transducer - 5 Watts RMS, 8 Ohms Impedance - 2 Pack - Turn Any Surface into a Speaker System https://www.amazon.com/dp/B001EYEM8C https://www.simhubdash.com/ https://www.accsetupcomparator.com/ https://www.iracing.com/ ( they have buy 2 years get discount during holiday but still can't afford it .. ) https://forza.net/horizon DiRT Rally 2.0 GOTY https://k4g.com/store?distribution[]=3&q=DiRT%20Rally%202.0%20GOTY&sort=price CrewChiefV4 https://thecrewchief.org/ https://app.tracktitan.io/sessions/3ebbf63b-93de-4309-9a42-9311a745209d/20241228052252 SUMMARY. User discusses sim racing challenges, costs, and setup tips. IDEAS. Sim racing requires time and investment. I Racing is expensive with annual costs. Set of Courses offers one-time payment. Proper setup enhances sim racing experience. Cable connections can complicate setup. Upgrading hardware improves performance. Arcade games like Forza offer casual play. Realistic sim racing demands dedication. License requirements vary between platforms. Remote gaming systems save space. Steering wheel upgrades improve smoothness. Dedicated spaces optimize sim racing. Balancing fun and realism is key. Multi-launchers manage gaming platforms. Hacked accounts may cause issues. Monitoring hardware wear is important. Shifting mechanisms enhance control. Curved monitors improve immersion. Time constraints affect sim racing participation. Exploring multiple games adds variety. RECOMMENDATIONS. Consider Set of Courses for a one-time payment. Invest in a proper setup for serious sim racing. Use a multi-launcher for managing games. Upgrade hardware for smoother performance. Buy specific tracks and cars to avoid costs. Opt for a dedicated space for sim racing. Check for license requirements before purchasing. Remote into gaming systems to save space. Replace plastic parts with bearings for smoother operation. Use a 7-speed shifter for better control. Avoid hacked accounts for reliability. Purchase multiple accounts for different players. Focus on arcade games for casual play. Prioritize a curved monitor for immersion. Use a standing desk for accessibility. Monitor cable connections to prevent setup issues. Upgrade steering wheel components for better experience. Balance fun and realism based on personal preference. Consider time investment for sim racing. Explore different racing games for variety. Provide feedback on this episode.

This show has been flagged as Clean by the host. Recording Kit General Lanyard - HPR - Your Name HPR Business Cards - Your email address HPR Stickers Pen A6 Notebook Android Mobile phone OpenCamera Zoom H2 Zoom H2 Handy Recorder Set format to best and date format to ISO8601 Replacement Batteries Micro USB Cable 2.5 mm headphones 3.5 mm Male to Male cable 3.5 mm earbuds Backup 1 Sansa Clip Rockbox Remove before flight key fob Set format to best and date format to ISO8601 Backup 2 Android Mobile phone AudioRecorder Set format to best and date format to ISO8601 Provide feedback on this episode.

This show has been flagged as Explicit by the host. SUMMARY The presenter outlines a practical cybersecurity workflow, covering ergonomic setups, browser isolation, virtual machine troubleshooting, AI-assisted scripting, and network tunneling methods utilized during active security assessments. ONE-SENTENCE TAKEAWAY Isolate browser environments, utilize automation scripts, and verify network paths before starting security tests to avoid workflow interruptions. TOOLS Talon Voice – Open-source voice recognition software enabling hands-free computer control and command execution. Obsidian – Local-first markdown note-taking application supporting secure, AI-friendly knowledge management. AutoHotkey – Windows scripting utility for creating custom macros and remapping keyboard inputs. Chrome Debug Commands – Browser developer tools allowing direct inspection of extensions, cookies, and storage. Whisper Diarization – Audio processing script that separates speaker tracks and converts recordings to searchable text. Hyper-V / WSL – Microsoft virtualization platforms enabling isolated guest environments and Linux subsystem integration. OpenConnect / OpenVPN – Command-line tunneling clients used for establishing secure, split-tunnel network connections. Jamboree Framework – Portable PowerShell environment that dynamically provisions development tools without altering system paths. MOBA Portable – Feature-rich terminal emulator supporting static/dynamic tunnels, auto-reconnect, and embedded X-server capabilities. Nmap – Network discovery and security auditing tool utilized for comprehensive port scanning and service detection. 00:00:00 Ergonomic Workspace Configuration Configures physical workstation elements to reduce strain during extended testing sessions. Proper alignment prevents repetitive stress injuries while maintaining focus on technical tasks. Monitor Positioning – Displays should align with eye level to maintain neutral neck posture; the speaker notes their curved 49-inch screen sits slightly high due to chair adjustments. Split Keyboard Layout – Utilizes a Freestyle 2 mechanical keyboard, allowing natural shoulder-width arm placement and reducing wrist deviation during prolonged typing. Postural Adaptation – Acknowledges that ergonomic equipment requires matching body alignment; elbow rests should sit between hip and shoulder height for optimal leverage. 01:45:00 Voice Control & Note Synchronization Utilizes auditory input methods and localized knowledge bases to streamline documentation workflows. Separating secure work notes from casual observations prevents data contamination. Talon Voice Integration – Runs continuously to handle navigation, text entry, and application switching without manual keyboard interaction. Obsidian Migration – Transitions from cloud-based keep apps to local markdown files, enabling direct querying by local AI models while maintaining offline accessibility. Note Categorization – Divides information into secure work records and insecure personal logs, ensuring clean data pipelines for future retrieval and analysis. 03:50:00 Browser Extension Management & Security Isolation Separates web browsing activities from primary work processes to minimize attack surfaces. Running dedicated user profiles prevents plugin conflicts and credential leakage. Jailed User Accounts – Creates restricted system profiles that only launch the browser, isolating extensions from core workstation operations. Shared Folder Synchronization – Establishes a single directory path bridging work and browsing users, allowing seamless file transfers without cross-contamination. Extension Audit Process – Leverages Chrome debug commands to enumerate installed plugins, verifying functionality before deployment on target networks. 06:15:00 Training Optimization & Audio Processing Accelerates mandatory compliance viewing through speed manipulation and automated transcription. Converting video content into searchable text enables rapid information retrieval. Global Speed Control – Increases playback rates up to sixteen times normal speed, drastically reducing time spent on repetitive corporate training modules. Whisper Diarization Pipeline – Downloads video tracks, separates speaker voices, and generates timestamped transcripts for quick reference during assessments. Download Management – Employs multi-threaded swarm downloaders and classic turbo managers to handle bulk media retrieval without interrupting active workflows. 10:40:00 Virtualization & Network Tunneling Protocols Establishes isolated testing environments using Windows virtual machines while managing connectivity constraints. Proper session handling prevents unexpected disconnections during remote engagements. Enhanced Session Mode – A Hyper-V feature providing higher resolution and shared clipboard functionality; disabling it is required before initiating certain VPN clients to avoid routing conflicts. Split Tunneling Mechanics – Routes specific traffic through the virtual network while keeping local resources accessible, preventing complete internet loss during connection tests. Certificate Verification – Identifies self-signed SSL mismatches early in the process, documenting them as preliminary findings before proceeding with authentication steps. 15:30:00 Macro Automation & Input Remapping Remaps frequently used keyboard shortcuts to reduce physical strain and accelerate command execution. Running scripts with elevated privileges ensures reliable input registration across virtual environments. Caps Lock Repurposing – Converts the caps lock key into a primary modifier, assigning copy/paste functions to adjacent letters for faster workflow navigation. Physical Typing Macros – Simulates keystrokes with deliberate delays, allowing seamless data entry into restricted VM consoles that block standard clipboard operations. Administrator Execution Requirement – Highlights that macro scripts must run with elevated privileges to successfully inject inputs across different desktop sessions. 20:15:00 Portable Development Environments & Python Management Deploys lightweight scripting frameworks that dynamically provision necessary tools without modifying host configurations. Verifying package contents prevents dependency conflicts during testing. Jamboree Framework – A PowerShell-driven utility that downloads and configures development stacks on demand, resetting environment variables to maintain system cleanliness. NuGet Package Filtering – Queries Microsoft's repository API to retrieve specific Python versions, ensuring compatibility with legacy tunneling scripts. Binary Verification Process – Checks extracted archives for bundled pip.exe or pip3.exe executables, eliminating manual module installation steps during rapid deployments. 28:40:00 AI-Assisted Scripting & Debugging Workflows Generates and refines PowerShell functions through iterative conversational prompts. Validating AI output against actual system behavior prevents silent configuration errors. Vibe Coding Approach – Relies on continuous feedback loops with language models to draft, minimize, and debug automation scripts in real-time. Parameter Standardization – Enforces strict formatting rules for PowerShell commands, avoiding hardcoded paths and ensuring cross-environment compatibility. Temporary Storage Management – Monitors extraction directories to prevent disk saturation, redirecting large package downloads away from constrained system partitions. 35:10:00 Terminal Emulation & Advanced Tunneling Strategies Facilitates complex network routing through dedicated terminal applications. Configuring dynamic and static tunnels enables reliable reverse connections for remote assessments. MOBA Portable Configuration – Utilizes an INI-based tunnel manager that automatically maintains connections across changing IP addresses or Wi-Fi networks. Reverse Shell Routing – Establishes outbound channels back to the tester, then proxies all subsequent traffic through those connections for consistent monitoring. Proxy Chain Integration – Forces non-proxy-aware applications to route through Burp Suite or custom interceptors using Windows utility wrappers like Priboxy. 42:30:00 Final Connectivity Testing & Engagement Wrap-Up Executes comprehensive port scans to verify target accessibility before documenting findings. Acknowledging workflow detours ensures realistic time management during active engagements. Nmap Verification – Runs full-port scans with verbose output to confirm host responsiveness and identify open services prior to credential testing. Connection Refusal Documentation – Captures screenshot evidence of failed routing attempts, providing clear proof of network restrictions for client reporting. Workflow Reflection – Recognizes that exploratory debugging adds value but requires time boundaries; balancing thoroughness with engagement scope maintains professional efficiency. Provide feedback on this episode.

This show has been flagged as Clean by the host. As a child, I was so fond of reading what I saw as beautiful. Even when above [lacking] the understanding. The perception of beauty came mainly from the cover and the fonts used ( same today ) or from the place the book occupied on my father's shelf. One of them was Tuareg. More than once I took this classic from the home library, only to glance at the first page with no interest. It was only decades later — some years ago — that I saw the title again, and read with pleasure and no difficulties. It's not a hard read, even to a 12 year-old, probably only the argument, the plot, didn't catch me by then. Anyway, I was attracted to the book, as if simply wanting to read it made me part of the classic, the intellectual. I have less of this today. [I mean,] Not the inability to read a lot of books, but the urge to be a consumer of certain cultural product. Maybe due to not greeding to build a personal image, content without exposition. Still, sometimes, wanting to consume a piece of media and being unable to.. I wanted a story of robots, the book I though was not available [(All Systems Red)], so I bought one of scientific fiction. Somehow it attracted me when I flipped the few pages of Planet of Exile at the bookstore, an item that could lead me to a travel outside of this place, an escape. At home, I immediately saw I couldn't grasp it. There were too many characters (too many = more than two in the first page, specially when another atmosphere is the location, a new physics to grasp). Too much information, can't retain the minimum necessary in the way to the second, third… fifth page, by when more is being presented. This was always a disability of mine, focused on a little tiny thing, intensely, for some time, all lost in some weeks or months. I know it happens to me , because people are able to remember the names of the cars of the movie Cars. Not an incredible feat for them (maybe they haven't paid attention, nor actively tried to remember, only did), and something strange to me. It was decided to register this because when I saw myself unable to read the [Ursula] Le Guin's book, I got nervous. I've already been forgetting more ultimately, take confused steps, and now I can't center on a book?! What's happening? But then I took a breath: much can be going way too wrong from current circumstances outside and inside, but your limited comprehension of the world (and its art) is very part of you. This trace, or frailty, of your character, is not your brain rotting, it's part of what you have been since always. Thank you. Provide feedback on this episode.

This show has been flagged as Clean by the host. In his early days as a writer, Heinlein wrote his stories in the context of a shared universe that he called the Future History. These were mostly short stories at first, with the occasional novella. But they include some great stories. The Future History, Part 2 There were a few key themes running through Heinlein's body of work. One we have already remarked upon, individual freedom, which had to be protected from any source of power, including both government and private corporations. This was essentially a libertarian perspective, but unlike many of today's libertarians he was equally averse to the corporate type of power as a threat. But he had a complex view of the world which has resisted some attempts to pigeonhole him. He started out as a socialist, and while he didn't remain one, he never became a knee-jerk reactionary either. In fact, he clearly despised them just as much. One way of looking at his body of work is that he explored the ramifications of different social policies through his stories, but in most cases the needs of a good story came first in the early years. In his later works he often surrendered to the temptation to pontificate, which reduced the enjoyment of them somewhat for anyone who was not already in agreement with his opinions The second major theme you see throughout all of his works is the idea of the competent individual. He admired anyone who could do a job well, and clearly did not care whether they were man or woman, nor black or white. Alexei Panshin writes, in Heinlein in Dimension: “There is one unique and vivid human Heinlein character, but he is a composite of Joe-Jim Gregory, Harriman, Waldo, Lazarus Long, Mr. Kiku, and many others, rather than any one individual. I call the composite the Heinlein Individual. . . . It is a single personality that appears in three different stages and is repeated in every Heinlein book in one form or another. “The earliest stage is that of the competent but naïve youngster. . . . The second stage is the competent man in full glory, the man who knows how things work. . . . The last stage is the wise old man who not only knows how things work, but why they work, too.” Harriman we have already encountered in The Man Who Sold The Moon, and the others appear later. The Heinlein Individual, as he is often referred to, appears in many of Heinlein's stories. A third major theme has to do with morality and religion. Heinlein grew up in what he considered the heart of the Bible Belt, in Missouri, and saw first-hand how the evangelical Christians operated, and despised what he saw. As someone who believed in individual freedom, he could never surrender to someone else's idea of how he should live his life. He saw them as a danger to his ideal libertarian society, and this shows up very early in his work. He personified the good, upright, church-going folk as “Mrs. Grundy”, and while you might want to draw the drapes to keep her from knowing what you were doing, you should never let her dictate how you would live your life. Revolt in 2100 begins the exploration of this in detail. There is a chart of the future history at Baen Books, and in it we see that the 1960s were what Heinlein called The Crazy Years. (Remember, he conceived this in the 1940s and 1950s.) But in 2012 the major thing occurred when Nehemiah Scudder, a backwoods preacher, managed to get elected as President. This would be the last election held under the U.S. Constitution as he established a religious dictatorship that lasted a couple of generations. IS this plausible? Heinlein wrote about this: “As for … the idea that we could lose our freedom by succumbing to a wave of religious hysteria, I am sorry to say that I consider it possible. I hope that it is not probable. But there is a latent deep strain of religious fanaticism in this, our culture; it is rooted in our history and it has broken out many times in the past. “It is with us now; there has been a sharp rise in strongly evangelical sects in this country in recent years, some of which hold beliefs theocratic in the extreme, anti-intellectual, anti-scientific, and anti-libertarian.“ His background in the Bible Belt is what informs a lot of his thinking. He goes on to describe how this might happen: “Throw in a Depression for good measure, promise a material heaven here on earth, add a dash of anti-Semitism, anti-Catholicism, anti-Negroism, and a good large dose of anti-“furriners” in general and anti-intellectuals here at home, and the result might be something quite frightening — particularly when one recalls that our voting system is such that a minority distributed as pluralities in enough states can constitute a working majority in Washington.” As the science fiction author David Brin points out, Heinlein accurately predicted much of what we are going through in the United States right now. There is an emerging dictatorship in the United States, promoted by right-wing religious groups. The “material heaven here on earth” is represented by the Prosperity Gospel, prominent in the Trump movement, and so on. Where the Prophet used a restored Ku Klux Klan as his muscle, we have The Proud Boys, and so on. It really does track very closely. Read David Brin's article for more on this. But nothing lasts forever. Empires rise and fall, governments change, and in this case a resistance movement arises. The revolt is depicted in the novella If This Goes On— (1940), and it is set in the year 2100, giving the title to the book. The main character is John Lyle, who is a young army officer assigned to the group protecting The Prophet in his capital of New Jerusalem. In the beginning he is thoroughly indoctrinated, but then begins to question his beliefs when he falls for one of The Prophet's virgins, Sister Judith. He has an older companion in the military who is not only unshocked when John confides in him about his doubts, but offers to help him. It turns out this companion, Zeb Jones, is a member of the underground group called The Cabal that is working to overthrow the theocracy. In the end they are successful, and in the course of this John Lyle does a lot of growing up. In this we see another common characteristic of Heinlein stories: a young, naive boy meets up with an older and wiser man who helps him to grow. In 2016 If This Goes On— won the Retro-Hugo Award for best novella of 1940. And in a personal note, I have T-shirt that says “Scudder for President 2012”. This baffles most people, but I enjoy the in-joke. What is interesting in this book is that Heinlein doesn't stop with a successful revolution. He then goes on in a second novella to describe the government that arose following the revolution, and this story is called Coventry (1940). The new government that arises after the revolution is called The Covenant, and it is an attempt to make sure that what happened with Scudder in 2012 could never happen again. It is a strongly libertarian government based on an agreement to be non-violent. In this society, scientists can cured criminal or violent tendencies, but any citizen convicted of such must agree to the treatment. The alternative to treatment is that they can be exiled to a place called Coventry. Coventry is outside of the Covenant society, and the Covenant society has nothing to do with them. Our protagonist, David McKinnon, is convicted of assault, and chooses to go to Coventry instead of getting treatment. He imagines it is a peaceful anarchy, but is disabused of this notion when he is robbed of all of this possessions upon entry and thrown in jail. A fellow inmate, Fader Magee, helps him escape, and we learn he is an agent of the Covenant government. They learn that two of the factions in Coventry have joined forces, and found a way to break through the barrier that surrounds Coventry. They plan to attack and overthrow the Covenant government. David and Fader separately work to escape and get back to warn the Covenant government, which they do successfully. And by doing this, David has demonstrated that he is no longer a danger to the Covenant society and no longer subject to treatment. This story won a Prometheus Hall of Fame Award, which is awarded by the Libertarian Futurist Society. And the Covenant society certainly has libertarian features. But this is not the Randian version of libertarianism, as exemplified by the fact that David is restored to the society because he demonstrated his concern for others. Heinlein always promoted individual freedom, but also the idea that people have a responsibility towards others. Finally, Revolt in 2100 contains the short story Misfit, w2hich we have looked at previously. The Past Through Tomorrow (1967) is a one volume collection of most of the Future history stories. I say most because just which stories belonged in this group could change from time to time. It also has the last version of the Chart of the Future History, and a few stories we have not yet mentioned (Methusaleh's Children, and The Menace From Earth). And many of his other works contain back references to these events that imply that they might be set in the same alternate universe. Heinlein gets the last word on this: “I have never been sure whether or not publishing that chart was a good idea or a bad mistake. Possibly it helped to sell some stories later—but certainly it caused me and still causes me to receive a lot nuisance mail from nitpickers. I have never felt bound by that chart; it was to serve me, not the other way around. If I found myself with a good story notion which fitted fairly well into the chart but not perfectly, I shed no tears—I went ahead and let the inconsistencies stand. I want each story to be internally consistent . . . but I won't let myself be painted into a corner through trying to fit that chart perfectly. I may start another “Future History” story tomorrow . . . and find that to make it a good yarn I must violate some item on that chart. I'll give the nitpickers something to pick, for I will not hurt a good yarn for the sake of “logic”—logic is not involved, as that chart is fiction, not Holy Writ.” Links: https://www.amazon.com/Heinlein-Dimension-Critical-Alexei-Panshin/dp/0911682015 https://web.archive.org/web/20151105170345/http://www.baenebooks.com/chapters/1439133417/1439133417___1.htm https://david-brin.medium.com/heinleins-future-history-coming-true-before-our-eyes-10356a95556a https://en.wikipedia.org/wiki/%22If_This_Goes_On%E2%80%94%22 https://en.wikipedia.org/wiki/Coventry_(short_story) https://en.wikipedia.org/wiki/Prometheus_Award#Hall_of_Fame_Award_inductees https://en.wikipedia.org/wiki/The_Past_Through_Tomorrow https://www.palain.com/science-fiction/the-golden-age/robert-a-heinlein/the-future-history-part-2/ Provide feedback on this episode.

This show has been flagged as Clean by the host. Kevie, Paul and Dave are back once again with the HPR Beer Garden, this time focusing on the relatively new style of beer called Double IPA. Kevie samples Man Overboard by Cromarty Brewing , Paul tastes Rumble by Beavertown and Dave opts for Ultra Phase by North Brewing Connect with the guys on Untappd: Dave Paul Kevie The intro sounds for the show are used from: https://freesound.org/people/mixtus/sounds/329806/ https://freesound.org/people/j1987/sounds/123003/ https://freesound.org/people/greatsoundstube/sounds/628437/ The next 3 beer styles to be reviewed: Belgian Blonde DDH IPA Amber Ale Provide feedback on this episode.

This show has been flagged as Clean by the host. 01 Introduction In this episode I will describe how to calculate elapsed time in bash or other shell scripts. While this may sound like a very simple and basic thing to do, there is a slightly more complex aspect to it if you wish to calculate elapsed time to a higher resolution than one second. 02 There are many reasons for calculating elapsed time in a shell script. For example you may wish to simply report how long an operation took to run. Another reason may be that you are trying to speed up a script and need to calculate benchmark data to see how different alternative methods perform. 03 What may seem like a simple task gets a bit more complicated if you want to do it for multiple different operating systems even if they are all unix related, as we shall see. -------------------- 04 Operating Systems Tested For the purposes of this episode, I ran tests on the current version of the following operating systems. Alma Alpine Debian FreeBSD OpenBSD RaspberryPi OpenSuse Ubuntu 2604 Alma is a close copy of Red Hat that we can take as representing Red Hat style distros. -------------------- 05 Simple Low Resolution Timing I will start with the simple and obvious method before describing the less obvious ones. This uses the date command to get the current time in seconds since the unix epoch. This is simply date '+%s' 06 Save this to a variable using whatever method you prefer. For example. starttime=$(date '+%s') 07 Next, do whatever operations it is you wish to time. Use the date command to get the current time again. endtime=$(date '+%s') 08 Now simply subtract the start time from the end time using shell arithmetic. This should be very obvious and basic. -------------------- 09 Higher Resolution Timing However, suppose we wish to measure time to greater than one second of precision. We need to do two things. The first is to obtain the current time at a higher degree of precision. The second is to conduct the calculations to a higher degree of precision. 10 Unfortunately, the standard time precision for POSIX shells seems to be 1 second. Some shells offer a higher precision, but others do not. Furthermore, standard shell arithmatic uses integer, which limits calculations to 1 second of precision. -------------------- 11 Bash High Resolution Shell Variable Fortunately, bash is one that does offer a high precision date. If you are using bash 5.0 or newer, there is a shell variable called EPOCHREALTIME which offers time since the the unix epoch (that is, since the first of January 1970, at 00:00:00 UTC) in seconds to 6 decimals of precision. 12 Example echo $EPOCHREALTIME 1779634800.184926 13 This is related to the similar bash variable known as EPOCHSECONDS which gives the number of seconds since the unix epoch. 14 Example echo $EPOCHSECONDS 1779634800 15 So if you are using bash, measuring time is very simple. -------------------- 16 But is it Really Bash? Is your script however actually using bash? Debian and derivatives actually have two shells. The first, the interactive shell is bash. The second, the non-interactive shell is dash, which stands for "debian almquist shell". 17 If you open a terminal, you get bash. If your script starts with a "bin/bash" shebang line, you get bash. However, if your script starts with a "bin/sh" shebang line, you get dash. Some people find themselves getting caught out by this one when they try something out in a terminal but find that it doesn't work in their script which started with "bin/sh". 18 Many other, but not all, Linux distros use bash for both the interactive and non-interactive shells, so "bin/sh" and "bin/bash" work the same with those ones. So if you intend to use bash, make sure your script calls for bash in the first line. -------------------- 19 The SHELL Variable So how can a script tell what shell it is running under? There is a shell variable called "SHELL" which will tell you the name of the shell. Well, sort of. 20 On Debian and derivatives "SHELL" will say "bash" regardless of whether the actual shell is bash or dash. On some other operating systems "SHELL" will simply say "sh" even if it is something else entirely. So we need to do some additional levels of checking to see what we have. 21 To start with though, here's what each of the test distros reports for SHELL. Alma : bash Alpine : sh Debian : bash FreeBSD : sh OpenBSD : ksh Raspberry-Pi : bash Suse : bash ubuntu2604 : bash -------------------- 22 Bash Versus Dash First, let's try to see which ones are bash and which ones are dash. The first thing we can check is for the shell variable BASH_VERSION. 23 Example echo $BASH_VERSION If the shell is bash, then it will report a version string. If the shell is not bash, then it will return an empty value. 24 Using this test, we can see that Alma and Opensuse are indeed using bash. We however need to check Debian, Raspberry Pi, and Ubuntu when running in an "sh" script. To check this we can use the "which" command to see what "sh" actually is. 25 Example echo $( ls -l $(which sh ) | rev | cut -d" " -f1 | cut -d/ -f1 | rev ) 26 "which sh" shows us the path to "sh" However, that is a link so we need to use "ls -l" to find the actual executable. "rev" reverses the string. 27 "cut" takes the first element separated by spaces. The second "cut" takes the first element separated by the "/" characters. The final "rev" takes that string and reverses it again to get it in the correct order. 28 In the case of Debian, Raspberry Pi, and Ubuntu it tells us that this is "dash". -------------------- 29 Openbsd Openbsd reports its shell as "ksh", which stands for Korn Shell. It is indeed Korn Shell, so we simply leave that one as is. -------------------- 30 Alpine and Freebsd Next we have Alpine Linux and Freebsd, which both report as "sh". In the case Freebsd there doesn't appear to be any further we can go that I am aware of. It's simple "sh". It is a basic POSIX shell which seems to be similar to the original unix shell, the Bourne Shell. Older versions of Freebsd used a different shell known as tsch (the C shell), but I haven't tested that so I will ignore that here. 31 With Alpine Linux however, we can get the actual shell using the same method that we used for Debian Linux. This reports as being "busybox". 32 Busybox is a limited shell intended for use in embedded systems. Alpine was originally an embedded distro, but some people started using it for containers. Alpine is Linux, but it is not GNU/Linux, and there are a number of areas which can trip you up if you are not aware of them. So, be extra careful if you are using it for anything, and test everything. -------------------- 33 Summary of Actual Shells Here is our revised list with the actual shell used when asking for "sh", so far as we can determine. Alma : bash Alpine : busybox Debian : dash FreeBSD : sh OpenBSD : ksh Raspberry-Pi : dash Suse : bash ubuntu2604 : dash 34 There are other shells, but none of them are the default shell for any of the distros on our list, so I haven't tested them. -------------------- 35 Solutions for Measuring Time Now we need to find solutions for bash, dash, ksh, sh, and busybox. -------------------- 36 Bash For bash, we can simply use EPOCHREALTIME, as mentioned above. -------------------- 37 Dash For dash, we can use the date command. This is a very conventional method, and is probably the first answer that anyone would give for this situation. However, while it will work in most cases, it will not work in all cases, so it is not a universal solution. 38 To use date we simply call it with the correct format string. This uses %s to get seconds since the epoch, and %N to get nanoseconds of the current second. If you put a decimal separator between the two it will appear in the output. You can use the correct decimal separator for your locale, but I won't go into that here. Instead I will just assume a period or dot. 39 Example date '+%s.%N' 1779634800.358916385 -------------------- 40 Problems with Date on Alpine and Openbsd Date will work for bash, dash, and sh on Freebsd. However it will not work for ksh on Openbsd, or for busybox on Alpine. 41 With busybox on Alpine, it simply ignores the %N format specifier and prints out the epoch in seconds only followed by the decimal separator. = Example date '+%s.%N' 1779634800. 42 With ksh on Openbsd it prints the epoch in seconds followed by the decimal separator and then the %N as a literal N. date '+%s.%N' 1779634800.N 43 Fortunately we have alternatives for these two cases. -------------------- 44 Openbsd Openbsd has the "ts" or timestamp utility installed by default. ts prints a time stamp in front of every line it receives from standard input. I won't go into details on all aspects of ts here, I'll leave that to someone else. Instead I will focus on how to use it for our specific purposes here. 45 We need to provide a format specifier to ts, which in this case is "%.s" We also need to provide something for standard input, or otherwise ts will simply sit there and wait for input. So what we need to do is to echo nothing through a pipe to ts while also giving ts the proper format specifier. 46 Example echo | ts "%.s" 47 This will output the epoch time in seconds to six decimals of precision. ts is installed in Openbsd and Freebsd by default and can be used in either. It can also be installed in many other distros. -------------------- 48 Busybox on Alpine None of the methods discussed so far will work for busybox on Alpine though. However there is a way, but it's a bit non obvious and somewhat hacky. 49 Busybox includes a command called "adjtimex". This is normally used to adjust the time hardware. However if it is run without arguments, it will report the current settings. 50 These include the current epoch time in seconds , and in another field the time in microseconds. These are reported as key value pairs. So what we need to do is to do the following 51 Run adjtimex Capture the output. Grep for "time.tv_sec" Grep for "time.tv_usec" Use cut to extract the time value in each case. Use tr to get rid of excess spaces in each case. Combine the two in a string with a decimal separator between them. 52 This takes a total of 4 lines of shell script. I will just describe them breifly here, see the show notes for details. 53 First we want to capture the output of adjtimex in a single operation. Run adjtimex and pipe the output through grep to capture lines containing "time.tv_" and save this to a variable. # Extract the current high resolution time from adjtimex. # We want two key value pairs, identified by time.tv_sec and time.tv_usec. tvals=$( adjtimex | grep "time.tv_" ) 54 Next echo the contents of this variable and pipe it through grep, cut, and tr to get first the seconds and then the microseconds while also removing excess spaces. Save these to two separate variables. "time.tv_sec" is the time in seconds since the epoch. "time.tv_usec" is the number of microseconds in the current second. # Get the time since the unix epoch in seconds and micro-seconds. timesec=$( echo "$tvals" | grep "time.tv_sec" | cut -d: -f2 | tr -d " " ) timeusec=$( echo "$tvals" | grep "time.tv_usec" | cut -d: -f2 | tr -d " " ) 55 Adjtimex does not zero pad the microsecond time value to provide leading zeros, so we need to take care of this using printf before we can append it to the seconds value. We didn't need to do this with date where the %N format character does this automatically. In this instance, the printf format string is '%06d' padusec=$( printf '%06d' $timeusec ) Now, combine these into a single number with a decimal separator by using simple string concatenation. # Combine them into a single number. timehires="$timesec"".""$padusec" -------------------- 56 Summary of Methods Let's summarize where we are so far in terms of methods we can use to get the current time as a high resolution number. Alma : use EPOCHREALTIME or date Debian (bash) : use EPOCHREALTIME or date Raspberry-Pi (bash) : use EPOCHREALTIME or date ubuntu2604 (bash) : use EPOCHREALTIME or date Suse : use EPOCHREALTIME or date Debian (dash) : use date Raspberry-Pi (dash) : use date ubuntu2604 (dash) : use date Alpine : use adjtimex and parse the output FreeBSD : use date or ts OpenBSD : use ts -------------------- 57 Other alternatives There are a few alternatives that we haven't discussed yet. 58 Bash with Dash In the case of Debian, Raspberry Pi, and Ubuntu running dash, since bash is available it is possible to write a separate bash script which simply echos EPOCHREALTIME and then call it from the dash script and capture the output. While this would work, there's probably not a lot of point to it. If you can rely on bash being there, then just change the first line of the script and make it a bash script. 59 Adding Packages to Alpine The ts or timestamp utility is a common unix utility that can be installed if it is not present by default. This does produce high resolution timestamps on Alpine. On Alpine Linux this comes as part of the "moreutils" package. To add the package, use the following sudo apk add moreutils echo | ts "%.s" 1779634800.959948 60 You can also add the GNU coreutils, which will provide a high resolution date command which works like in the other examples. To add the package use the following sudo apk add coreutils date '+%s.%N' 1779634800.212897332 61 If you can install more packages into your Alpine system, either of the above two is probably going to be preferable to parsing the output of adjtimex. 62 Custom Timestamp Programs You can also write a very short program in python, perl, tcl, or some other language and have it output the current epoch time. I won't discuss that here though. -------------------- 63 Calculating Time Differences Shell arithmetic is integer only. If we wish to use high resolution timing data, we need to do something so we don't lose the precision we have worked so hard to get. There are several possible solutions. 64 Change the Time Base One method is to change the time base from seconds to milli, micro, or nanoseconds. This can be done by simply multiplying the time values by the appropriate amount (e.g. 1000, 1,000,000, etc.) before subtracting them. This allows for integer arithmetic on high resolution values without losing precision. 65 Use the Shell bc Arbitrary Precision Calculator The bc command line calculator will perform calculations using real numbers and is easy to use in scripts. It is present by default in most distros. echo "scale=9; $endtime - $starttime" | bc where endtime and starttime are variables containing time values. 66 However, for some inexplicable reason, neither Debian nor Opensuse install it by default. It is present in Ubuntu and Raspberry Pi which are Debian derivatives, and it can be added to distros which lack it. 67 Use awk awk can also perform calculations using real numbers and it is present in nearly all distros including in all of the ones we tested here. echo "$endtime $starttime" | awk '{printf "%.6fn", $1 - $2}' -------------------- 68 Benchmarks And of course no comparative evaluation would be complete without benchmarks where we see how each method compares to another in terms of speed. In the benchmark test I ran each method in a loop through multiple iterations, measured the elapsed time, subtracted out the time for an empty loop, and then compared it to alternate methods. For anything other than EPOCHREALTIME, the empty loop time is negligible and has no real effect on the results. 69 Rather interestingly I came across a bug which caused date to run very slowly if called immediately after using EPOCHREALTIME in bash. The effect of the bug was to make the date benchmark test roughly 24 times slower. This has been fixed in newer releases, but if you are using an older distro release then beware of this bug. I was able to get around it either putting a sleep delay between benchmarking EPOCHREALTIME and benchmarking date, or by simply testing date before testing EPOCHREALTIME. 70 To be able to conduct additional tests I installed ts in Ubuntu and Alpine, and the GNU version of date in Alpine. 71 EPOCHREALTIME Versus date in Ubuntu 2604 bash The EPOCHREALTIME method is 3103 times faster than date. However, when the same test is run on Ubuntu 2404 when the date test is run before the EPOCHREALTIME test, EPOCHREALTIME is 1240 faster than date. Other Linux distros show performance to Ubuntu 2404. It appears that a side effect of fixing whatever the bug is has the effect of slowing down date. However, this is probably not a significant issue in normal circumstances. 72 date versus ts in Ubuntu 2604 bash The date method is 3.7 times faster than ts 73 date versus ts in Ubuntu 2604 dash The date method is 4.9 times faster than ts 74 date versus ts in Freebsd sh The date method is 2.5 times faster than ts 75 date versus adjtimex in Alpine Busybox The date method is 6.0 times faster than adjtimex 76 date versus ts in Alpine Busybox The date method is 20.0 times faster than ts 77 bc versus awk in Ubuntu 2604 I compared calculating the difference between two numbers when using bc versus awk. The difference is negligible, with bc being only 7% faster than awk. 78 Conclusion for Benchmarks Based on these results, if you need to measure elapsed time to high resolution and care about runing the command with as little overhead as possible, then the order of preference should be the following. 79 If you are using a newer version of bash, then use EPOCHREALTIME. If that is not available, then use date, provided it allows for high resolution times. If the above two cannot be used, then use ts. If you are using Busybox and cannot install either GNU date or ts, then use adjtimex. Date is the closest in terms of being the universal portable solution, but it does not work in all cases. 80 I have not compared different platforms to each other in terms of performance, as that would be a much more involved problem that is outside the scope of this episode. However, different operating systems implement different commands in different ways. 81 For example, on Openbsd and Freebsd, ts appears to be an ELF binary. That is, it is executable machine code, possibly written in C. On Ubuntu however, ts appears to be a perl script. As a result of this, the advantage that date has over ts is much less in Freebsd than it is with Ubuntu (and likely other Linux distros) as on Freebsd it doesn't need to load a perl interpreter to run ts. -------------------- 82 Overall Conclusion You no doubt thought that measuring elapsed time was going to be so simple, and how could someone get an entire podcast out of such a simple subject? And yet here we are half an hour later with just a basic overview of the subject. 83 I hope you found this interesting and informative. Please let us know in the comments if you think that I have done anything incorrectly, or if you have another way of doing things. I hope to see you all again in another future episode of HPR. -------------------- Provide feedback on this episode.

This show has been flagged as Clean by the host. This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems. In UNIX Curio #8 ( HPR episode 4657 ), I talked about using standard utilities to compare files. Left unmentioned, however, was a method commonly used today—the hash function. As I've stated in previous entries, while I am an engineer, I don't have a background in computer science, so my understanding of the mathematics is limited. But I can give a practical description of what a hash function does. It takes an input, performs a set of calculations on it, and produces an output. As hash functions are practically used, the input is a set of bytes, such as a file or another piece of data like a password. The output is a numerical value in a fixed range—most often, expressed as hexadecimal characters. Because this "hash value" can always be represented in a certain number of bytes, its length as printed is usually a constant number of characters, padded with leading zeros if necessary. This episode will not cover the use of hashes in programming, focusing instead on using them to validate data. A hash function, or more specifically, a cryptographic hash function, has an additional property. It should be very difficult to predict what changes to the input would be required to produce a specific change in the output. An older, related concept is called a "checksum". While these are designed to vary when the input data is damaged or digits are transposed, they do not necessarily have that last property mentioned for cryptographic hashes. You have probably already encountered a checksum, even if you didn't recognize it. On a 16-digit number assigned to a Mastercard or Visa 1 credit or debit card, the first six digits identify the card issuer (such as a bank), the next nine digits are assigned to you by the issuer, and the last digit is a check digit. The check digit is calculated using the values of the previous 15 digits, and it is a simple way to avoid typos in entering a card number. In another example, every Ethernet frame that your devices send or receive includes a checksum 2 to help ensure that the contents weren't scrambled in transit. This is 32 bits long and is called a cyclical redundancy check, commonly referred to as a CRC. A CRC is also used in many other places—for example, the .zip file format includes one for each archive member, and this allows a program extracting files from the archive to identify if any were damaged. Our UNIX Curio for today is another example, the cksum utility 3 . It generates a 32-bit CRC based on the Ethernet algorithm. It operates on either a named file or standard input and outputs the CRC value, the length of the input, and the pathname if a file was given as an argument. Unlike most modern hashing programs, the checksum is printed as a decimal integer and is not padded, so it can be anywhere from one to ten digits long. The length value is the number of bytes in the input (actually specified as the number of octets , as systems could potentially use a byte that isn't eight bits long), also expressed as a decimal integer. There are two major ways that one could use cksum to check the validity of a file. First, if you are transferring a file from one UNIX-like system to another, you could run cksum against it on both systems and check that the CRC and length are the same. The utility can also be given multiple filenames as arguments, which would generate a list that can then be compared. The second way would be for someone publishing a file or set of files to also publish the CRC values, lengths, and names so that people downloading them could verify that they match. However, I don't think the practice of publishing lists like this really started until more recent hash functions like MD5 and SHA-1 came about so it is unlikely that anyone would publish CRC values instead. The advantage of these tools should be pretty obvious in comparison to cmp , one of the utilities discussed in UNIX Curio #8. To verify a file using cmp , you need two files to compare—if you're trying to check a large file you downloaded, you would need to spend the time and bandwidth to download a second copy. And if they didn't match, you would have no idea which of the two, if either, was correct. By contrast, cksum is quicker to run, doesn't require downloading a massive amount of excess data, and if run against the original file, makes clear what the correct value is. This utility is a follow-on from a program called sum , which operated very much the same. I had a bit of trouble tracking down the exact development history, but what seems clear is that two different variants 4 were popular: a BSD version and a System V version. Both output 16-bit checksums, but used different algorithms so they didn't give the same results. Also, the BSD version printed the length of the input data as the number of 1,024-byte blocks, while the System V version instead gave a count of 512-byte blocks. (Some sources claim that System V sum generates a 32-bit checksum 5 , which could possibly be true internal to the algorithm, but I have tested several independent implementations of the utility and all of them output a 16-bit value for both the System V and BSD algorithms.) From what I can tell, the BSD version 6,7 came first; it was in 3BSD but probably appeared even earlier. An identical copy of BSD's sum was included with UNIX/32V 8,9 , which was AT&T's 1979 port of Seventh Edition UNIX to the VAX and became one of the ancestors of System III. The divergence seems to have started with System III, released in 1980; its version of the sum utility 10,11 changed to a new default algorithm, though it could be made to use the BSD algorithm via the -r option. System V looks to have kept the same behavior as System III. It's not clear to me why this algorithm is universally called the "System V algorithm" rather than the "System III algorithm"; perhaps it is because System V saw much more widespread use. Instead of trying to reconcile these differences, the POSIX committee decided to create a new utility with a unique name, use a separate algorithm entirely, and avoid the block-length dispute by printing the length in octets instead of blocks. I should point out that POSIX states that the CRC algorithm for cksum does not strictly meet the mathematical definition of a "checksum". I don't know enough to say exactly why it doesn't qualify or to say whether either of the sum algorithms do. However, in less-formal usage the term "checksum" has gathered the meaning of any value used to represent or validate a set of data, so I am fine with using it no matter the technical details of the algorithm. When two different inputs produce the same checksum or hash value, this is called a "collision". Because the output value has a limited range, there are an infinite number of possible inputs that could produce a collision. From a practical standpoint the possibilities are more limited—the majority of these inputs are larger than the number of atoms in the universe, which can't fit on any machine. Unlike a cryptographic hash algorithm, the CRC is not specifically designed to resist an attacker crafting a malicious input that would cause a collision. However, it should be sufficient to detect accidental damage. Programs implementing more modern cryptographic hash algorithms are superior to the checksum utilities in avoiding collisions (whether malicious or accidental), but there are still three advantages that the older programs have. First, a system running a historical operating system might not have the hash programs available, but is more likely to have cksum or sum already included. Second, the checksum values are much shorter than the hashes output by the newer programs, making them easier for a user to compare by looking at them. This advantage is not as great as it might appear at first, because a common way to check a hash these days is to save a list of hashes and filenames—the hash programs can use that and do the comparison themselves, sparing the user from having to validate it character by character. The third advantage is that cksum prints the input length in bytes. This greatly limits the number of inputs that could be maliciously crafted to create a collision. I did a moderate amount of research on implementations of modern cryptographic hash algorithms and found that some, such as MD5, SHA-1, and SHA-2, do use the length of the input (often termed "message length" in the literature) as part of the material fed in to the algorithm, but none of the hashing utilities present this length to the user as part of its output. There are two possible reasons for this that seem evident to me. First, if one is hashing a password, you would certainly not want to give a clear indication of its length—that would give any attacker a massive head start on guessing the password. However, that doesn't explain why one would avoid printing the input length for a file that is made publicly available. Second, it is convenient in many contexts, such as database entries or in software (such as git ), for the hash to be a fixed length. Including an extra value that can be of variable length would complicate those use cases. However, the length value could simply be dropped and they would be no worse off than they are currently. Historically on UNIX, password hashing was treated differently from checksumming files— the crypt() function 12 was used for passwords while sum and later cksum were used to confirm a file's integrity. So even rather early on, these two use cases employed algorithms with different properties, but I haven't dived into the history deeply enough to know how intentional this was. My discussion in this episode focuses on the file use case, so understand that I'm largely avoiding the topic of password hashing. Digital signatures are yet another use case, one that I'm ignoring entirely. Every few years, some security researcher declares a particular hash algorithm to be "broken" and that everyone should move over to a new one, which generally has a longer hash. While the larger hash space certainly reduces the opportunity for collisions, this disrupts workflows, such as publishing information about software releases by e-mail, which still tends to observe a 78-character limit on each line 13 , making it harder to include a list of hashes with filenames next to them. This is in addition to the work of modifying software and scripts to use the new algorithm and managing how to treat past data. It seems to me that publishing the input length along with the hash would make it far more difficult to craft a malicious input that matches both, but I haven't found discussion of that during my investigation. (See the Appendix for a possible implementation.) Perhaps someone listening can record a response episode for HPR explaining that. References: Payment card number https://en.wikipedia.org/wiki/Payment_card_number Ethernet frame: Frame check sequence https://en.wikipedia.org/wiki/Ethernet_frame#Frame_check_sequence Cksum specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/cksum.html GNU coreutils manual: sum https://www.gnu.org/software/coreutils/manual/html_node/sum-invocation.html FreeBSD 15.0 sum manual page https://man.freebsd.org/cgi/man.cgi?query=sum&sektion=1&manpath=FreeBSD+15.0-RELEASE+and+Ports 3BSD sum manual page https://www.tuhs.org/cgi-bin/utree.pl?file=3BSD/usr/man/man1/sum.1 3BSD sum source https://www.tuhs.org/cgi-bin/utree.pl?file=3BSD/usr/src/cmd/sum.c UNIX/32V sum manual page https://www.tuhs.org/cgi-bin/utree.pl?file=32V/usr/man/man1/sum.1 UNIX/32V sum source https://www.tuhs.org/cgi-bin/utree.pl?file=32V/usr/src/cmd/sum.c System III sum manual page https://www.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/man/man1/sum.1 System III sum source https://www.tuhs.org/cgi-bin/utree.pl?file=SysIII/usr/src/cmd/sum.c Crypt specification https://pubs.opengroup.org/onlinepubs/009695399/functions/crypt.html RFC 2822: Internet Message Format: Line Length Limits https://datatracker.ietf.org/doc/html/rfc2822#section-2.1.1 OpenSSH 10.1 released https://lwn.net/ml/all/dd12623ae86aa5eb@cvs.openbsd.org/ Appendix The MD5 hash algorithm was (and still is) widely used, but many people characterize it as being "broken" and discourage its use. Let us imagine a variant of this, called MD5.L, where the normal MD5 hash is followed by a "." character and the input length expressed as a hexadecimal number. Take, for example, the e-mail message announcing the release of OpenSSH 10.1 14 . At the bottom, it includes an SHA-1 hash and an SHA-2 256-bit hash for the available gzipped tar files. That longer hash is encoded with Base64 because if it were given as a hexadecimal number, it would make the line longer than 78 bytes. The MD5.L hash of the file would be one character shorter than the SHA-1 hash, as shown below. (The extra length of the name makes them both consume the same number of characters. The hashes shown are for the "portable" version of OpenSSH.) Some people claim SHA-1 is also broken, seeking to have people use newer and longer hash functions. For an attacker to compromise MD5.L in this example, they would not only have to create a valid tar file compressed with gzip containing a malicious payload having the right MD5 hash, that file would have to be exactly 1,972,831 bytes long (the decimal equivalent of 1e1a5f). While there are still many possible inputs that could be tried (256 1972831 , to be exact*), this is far fewer than the infinite possibilities for plain MD5, SHA-1, or SHA-2. If for some reason it is super important to have a fixed hash length, let's imagine another variation called MD5+L. In this one, instead of L being the input length, it is the input length modulo one terabyte (2 40 bytes), which can be represented by 10 hexadecimal characters, left-padded with zeros. While this approach substantially increases the number of possible inputs an attacker could try, it is likely that an intended victim would notice that the file they downloaded is larger (or smaller) than expected by that much. The MD5+L hash is longer than a SHA-1 hash, but still shorter than a 256-bit SHA-2 hash. SHA1 (openssh-10.1p1.tar.gz) = 7fd17b99d1beffb47cd380d64079e920bb0bd91f SHA256 (openssh-10.1p1.tar.gz) = ufx6K4JXlGem8vQ+SoHI4d/aYU3bT5slWq/XAgu/B1g= MD5.L (openssh-10.1p1.tar.gz) = 80dd9bb00a86519934710d05903fdf07.1e1a5f MD5+L (openssh-10.1p1.tar.gz) = 80dd9bb00a86519934710d05903fdf07+00001e1a5f Of course, if MD5 is considered to be too weak even with the inclusion of the length, one could produce a ".L" or "+L" version of any hash function. However, longer hashes will end up running into the 78-character limit. *This is a number with 4.75 million digits that the bc utility on my laptop took almost 5 minutes to calculate. Provide feedback on this episode.

This show has been flagged as Clean by the host. -------------------- 01 Introduction This is the second follow up to my 8 part series on nuclear power. In this episode I will attempt to answer a question posed by brian in ohio in a comment on HPR4583. In that comment he said: 02 -------------------- Loving this series. Maybe Whiskey Jack could give some cost comparisons between large and small reactors. He could also give us a realistic look at nuclear plant safety/accidents compared to conventional power production. Looking forward to the episode on FORTH generation reactors ;-) -------------------- 03 End of quote. The first question I answered in my previous follow up, which was HPR4628. In this episode I will attempt to answer the second question, which was about the safety of nuclear power compared to other sources of electrical power generation. One of the HPR janitors encouraged me to make this episode, so I think we can thank him for getting another HPR episode made. 04 Defining the Scope First, let's define the scope of the question. This will cover electrical power generation only. Within that scope I will consider only the following sources of energy. 05 Coal Oil Natural Gas Hydroelectric Nuclear Wind Solar I won't cover geothermal, wave, or tidal power as these are only used in very small amounts and so there simply isn't enough literature on them to base a discussion on . 06 Foreshadow Conclusion I should mention right away that I cannot provide absolute answers to this question in the form of a nice, neat ranking table based on numbers from peer reviewed scientific sources. The reasons for this will become apparent, but to put it briefly, the data on which to base such a ranking simply doesn't exist. I will however provide context within which people can think about the issue. Wherever possible, I will provide links to the references that I used in the show notes so you can read further on this yourself. -------------------- 07 Energy Catastrophism versus Energy Uniformitarianism First though I need to go off on a slight geological detour in order to explain an important analogy that I will use. 08 In the 19th century there was a great debate among geologists over what is known as catastrophism versus uniformitarianism. In seeking to explain the origins of the earth and of the landscape that we see around us, there were two points of view. 09 One was "catastrophism". This is the belief that the mountains, valleys, and plains that we see around us were formed as a result of great catastrophes which occurred relatively recently in earth's history. This explanation was necessary in order to fit geological features into an earth that was believed to be only a few thousands of years old. This view was heavily influenced by religious belief. In this view Noah's flood was the great catastrophe and the fossils of dinosaurs were the remains of animals who had not been saved on the ark and so had died in the flood. 10 The other point of view was uniformitarianism. This was the hypothesis that the landscape we see around us can be explained by the very slow accumulation of very small changes over very long periods of time. For this to be true however, the earth had to be far older than the few thousand years that a literal reading of the bible would suggest. The earth in fact had to be many, many, millions of years old. 11 Eventually, the uniformitarian view won out and people understood that while some catastrophes can take place, the shape of the landscape is overwhelmingly due to small changes over very long periods of time. 12 How is this Relevant to this Episode You Ask? How this is relevant is that I will use this analogy to explain how we need to think about energy and safety. Very small numbers of deaths and injuries multiplied over many occurrences can add up to big numbers, comparable in scale or possibly even larger than a single catastrophe or even several of them. 13 I don't know if anyone else has used this analogy before, I have just thought of this when writing the script for this podcast. None the less, I think it is a very useful way of helping to understand the issues. 14 As an example of this, think about the well known case of the safety of flying versus the safety of travelling in your car. Air crashes are catastrophes that make the headlines. Automobile crashes are seldom more than local news at best. You have probably heard many times the claim that if you making a trip somewhere, you are safer to fly than to drive yourself in your car. 15 Example - Hydro versus Solar I will now present an example of this. Hydro electric power has some notable large scale catastrophes associated with it. Roof top solar power does not have any notable catastrophes that I am aware of. However, which is safer? 16 Hydro Catastrophes Here are three examples of hydro electric catastrophes in just one country, Italy. The Vajont Dam which collapsed in1963 An estimated 1,917 to 2,500 people died. The Sella Zerbino dam which collapsed in 1935. More than 100 people died. The Gleno Dam which collapsed in 1923. An estimated 350 people died. https://damfailures.org/ https://pmc.ncbi.nlm.nih.gov/articles/PMC4997708/ 17 I haven't tried to compile a global list of the worst hydro electric dam collapses, as this sort of information is actually very difficult to find, even on web sites dedicated to dam failures. An additional problem is that information on whether a dam was used for electric power generation or not is often not available. 18 Dam failures where contradictory or insufficient information is available on whether there was an associated hydro power plant include the 1975 Banqian Dam failure, where death estimates range up to a quarter of a million. 19 Solar Panel Slow Accumulation Contrast this with roof top solar panels. Many small accidents can add up to big numbers as well. 20 Health and safety literature discussing solar panel safety mention things such as Falls from roofs. Electric shock. Arc flash (burns from electrical arcing). Normal electrical safety procedures which are based around locking out sources of energy do not work with solar panels which makes safety more difficult. Heat stress due to working exposed in the hot sun. Warning from US government on falls by solar panel installers. https://stacks.cdc.gov/view/cdc/228946 https://www.osha.gov/green-jobs/solar 21 Why We Cannot Compare the Two Hydro catastrophes are not well documented, but we can at least find records of some of the most notable ones. However, even those have very large variations in estimates of deaths. 22 Roof top solar deaths however are largely undocumented. The industry is largely unregulated. There is no central authority which accumulates many individual deaths or injuries. At best there are worker and public safety bodies who simply accumulate those statistics into general construction or household injuries. 23 Thus we have no reliable means of comparing the two energy sources on a comparable basis. We face the same problem with all other major electrical energy sources. So far as I am aware, there are no peer reviewed scientific studies which compare the relative safety of all of the major electrical energy sources we are considering here based on actual numbers. -------------------- 24 Safety Risks I will now try to list some the major hazards for each of energy sources we are considering. There is however limited data available. In many cases we just have reference to worker safety organizations as to what the hazards are. I will not attempt here to put numbers to these here. Categories 25 Coal, Oil, Natural Gas The hazards are Air pollution Mining and oil field accidents Pipeline explosions Transportation accidents. These- move a lot of material so these are significant. 26 Hydroelectric These include Dam collapse Drowning 27 Nuclear These include Radiation exposure 28 Wind These include Falls Confined space deaths (there is not much detail on this) Electric shock Ice throws (that is, throwing pieces of ice off the blades) This technology has a significant problem with people working alone which greatly increases risks associated with other dangers. 29 Solar These include Falls Electric shock Arc flash Heat stress 30 I have not tried to cover all possible risks associated with each category, just the ones which each industry considers to be the risks they concern themselves with. There does not exist any means by which risks of similar types are compared across different industries. 31 Reliability of Supply is Also Safety In a completely electrified net zero society, reliability of supply is a safety matter. People will die in very large numbers in cold climates if they do not have heat. If we have no fossil fuels, we need to also consider how reliably does a grid based on any of the options work. I have not seen anyone attempt to address this question and will not attempt to address it here. However, it must be addressed in any comprehensive attempt to rank safety. -------------------- 32 Studies or Articles on Estimates of Relative Safety Despite the difficulties of comparing the safety of different sources of energy, some people have attempted this anyway. Different estimates done at different times had different focuses, so unfortunately we do not have a nice set of studies that we can neatly use to cross check one another. I will however list the names and the authors and summarize the results. -------------------- 33 The Health Hazards of Not Going Nuclear By Dr. Petr Beckman Published in 1976 The author of this book tried to address the relative safety of different sources of energy in the mid 1970s. However, it is old at this point, so I won't bother digging through its pages to find his figures. 34 He mainly focused on comparing electric power generated with coal to nuclear. His conclusion was that if the goal was to prevent deaths or ill health in the process of generating electricity, then the logical conclusion was to replace coal fired power plants with nuclear. 35 The book was relatively well known at the time, as least as far as books on energy are concerned, so I thought it was still worth mentioning. I happen to have a copy of this book which I bought back in that time period It was the 8th printing of the book, so it would appear to have had relatively good sales. 36 The author did address the issue of what I have termed "catastrophism" in his comparison of different energy sources, although I don't know if he used this phrase. I don't know if he was the first to use this sort of analysis, but he certainly was very influential in terms of popularizing it. -------------------- 37 Risk of Energy Production by Herbert Inhaber Publication AECB 1119 March 1978 This study is a scientific paper from the same time period as the book "The Health Hazards of Not Going Nuclear". 38 He based his risk estimates largely on estimates of the amount of material which was used in the construction and operation of various power sources. While we could argue over whether or not this is a valid methodology, I think any such argument would be pointless as I think the age of the study alone renders it not relevant today anyway. Advancements in materials have changed the basis results significantly by now. However, as it exists I thought I would mention it to show that the idea of comparing energy sources to each other is not a new one. The author compared a wider variety of potential sources than Beckman did. 39 Here's his conclusions. He assumes equal amounts of energy produced by each method. The numbers are normalized such that the total sums to 100%. You can think of it in terms of what proportion of total deaths or injuries would result from each source if each were equally used. 40 Coal 27.5% Oil 25.6% Methanol 16.7% Wind 10.8% Solar photovoltaic 9.2% Thermal 8.1% Solar space heating 1.5% Ocean thermal 0.4% Nuclear 0.13% Natural Gas 0.08% 41 His natural gas estimate is drastically different from that of other authors. I am not going to worry about explaining it however, as the study is as I said old enough to be not very relevant anyway. I am mainly including this here out of historical interest. 42 As a footnote, the methanol he refers to would be synthesized from wood. This was a popular idea in that era as a means of providing liquid fuels for transportation. Practical battery electric cars in those days were strictly science fiction. 43 The ocean thermal category is a real blast from the past and I had forgotten all about that concept. It was a very popular idea at that time and was supposed to be *the* big and upcoming thing in renewable energy. It involved various means of attempting to extract energy from differences in water temperature at different depths in the ocean. It gradually faded away however, as despite great efforts being put into it, designs never proved to be practical. -------------------- 44 Electricity generation and health Anil Markandya, Paul Wilkinson Published in the Lancet, Vol 370, 15 September 2007 45 This is more recent than the previous one, although it is nearly 20 years old at this point. Unfortunately it doesn't cover wind or solar, just fossil fuels and nuclear. However it is still useful, and the Lancet is a very reputable peer reviewed journal. 46 I will present just the results rather than discussing the whole paper. The authors break it down into deaths among the public, occupational deaths, and air pollution related deaths, serious illness, and minor illness. 47 They break the energy sources down into lignite, coal, gas, oil, biomass, and nuclear. Lignite is a type of very low grade coal used mainly for electric power generation. In this paper biomass refers to energy crops and forest residues. 48 I will summarize the results by category rather than trying to describe a table that has 6 rows and 5 columns. All numbers are normalized in terms of deaths or cases per TWh. 49 Occupational deaths from accidents lignite 0.1 coal 0.1 gas 0.001 oil no data biomass - no data Nuclear is 0.019. 50 Deaths among the public from accidents lignite 0.02 coal 0.02 gas 0.02 oil 0.03 biomass no data Nuclear 0.003 51 Air pollution deaths lignite 32.6 coal 24.5 gas 2.8 oil 18.4 biomass 4.63 Nuclear 0.052 52 Air pollution serious illnesses lignite 298 coal 225 gas 30 oil 161 biomass 43 Nuclear 0.22 53 Air pollution minor illnesses lignite 17,676 coal 13,288 gas 703 oil 9,551 biomass 2,276 Nuclear no data 54 Natural gas edges out nuclear power slightly in terms of occupational safety, but in every other category nuclear is drastically lower in terms of ill effects than any of the alternatives. -------------------- 55 2020 Fatalities for US Roofers Increased 15% as Solar Roof Installations Increase Published in The Next Big Future July 6, 2021 by Brian Wang 56 This seems to be written by someone who has a popular science blog. I'm not familiar with it personally, but he addresses the subject so I'll list it. The title implies that it's all about rooftop solar, but he provides comparative numbers for the other energy sources of interest, so that is useful for our purposes. However, he doesn't describe his methodology, so we need to treat them with some caution. Here are his results These are deaths per thousand terawatt hours. 57 Coal - 100,000 Oil - 36,000 Natural gas - 4,000 Hydro - 1,400 Rooftop solar - 440 Wind - 150 Nuclear - 90 58 If we plot these numbers on a bar chart, coal and oil are so large that all of the others are squished to the bottom of the chart and are difficult to see at all. Let's therefore look at these in terms of orders of magnitude. Keep in mind that this is a logarithmic scale. This means that the difference between 4 and 5 is much greater in linear terms than the difference between 1 and 2. 59 Coal - 5 Oil - 4 Natural gas - 3 Hydro - 3 Rooftop solar - 2 Wind - 2 Nuclear - 1 60 Each of these numbers represents an order of magnitude, that is a power of ten. We can see that with rooftop solar, wind, and nuclear, the numbers are so close and the uncertainties are so great and their relative values so small compared to say coal that they can be seen as equivalent so far as safety is concerned. -------------------- 61 What are the safest and cleanest sources of energy? by Hannah Ritchie Published in Our World in Data First published in 2017, updated in 2022 and 2024 62 The author of this study addressed both deaths and greenhouse gas emissions. Deaths from accidents and air pollution are normalized to per TWh of electricity, while greenhouse gas emissions are normalized to GWh of electricity over the life cycle of the plant. 63 Here are the death figures. Coal 24.6 Oil 18.4 Biomass 4.6 Natural Gas 2.8 Hydro power 1.3 Wind 0.04 Nuclear 0.03 Solar 0.02 64 For greenhouse gas emissions the figures are Coal 970 tons Oil 720 tons Natural gas 440 tons Biomass 78 to 230 tons Solar 53 tons Hydro power 24 tons Wind 11 tons Nuclear 6 tons 65 If we take the death figures and rank them by order of magnitude as we did with the previous article, we get the following. 66 Coal - 4 Oil - 4 Biomass - 3 Natural Gas - 3 Hydro power - 3 Wind - 1 Nuclear - 1 Solar - 1 67 Keep in mind that the previous article covered only rooftop solar and not large industrial installations, and so is not directly comparable. Also the units are different, with the previous article being in terms of thousand TWh, and this one being in TWh. If we exclude solar (as the numbers are not comparable), Brian Wang's numbers are between 1.5 to 4 times higher than Ritchie's, except for hydro which are almost identical. I think this latter is due to both sets of numbers are dominated by one exceptionally big hydro accident. 68 Overall however, the relative rankings are quite comparable. Ritchie's numbers for deaths from coal, oil, and natural gas appear to be directly from the study by Markandya and Wilkinson mentioned above. For the benefit of those who are wondering, Ritchie specifically states that her numbers for nuclear include the Chernobyl and Fukushima accidents. -------------------- https://www.iaea.org/publications/magazines/bulletin/21-1/solar-power-more-dangerous-nuclear Direct link to file https://www.iaea.org/sites/default/files/publications/magazines/bulletin/bull21-1/21104091117.pdf https://ourworldindata.org/safest-sources-of-energy https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(07)61253-7/abstract https://www.nextbigfuture.com/2021/07/2020-fatalities-for-us-roofers-increased-15-as-solar-roof-installations-increase.html -------------------- 69 Conclusion from Studies Remember that in engineering terms, when comparing groups of numbers which contain both both very small numbers and one or more very large numbers, the differences between the small numbers are often not significant. The differences between the small numbers may be the product of our ability to measure these things rather than any real differences. 70 For example, in the article by Ritchie wind power would appear to be twice as dangerous as nuclear. However, the difference between them is 0.02 compared to 24.6 for coal. In other words, the difference between apparently "dangerous" wind and apparently "safe" nuclear is equivalent to 0.08% of the total for coal. It's therefore meaningless and a red herring to even worry about. 71 With the above taken into consideration, generally the different sources of energy fall into two broad categories in terms of number of deaths, injuries, and illnesses. The fossil fuels and biomass fall into one group and wind, solar, and nuclear into another group. 72 Hydro power would seem to fall into the higher risk category or at least somewhere between the two, but this I suspect is mainly due to one exceptionally large dam collapse in China, the Banqian Dam failure in 1975. This is mentioned as being specifically included in the article written by Ritchie. This was a multi-purpose dam, and information on this dam is difficult to find. It is not clear to me whether it had a hydro electric generator associated with either it or another dam that was part of the same system. 73 Some people therefor may argue for its exclusion from the numbers. Of course some people may argue for its inclusion anyway, as it was a dam regardless of whether it actually had an electric generator attached. If we exclude it, then I think the numbers for hydro power would fall into the same range as for nuclear, wind, and solar. 74 Most people would consider hydro power to be safe and clean enough regardless of this and I will rank it as such in any conclusions that I come to. As you can see, even if we have numbers, it can be a matter of opinion as to how to interpret them. -------------------- -------------------- 75 Taking a Systems Approach Now let's take a look at the broader energy picture today and into the future. Many countries in many parts of the world have committed to the concept of "Net Zero", which means eliminating carbon emissions on a net basis. Net zero essentially means the complete electrification of society. We must therefore have electrical energy on demand and at low cost. We must as a result of this look at complete electrical systems rather than individual sources in isolation. 76 At one time many electrical systems were entirely coal or entirely hydroelectric. This is no longer the case. There are now major amounts of wind and solar involved in many countries. However these are inherently intermittent. This means that other sources of energy are inherently also required to have a functional system. 77 If any particular solution inherently requires fossil fuels to meet part of the demand, then the safety, pollution, and climate issues relating to those fossil fuels have to be factored in to that complete system when trying to come up with a relative ranking. Talking about Individual sources in isolation are therefore meaningless in these countries. 78 There are battery systems, but these are mainly used to stabilize and regulate the grid plus to a lesser degree to smooth out short term daily peaks in demand. They do not have the ability to store large amounts of electricity on a large scale for an entire grid for days, weeks, and months to make up for intermittency. 79 So a serious attempt to rank sources of energy would need to look at a variety of representative countries and for each one come up with a plan that involves 'x' megawatts from source 'a', 'y' megawatts from source 'b', etc., and total up the values for each. 80 I am not aware of anyone who has studied this larger issue. However, the problem has to be addressed from this perspective in order for any answer to be useful. Not taking this into account is like ordering a diet soft drink to go with with a high calorie meal and assuring yourself that your plans to diet are fine. 81 This is not to imply there is anything inherently wrong with wind or solar. It does mean that if your goal is to achieve both net zero and a clean environment, you have to look at your entire energy system as a complete system rather than focusing on what you feel are the most reassuring parts of it while ignoring the rest. This does however add to the argument that it is in fact inherently very difficult to come up with a system of ranking energy sources for safety. -------------------- 82 Nuclear, Climate, and Clean Air - Contrasting Examples To give a tangible example we will now look at two different places that followed two divergent paths at roughly around the same time frame. These are the province of Ontario in Canada, and Germany. 83 Ontario had a mix of coal, hydro electric, and nuclear generating plants. Germany had a mix of coal, nuclear and natural gas plants. Ontario shut down their coal fired plants and kept their nuclear plants. Germany however shut down their nuclear plants and kept their coal fired plants. 84 The Phase Out of Coal in Ontario In 2003 Ontario decided to close all of its coal fired generating plants, which consisted of 19 units (that is boilers and turbines) totalling 8,800 MW. This phase out was completed by 2014. 85 Here are the figures for amount of power generated by each energy source in 2003 and 2014. Nuclear went from 42% to 60% Hydro went from 23% to 24% Gas went from 11% to 9% Coal went from 25% to 0% Non-hydro renewable went from 0% to 7%. 86 As you can see, the bulk of that replacement came from increased use of nuclear power. Furthermore, this did not result in simply replacing coal with natural gas. While gas is cleaner than coal, it still has emissions and if you recall from the studies that we looked at earlier, had an estimated death rate roughly 2 orders of magnitude greater than nuclear, solar, or wind. 87 To put this in more practical terms, at one time Toronto regularly had clouds of smog obscuring it, to a large extent due to these coal fired power plants With the phase out of coal, smog days went to zero in 2015 compared to 53 a decade earlier. The 2023 figures for Ontario show carbon emissions of 53 grams per kWh of electricity generated. We can use this as a rough benchmark comparison for total emissions. 88 The Phase out of Nuclear in Germany Until March of 2011, Germany generated one quarter of its electrical power from nuclear. Starting in 2011 however, they began shutting down their nuclear power plants. These were then phased out over the next decade. However, the coal plants were to be kept to 2038. In 2026 Germany began talking about increasing use of coal in order to save gas. In the same year the German chancellor Friedrich Merz stated that the phase out of nuclear was a quote “serious strategic mistake”. EU Commission President Ursula von der Leyen said it was "a strategic mistake for Europe to turn its back on a reliable, affordable source of low-emissions power". 89 I won't go into the details of the phase out, but let's look at some emissions numbers for Germany. If we look at the official numbers from the European Environmental Agency for 2024, for Germany their emissions were 298 grams per kWh of electricity generated. Recall that we are using emissions as a very rough guide to amount of air pollution, and that this has a direct effect on the safety of the overall electrical energy system. 90 So, who actually made their people safer, Ontario who phased out their coal plants and kept their nuclear plants, or Germany who phased out their nuclear plants and kept their coal plants? 91 If you want a comparison directly within Europe, then Germany has one of the highest rates of emissions per kWh of electricity generated, whereas France, who use mainly nuclear power, have one of the lowest at 43 grams per kWh of electricity generated. Again, who is making their people safer, Germany or France? 92 I don't want to make it sound like I am picking on Germany. I am also not going to tell them how they ought to run their country. However they provide a good real world example of how we need to look at things in overall context when we are thinking about the choices that we make. https://www.ontario.ca/page/end-coal https://www.cbc.ca/news/canada/windsor/smog-study-shows-significant-decreases-in-pollutants-in-ontario-1.4151183 https://www.eea.europa.eu/en/analysis/indicators/greenhouse-gas-emission-intensity-of-1 https://world-nuclear.org/information-library/country-profiles/countries-g-n/germany https://www.politico.eu/article/friedrich-merz-is-right-to-reject-germanys-nuclear-phase-out-says-iea-chief-fatih-birol/ https://www.politico.eu/article/germany-considers-ramping-up-coal-power-to-avert-energy-crisis/ https://www.iea.org/countries/estonia/electricity https://www.iea.org/countries/malta/electricity -------------------- 93 Conclusions As we can see, there don't appear to be an abundance of peer reviewed scientific studies that we can simply point to in order to answer the question of safety of all possible major different energy sources once and for all. Collecting the data to even attempt to answer the question is inherently very difficult as we cannot readily conduct experiments to answer the question, and sources of data are not collected or consolidated in a manner which can answer this question adequately. 94 The essence of the problem is that most energy industries are not as tightly regulated and monitored to the same degree that say nuclear power or commercial airliners are, so this data is simply not being systematically recorded. However, a number of people have attempted to make estimates. 95 Their conclusions would seem to be that nuclear, wind, and solar are roughly equivalent in terms of safety. All fossil fuels are much less safe than nuclear, wind, and solar, by as much as several orders of magnitude. 96 We can however say with a reasonable degree of certainty that if a country shut down their nuclear power plants and kept their fossil fuel plants, particularly coal, then they probably made their people less safe than if they had done things the other way around. 97 I hope that I have provided some context in which to think about the issue. Thanks again to brian in ohio for providing the question upon which this episode is based. -------------------- Provide feedback on this episode.

This show has been flagged as Clean by the host. This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems. The echo command is very useful—it prints the arguments given to it, followed by a newline character. (The newline is sometimes also called a linefeed character depending on who is writing or speaking, and has the ASCII decimal value 10.) It has many uses, either in a script or interactively on the command line. The echo utility is used to display text, the value of a variable, or the result of a pathname expansion. It can also feed text to another command in a pipeline. As useful as echo is, it should come as no surprise that it first appeared early on in Bell Laboratories' Second Edition UNIX 1 in 1972. This initial version accepted no options 2 —although the manual page doesn't explicitly say output is followed by a newline character, the description of writing "as a line" seems to imply it. In Seventh Edition UNIX, the manual page 3 makes that clear, and also features the addition of the -n option, which causes echo to print the arguments without a trailing newline character. Eighth Edition UNIX's echo 4 gained the -e option, which allows certain escape codes from the C programming language to be used. These variations caused differences in behavior between different versions of echo . Will running echo -n something on your system output the text "something" without a newline, or "-n something" followed by a newline? Things get even trickier when the command arguments include parameter or pathname expansions. If there are files named "-n" and "something" in the current directory, what does echo * output? Like the previous question, that depends on whether or not your version of echo treats -n as an option. You can't get around this ambiguity by quoting or escaping the "*", because that just causes echo to print a literal asterisk. Example using GNU utilities on Debian 12; both the "echo" utility and the "echo" builtin of bash recognize "-n" as an option. $ ls -1 -n something $ echo * something$ #Shell prompt is on the same line because "-n" was treated as an option to echo $ echo "*" * The solution was to create a new utility, which is the first UNIX Curio for today: printf . This command allows a user to print text similar to the way the identically-named function works in the C programming language. You run printf 5 followed by a format string, followed by zero or more arguments. No newline characters are printed unless specifically indicated by the format string or the arguments. To use printf to print "something" without a newline, that would just be printf something . This demonstrates that you don't need any arguments—in this example, the format string is just a set of regular characters to be displayed. If you wanted a newline character at the end, printf "somethingn" would give you that. (In this case, the format string needs to be quoted so the "n" isn't interpreted by the shell.) In addition to "n" for a newline, you can also use "a" for an alert (rings the terminal bell), "b" for a backspace, "f" for a formfeed, "r" for a carriage return, "t" for a horizontal tab, "v" for a vertical tab, and "\" to get a literal backslash. In addition to these special characters, any arbitrary byte can be included using a backslash followed by one to three octal digits; however, it might be difficult to predict what will be output because it can differ based on the character set the terminal is using. It is safer and more portable to stick to the pre-defined characters if possible. The real magic of the printf utility comes from using "conversion specifications" in the format string. Probably the simplest of these to explain is the "%s" conversion specification—it represents a string of any length. The command printf "Hi, %s, how are you?n" followed by a list of names would print the greeting for each name, putting it in the place occupied by the "%s". $ printf "Hi, %s, how are you?n" Alice Bob Carol Hi, Alice, how are you? Hi, Bob, how are you? Hi, Carol, how are you? The format string is reused as many times as needed to consume all of the arguments. Take, for example, the command printf "Hi, %s, have you met %s?n" . If this is run with two name arguments, it would print the sentence on one line, using both names. If run with four name arguments, it would print the sentence twice, once with the first two names and again with the second two names. If you only gave it three names, the last "%s" conversion specification would be replaced with a null string. $ printf "Hi, %s, have you met %s?n" Alice Bob Hi, Alice, have you met Bob? $ printf "Hi, %s, have you met %s?n" Alice Bob Carol David Hi, Alice, have you met Bob? Hi, Carol, have you met David? $ printf "Hi, %s, have you met %s?n" Alice Bob Carol Hi, Alice, have you met Bob? Hi, Carol, have you met ? Three other items can also be given in each conversion specification: flags, the field width, and the precision. The exact meanings of these depend on which type of conversion specifier character you are using. For "%s", using a "-" as the flag causes the text to be left-justified instead of the default right-justified, a field width causes the printed field to be at least as long as the number given, and a precision limits the number of bytes written from the string to the number given. $ #Example of %s with a precision value $ printf "Hi, %.3s, how are you?n" Alice Bob Carol Hi, Ali, how are you? Hi, Bob, how are you? Hi, Car, how are you? $ #Example of %s with a field width $ printf "Hi, %8s, how are you?n" Alice Bob Carol Hi, Alice, how are you? Hi, Bob, how are you? Hi, Carol, how are you? $ #Example of %s with a left-justify flag and a field width $ printf "Hi, %-8s, how are you?n" Alice Bob Carol Hi, Alice , how are you? Hi, Bob , how are you? Hi, Carol , how are you? $ #Example of %s with a left-justify flag, a field width, and a precision $ printf "Hi, %-8.3s, how are you?n" Alice Bob Carol Hi, Ali , how are you? Hi, Bob , how are you? Hi, Car , how are you? While "%s" is probably the most commonly-used conversion specification, others are available. A whole set of them are dedicated to printing integer values as a signed decimal, an unsigned decimal, an unsigned octal, or an unsigned hexadecimal number. These also can take flags, a field width, and a precision. I think the details and nuances of all this are too complex to clearly explain here, so I will just refer you to the POSIX "file format notation" specification 6 . Be aware that unlike the printf function in the C programming language, the printf utility is not obligated to accept conversion specifications for floating-point numbers. While some implementations might support this, scripts intended to be portable should limit themselves to the restricted set required by the POSIX standard (%d, %i, %o, %u, %x, %X, %c, and %s, plus %b and %% described below). Two more conversion specifications are worth mentioning. The first is only required by the standard for the printf utility, not the C function, and is "%b". This is the same as "%s", except that certain backslash escape sequences in the argument will be treated specially. This includes all the ones described above except for the one using octal digits to represent a byte. In an argument, this is instead represented by "" followed by one to three octal digits. An additional backslash escape sequence accepted is "c"—this does not print anything itself, but causes printf to immediately halt output. The final conversion specification is "%%", which just outputs a literal "%". You can't use a bare "%" in the format string, because printf expects that to introduce a conversion specification. Be careful not to be tripped up by this when trying to print some value as a percentage. Example assuming that the hypothetical "/dev/batterycharge" file on your laptop outputs the battery charge level (42% in this case). As you can see, in some cases an error message might be displayed, but in others it might just behave in a way you didn't intend without complaining. GNU's "printf" utility and the "printf" builtin of bash both support "%e" as a conversion specification as an extension to POSIX. $ cat /dev/batterycharge 42 $ #Wrong $ printf "Your laptop's charge level is $(cat /dev/batterycharge)%.n" bash: printf: `': invalid format character Your laptop's charge level is 42$ #Shell prompt appears here from the error $ #Right $ printf "Your laptop's charge level is $(cat /dev/batterycharge)%%.n" Your laptop's charge level is 42%. $ #Next one treats %e as the specifier, with the space and "l" as flags $ printf "Your laptop has $(cat /dev/batterycharge)% level of charge.n" Your laptop has 42 0.000000e+00vel of charge. $ #Because no arguments were given, "0" was used for the value to convert Let's go back to the situation I was describing with echo —we have files named "-n" and "something" in the current directory and want to print all their names, separated by spaces. We could do that with printf "%s " * , which would not treat the "-n" as an option. However, the output might look a little weird because there wouldn't be a newline character at the end. We could insert a newline by using "%b" instead of "%s" and following the asterisk with a "nc" as the second argument. The "c" is there to prevent the final space in the format string from being printed after the newline. $ ls -1 -n something $ printf "%s " * -n something $ #No newline was printed here $ printf "%b " * "n" -n something $ #There's a newline, but also a spurious space before the shell prompt $ printf "%b " * "nc" -n something $ #No space before the shell prompt this time Using the "%b" conversion specification can therefore solve one problem, but it also introduces another. Arguments which include a backslash can be interpreted as escape sequences, and many systems are fine with allowing backslashes in filenames. In cases where you're just using the printf utility to display text, it's usually not a big deal if the output looks a little wonky. Where you really need to be careful is when the text is being piped to another program, as control characters and other oddities might cause unexpected results, and can potentially create security problems if processed by a script or utility running as a privileged user. $ #GNU "ls" displays filenames containing a backslash in single quotes $ ls -1 apple banana 'cherry' durian $ printf "%b " * "nc" apple banana $ #"c" in "cherry" stops output immediately The printf utility looks to have shown up first in 1986's Ninth Edition UNIX 7 , though the earliest manual page I could find 8 is from the Tenth Edition. Its first appearance in BSD seems to be from 1990 in the 4.3 Reno release 9 . Two years later, it was added to Issue 4 of The Open Group's CAE Specification. From what I can tell, it did not seem to be in AT&T's System III—presumably the printf utility did make it into System V at some point but I found it difficult to track this down. While echo is still suitable for use where you know for certain that you want a newline character printed at the end and none of the arguments will start with a hyphen, consider using the printf utility instead for displaying text. It offers more flexibility and features than you are guaranteed to get with echo , although it does require a bit of forethought in constructing a proper format string and arguments. That is not necessarily a bad thing, because a script's author should be thinking about what might happen if it is called with "strange" text or filenames. This episode also provides a good case for being careful when naming files—many filesystems will allow you to use hyphens, control characters, quotation marks, and potentially any character other than a slash or a null byte in a filename. As we've seen, some of these characters can create problems for standard utilities. While it can feel limiting, especially for people not using English, the safest filenames to use on a UNIX-like system consist only of characters in the "portable filename character set" as defined by POSIX 10 and where the first character is not a hyphen. This set includes the lowercase and uppercase letters "a" through "z", the numerals "0" through "9", and the period, underscore, and hyphen. Notably, it does not include the space character. That leads me to another UNIX Curio that I only just now discovered while researching this episode. This is the pathchk utility 11 . It can be run with one or more strings as arguments, checks each one against a set of rules for pathnames, and outputs an error message for each problem found. By default, it checks against the following limits on the system where it's being run: maximum number of bytes in the full path, maximum number of bytes in any component of the path, all byte sequences must be valid in the given directory, and the user running the program must have access to all directories referenced. If run with the -p option, instead of those limits, it checks against POSIX limits: a maximum of 256 bytes in the full path, a maximum of 14 bytes in each component of the path, and each component must only include characters from the portable set. The -P option adds warnings if any component starts with a "-" or if the pathname is completely empty. While the exit status will tell you if the checks succeeded or not, I don't feel like the pathchk utility is well suited to be used in an automated fashion, as the exact wording of its output is not specified and checks cannot be selected individually. However, it can be used interactively to validate pathnames you aren't sure about. See the linked specification for full details. References: A Research UNIX Reader: Combined Tables of Contents https://archive.org/details/a_research_unix_reader/page/n99/mode/1up A Research UNIX Reader: Second Edition UNIX echo manual page (although this page has "v1" typed at the top, the date and the tables of contents indicate it first appeared in v2, a.k.a. Second Edition) https://archive.org/details/a_research_unix_reader/page/n22/mode/1up Seventh Edition UNIX echo manual page https://man.cat-v.org/unix_7th/1/echo Eighth Edition UNIX echo manual page https://man.cat-v.org/unix_8th/1/echo Printf specification https://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html File Format Notation specification https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap05.html A Research UNIX Reader: Ninth Edition Table of Contents https://archive.org/details/a_research_unix_reader/page/n95/mode/1up Tenth Edition UNIX echo/printf manual page https://man.cat-v.org/unix_10th/1/echo 4.3BSD Reno printf manual page https://man.freebsd.org/cgi/man.cgi?query=printf&sektion=1&manpath=4.3BSD+Reno Definitions: Portable Filename Character Set https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_282 Pathchk specification https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pathchk.html Provide feedback on this episode.

This show has been flagged as Clean by the host. I started out with Basic on the TI-99/4A in 1984. The bare machine could not be programmed by the user in machine code. In 1985 I bought a ZX Spectrum, that gave me total control over the machine. I wrote two FORTH systems on the ZX-Spectrum. In 1988 I got my first 8088 PC, also programming it in FORTH. In 1992 I got an 80386 PC and I ran Linux on it. MCC Interim Release from v. This was the first Linux distro. I have been using Linux ever since. From then on I obtained newer PCs, such as a Pentium in 1995, a Pentium-2 in 1998, a Pentium-4 in 2003 and a Core-2 Duo in 2006. I used several

This show has been flagged as Clean by the host. Pokémon Go (stylized as Pokémon GO) is a 2016 augmented reality (AR) mobile game developed and published by Niantic, in partnership with Nintendo and The Pokémon Company, for iOS and Android devices. The game uses GPS to locate, capture, train, and battle Pokémon. It is free-to-play, featuring a freemium model that includes local advertising and offers in-app purchases for additional in-game items. Pokémon Go launched with approximately 150 Pokémon species, with new species regularly introduced. -- Wikipedia.org Provide feedback on this episode.

This show has been flagged as Explicit by the host. Some links to other tutorials (in case you need a visual element, these helped me out) Mixing board tutorial https://youtu.be/H-7kQmpjBds Drone Box tutorial https://youtu.be/50C6DBsqy24 Provide feedback on this episode.

This show has been flagged as Clean by the host. T-DOSE TDOSE 2027 Mark you calendars #TDOSE 2027 on 5 and 6 June '27 in the Weeffabriek, Geldrop. T-DOSE Info Booth Hackalot Laptop Revive Free Software Foundation Europe Doeidag and Banray Debian Angry Nerds Podcast Freie Software Freunde - Free Your Model Train Hacker Public Radio: The community Podcast UBports Adfinis Credits The Technical Dutch Open Source Event (T-DOSE) In hpr4641 :: Technical Dutch Open Source Event (T-DOSE) , Ken interviewed Peter van Ginneken about the T-DOSE conference. The Technical Dutch Open Source Event (T-DOSE) is a free conference to promote the use and development of Open Source software. This event has is organised yearly since 2006 in the Brainport region, near Eindhoven, The Netherlands. During this event, Open Source projects, developers and visitors can exchange ideas and knowledge. Peter van Ginneken Opens the Event. We catch up with him at the start of Day 2. Info Booth The backbone of any event is the Info booth and catering. Here we talk to Nick Hibma who when not serving on the Info Booth is treasurer of the T-DOSE organisation. Ready to serve sandwitches, sell T-Shirts, Magic Mugs, and club-mate T-Shirts club-mate Magic Mugs Hackalot Hackalot is the Eindhoven and surrounding area hackerspace. A hackerspace is a place where hackers can work on their own or collaborative projects. You can work and talk together, but you can also do your own thing. Together we can also purchase a lot of cooler tools such as lasercutters and 3d printers. Often there is no suitable place for equipment at home. So if you know someone, you are either an electronics/computer/technical hobby that got out of hand, come on by! Boekenwuurm at the Hackalot stand. The Hackalot stand. Boekenwuurm@hsnl.social boekenwuurm.nl Hackalot Laptop Revive Laptop Revive collects discarded laptops, that are still working. We then install Linux Mint to provide a working laptops to students who cannot afford laptops. We are socially involved, sustainable and open. Alex Kok Laptop Revive Laptop Revive Free Software Foundation Europe Free Software Foundation Europe (FSFE) information booth, with information material, stickers and merchandise. Nico was so busy that we were unable to snag an interview this time. However check out our talk with him at the NLUUG Spring Conference 2026 . Free Software Foundation Europe Doeidag and Banray We also interviewed Geert-Jan Meewisse in hpr4639 :: NLUUG Spring Conference 2026 but this time he is here talking about banray.eu In 2025, Meta sold over seven million pairs of camera-equipped glasses that look like regular Ray-Bans. The person wearing them looks like anyone else. But these people are now products, as is everyone they interact with. He then also mentioned the Doeidag project where they encourage people to drop one service at a time on the first Sunday of the month https://doeidag.nl/ https://banray.eu/ Geert-Jan Meewisse Doeidag and Banray Debian The Debian Project is an association of Free Software developers who volunteer their time and effort in order to produce the completely free operating system Debian. Ken Talks to Joost van Baal Llić from the Debian Project Debian Angry Nerds Podcast Angry Nerds, met extra cyber! The Angry Nerds is a Dutch Language podcast about privacy and security It's a live show that is topical and often humorous tech podcast where a group of enthusiastic nerds discusses current technology, IT and cybersecurity topics. The hosts combine technical depth with background conversations, humor and the occasionally a good dose of cynicism. Expect conversations about everything from network infrastructures to software development, from privacy issues to bizarre tech trends. Ken on the Angry Nerds Podcast You can listen to the recording at Angry Nerds op T-DOSE 2026 deel 2 (prikkelarme versie). Angry Nerds Podcast Freie Software Freunde - Free Your Model Train We are a non-profit organization. We are committed to Free Software and Open Standards. Software is not just technology, it's an important part of our daily life. We want to raise awareness of the importance of Free Software and Open Standards. That is why we are concerned with topics outside of technology: politics, education, ethics, psychology, ecology and economics, licenses, ... One of our projects is "Free your model train". Our goal is to raise awareness of the benefits of open standards. Birgit Hücking (@akkolady) standing at the freie-software.org The freie-software.org table with two large train loops, a smaller internal one. Two knitted Tux Mascots. And a lot of information. Close up of the two knitted Tux Mascot. @akkolady@chaos.social @FreieSoftwareFreunde@mastodon.social Freie Software Freunde Free Your Model Train https://fymt.de Hacker Public Radio: The community Podcast Hacker Public Radio is a technology focused podcast that releases shows every weekday Monday to Friday. Our shows are created by people like you, and can be on any topic that is of interest to hackers, hobbyists, makers, etc. We are a welcoming community that offers positive feedback and encourages respectful debate. This is our 21st year of operation, and we will release our 5,000th show in August. Everything we do is released under a Free Culture License. We do not vet, edit, moderate or in any way censor any of the audio you submit, we trust you to do that. We will be available to guide you in sharing your knowledge with the community. Having had a stand at FOSDEM (BE), OggCamp(UK), Linux Fest North West(US), Spectrum (FR), we are available to show you how easy podcasting can be. We will be answering your questions, and conducting interviews with anyone with anything interesting to say. The HPR booth. Hacker Public Radio UBports We are developing an open source Linux mobile OS built to be your daily driver... ...and we'd like to welcome you to our community. Next up is a chat with Sander Klootwijk about UBports and Ubuntu Touch. Their website has a list of supported devices . We talk with Sander Klootwijk Proof it's running on actual hardware Yumi The UBports Installer Mascot was not available for comment. Ubuntu Touch on a Fairphone @BallonQuartier@mastodon.nl UBports https://devices.ubuntu-touch.io/ Adfinis Accelerate your business with open source-driven automation, security, cloud, and DevSecOps solutions from Adfinis, your end-to-end partner for robust, flexible IT that drives growth and innovation at any scale. Welcome to Our World Full of Open Source At Adfinis, we believe in the transformative power of open source technology to foster innovation, transparency, and collaboration. We are committed to providing solutions free from vendor lock-in, ensuring our clients retain full control and flexibility over their systems. Digital sustainability lies at the heart of our approach, as we strive to create technologies that not only serve the present but also support a long-term, environmentally responsible future. Additionally, we champion digital sovereignty, empowering organizations and communities to own and control their data, infrastructure, and technological destiny. These principles drive us to build a more open, sustainable, and inclusive digital world. Finally we chat to Coen hamers , Robert de Bock , and Annebelle van Waardenburg from Adfinis whose sponsorship made the event possible. https://www.adfinis.com/en/solutions https://www.adfinis.com/en/career Credits Record Needle Rip Free Software Song Provide feedback on this episode.

This show has been flagged as Clean by the host. Wandering along the suggestion of topics in Hacker Public Radio, we come to this: how to obtain the answer to something that I don't even know how to ask? My opinion is: you can't get an answer without the desire, be it represented by a question or a quest. So, how to pose the correct question? There are a few steps! 1st, interest . Get motivated to know, even if you don't know exactly what you want specifically. 2nd: be open to deconstruction. To learn is to recognize you didn't know before (self-pride has to give place for learning). And when confronted with the new, things you thought you knew — even ones you relied on before — may need to be now rejected, if incompatible with the truth you only now receive. 3rd: when the barriers are down, it's possible to study the theme. Because, at the moment, you don't know what you don't know; while one ignores something, he's only ignorant. One have to listen to others, to practice and /or reflect on the subject until he reaches something specific (and not easily or promptly answerable) worth being the object of his quest or question. 4th: with the question in hand, go for the answer. If it's valuable for you, more time or less time you are definitely going to get it. If it's too broad, or you lose interest, or if you need more after getting the initial answer, no problem: you can reformulate and start from the beginning. Great part of the necessary questions only come after the answer already came by a long course of inquiries [questions] (natural consequence of studying with a desire). * * * Credits: 1) Writing is putting thoughts in order and being able to detect what is loosen or unfounded (needing refinement of discard). “The reason so many people have trouble writing is that it's fundamentally difficult. To write well you have to think clearly, and thinking clearly is hard.” ( https://paulgraham.com/writes.html ) 2) “What Are the Answers I Need, To the Questions I Don't Know Enough to Ask?” ( https://hackerpublicradio.com/about.html#requested_topics ) ( 1 and 2 came at a particular time together and where more deterministic to the line of this episode. ) 3) Teacher Roosevelt, a course administrator (coordinator) of my post-graduation at Escola Paranaense de Direito, that ministered about the end of course paper and how to pose the question you are going to pose in order to write about a topic. 4) Gym exercises (because doing seated leg curls is difficult and gives strange sensations to many men; as thinking , gives a strange feeling one can't even discern immediately if it's good, bad, or only odd). Sound effect for credits numbering: “Number pad 1”, by phatkatz4 on Freesound ( https://pixabay.com/sound-effects/musical-number-pad-1-105399/ ) Thank you! Provide feedback on this episode.

This show has been flagged as Clean by the host. Blog post: https://claudiomiranda.wordpress.com/2026/05/19/laptop-computer-woes-or-how-i-learned-to-love-my-tech-hoarding/ HP ProBook 4540s specifications: https://support.hp.com/au-en/product/product-specs/hp-probook-4540s-notebook-pc/ Tonymacx86 forum post on the 1920x1080 screen upgrade for the ProBook 4540s: https://www.tonymacx86.com/threads/4540s-lcd-screen-1080p-replacement.88212/ Panelook.com link to the LG156WF1 screen variants: https://www.panelook.com/modelsearch.php?keyword=LP156WF1&search_id=1839426 HP ProBook 4540s teardown and screen upgrade video by JSnake Repair: https://youtu.be/3ceZ1m7JFYw?t=1252 Provide feedback on this episode.

This show has been flagged as Clean by the host. In his early days as a writer, Heinlein wrote his stories in the context of a shared universe that he called the Future History. These were mostly short stories at first, with hte occasional novella. But they inclode some great stories. The Future History, Part 1 One thing Heinlein became well known for was his Future History. This placed many of his stories in a common framework of a future environment, and allowed events from one story to influence events in other stories. Here is what he had to say about it, in a post I found on the Heinlein Society Facebook site: “I never “created” or “invented” a “Future History.” On April Fool's Day 1939 I started to write commercially; by the middle of August I had written 8 shorts & a serial. As 5 of these items were more or less to the same fictional background, I found that I was continually having to check back to keep from tripping over my own feet. So I took an old navigation chart, about 3×4 feet, turned it over, made the time scale vertical, then set up 5 columns: stories, characters, technical data, sociological, remarks. Then I checked those first 5 stories, filled data into proper columns at the proper height for the fictional date—and continued to do this with other stories later. The chart was on the righthand wall near my elbow and was unusually messy as I never took the chart down to add to it—just reached over and scrawled on it.” Source: https://www.facebook.com/HeinleinSociety/posts/i-never-created-or-invented-a-future-history-on-april-fools-day-1939-i-started-t/1092968002874634/ One thing that became clear as his Future History developed is that he was not looking at our future exactly. He was very clear in his mind that he was writing fiction, and not issuing prophecies. If you are reading it today, it is best to think of this as a kind of alternate timeline, and this is something that holds true through a lot of his work. Even in his later novels, which were never formally part of his Future History, he would mention events from that past group of works, which may implicitly incorporate them. But this is an area where scholars are in disagreement as to which if the later novels, if any, should be incorporated. And there were unwritten stories that appeared on the chart that would have given further background to the stories that were written. They were stories Heinlein seems to have intended to write at some point, but never got around to writing. You can get more information about this in his book Revolt in 2100. The Future History stories were initially collected primarily in three books: The Man Who Sold the Moon (1950), The Green Hills of Earth (1951), and Revolt in 2100 (1953). Each of them fleshes out this hypothetical world in different ways. The first one, The Man Who Sold The Moon, introduces us to a businessman named D.D. Harriman, who is obsessed with going to the moon. But he thinks it should be done by private enterprise rather than by government. So he concocts a scheme to do this. He promotes a legal theory that the rights to the moon belong to the countries that it directly flies over, sort of like air rights taken to infinity. Then he uses the chaos of competing interests to throw this into the United Nations, and then gets the U.N. to give him the rights. He finally gets to launch a mission to set up a Moon base, but cannot join the expedition because the corporation considers him too valuable to risk. In a sequel story, Requiem, he does get to the moon just in time to die there. Heinlein was never above writing a tear-jerker. Of course, the book has other stories not linked to D.D. Harriman. Heinlein's fist story, Life-Line, is also collected here. And his second story, Let There Be Light, anticipates the development of solar power panels, but similarly to Life-Line, this earns the enmity of corporate interest, in the form of the Power Syndicate. The Roads Must Roll postulates moving roadways in the future, but the story really is about the sociology of technology in the future. And Blowups Happen, originally from 1940, anticipates nuclear fission as a power source, but it proves to be dangerous. They claim that the craters on the moon were really caused by a series of explosions to reactors that wiped out an earlier civilization. So they move the reactor into space for safety. And this feeds back into The Man Who Sold The Moon when this reactor in space blows up. In these early stories we can already see that Heinlein has a complex view of society. In Life-Line and Let There Be Light corporate power is the villain of the story, and some of this also shows up in Blowups Happen. But in The Man Who Sold The Moon we see that private enterprise is preferred to government action. I think the way this can be reconciled is to see that Heinlein is always concerned with individual personal freedom and opposed to anything that might endanger that, whether from too much government or too powerful corporate interests. The Green Hills of Earth contains the story of the same name, which concerns a former space engineer, Rhysling, now blinded by radiation and unemployable, who is also a poet. And one of his poems has that title. The crew of Apollo 15 named a crater on the moon “Rhysling”, and they planned to read a bit of it at the crater, but those trips could get very busy. Still, as they were getting ready to leave the moon there was this exchange. Note that Allen is the Capcom, and Scott and Jones are the astronauts : “Allen: As the space poet Rhysling (the blind poet in Robert Heinlein's The Green Hills of Earth) would say, we're ready for you to “come back again to the homes of men on the cool green hills of Earth.” [Scott – “That's from the Green Hills of Earth. That's one we talked about before the flight. Have you read that one?”] [Jones – “Oh, yeah! That was a favorite when I was a kid. Had you read it?”] [Scott – “Sure. (Quoting from memory): We pray for one last landingon the globe that gave us birthTo rest our eyes on fleecy skiesand the cool green hills of Earth.” Although two of the stories in this collection were older, from 1941, most of them are from 1948 and 1949. And there is a reason for that. On December 7, 1941, the United States found itself at war with Imperial Japan, and few days later Nazi Germany. Coming from a family that had fought in every American war you would expect Heinlein to get involved somehow. He could not enlist due to his medical retirement from the Navy, but since he had an engineering background so he became a civilian employee at the Philadelphia Navy Yard, where he was joined by fellow science fiction writers Isaac Asimov and L. Sprague de Camp. A nice retelling of this can be found at Kirkus Reviews, and Asimov also discusses this in his biography. The upshot is that there is a gap of about 5 years when Heinlein did not publish anything. It is also notable that Heinlein by this point had escaped from the pulp science fiction magazines and gotten published in what were called the “slicks', so-called because the paper they were printed on was slick and higher quality than the pulps. His stories began to be published in places like The Saturday Evening Post, Argosy Magazine, and Town & Country. And these outlets paid higher rates than the pulps, a significant matter for any writer. Heinlein always maintained that the only reason anyone would write was to make money. And the stories were getting to be quite good as well. Delilah and the Space Rigger (1949) tells the story of a woman who joins a construction crew on a space station and faces discrimination, but wins out in the end, which was pretty progressive for the time, but not atypical for Heinlein. Space Jockey is a fairly pedestrian story about a rocket pilot dealing with his every day life. But The Long Watch is an important story to Heinlein's view of the important things in life. A young officer is assigned to duty on the lunar base, where there are nuclear weapons stored. His superiors want to stage a coup, using those weapons, which can threaten the Earth while being beyond the reach of retaliation. The young officer sacrifices himself to prevent their plot from succeeding, and becomes recognized in a death as a great hero. And this becomes part of the background to a later juvenile novel Space Cadet, as well as being referenced occasionally in other stories, so you can see that he regarded it as an important statement. Gentlemen, Be Seated is a cute little story about a man who saves people when a leak happens in a tunnel on the Moon by plugging the leak with his rear end. The Black Pits of Luna is little thing about a boy scout who is able to rescue his little brother, but it foreshadows the Juvenile novels he later wrote. It's Great To Be Back! is about a couple who have moved to the Moon, but continually find fault with the living arrangements. They finally decide to go back to Earth, but discover that it was not really the place they had remembered, and they then return to the Moon, which they now realize is home. -We Also Walk Dogs is a gem of a story concerning a company called General Services that basically does things for their clients. Their advertising slogan is “Want somebody murdered? Then DON'T call General Services. But for anything else, call…. It Pays!” They deal a few different problems in this story, but the main one is the development of anti-gravity, and it features a Chinese porcelain bowl. Ordeal in Space is about a spaceman who has an accident that gives him a fear of heights and washed him out of space. But he has to face his fear when he needs to rescue a kitten from the 35th floor. One thing about Heinlein is that he was a firm and devoted cat fancier, so it no accident that a kitten is the one that has to be rescued. And the final story, Logic of Empire, he discusses the development of slavery in the Venus colony as a natural consequence of machinery being expensive and humans being cheap. And in this story there is a background reference to Nehemiah Scudder, who will soon be important in the Future History. One of the things that is worthy of a brief discussion at this point is exemplified by the story Logic of Empire, and that is the reference to the Venus colony. We now know that Venus can best be described as hellish, with crushing air pressure and temperatures high enough to melt metals. The best designed landers can last no more than minutes before being destroyed. But this was not known when Heinlein was writing these early stories. The prevailing view at that time was that Venus was shrouded in clouds because it was very wet and swampy, so that is what Heinlein went with. Similarly his Mars had canals and was inhabited. You just have to go with it in these stories, as you have to do with so much of Golden Age science Fiction, let alone pre-Golden Age. Links: https://www.facebook.com/HeinleinSociety/posts/i-never-created-or-invented-a-future-history-on-april-fools-day-1939-i-started-t/1092968002874634/ https://www.amazon.com/Man-Who-Sold-Moon/dp/0671578634 https://www.amazon.com/Green-Hills-Earth-Robert-Heinlein/dp/0671578537 https://www.amazon.com/dp/B0011GBTKM/ https://www.kirkusreviews.com/news-and-features/articles/asimov-de-camp-and-heinlein-naval-aviation-experim/ https://www.palain.com/science-fiction/the-golden-age/robert-a-heinlein/the-future-history-part-1/ Provide feedback on this episode.

This show has been flagged as Clean by the host. In this episode Kevie does a step-by-step approach to record an episode of HPR using the FFMPEG tool on the Linux command line. Before beginning please ensure that FFMPEG is installed, it is available in the vast majority of Linux repositories. Start by making a new folder to keep all your files in and move into the new folder (these will be numerous by the end of your recording): mkdir Podcast cd Podcast To start recording audio use the command: ffmpeg -f pulse -i default file01.flac and finish the recording by pressing ctrl+c. I would recommend recording a test piece of audio to ensure that you are recording from your desired microphone and that the levels are to your liking. To listen to the audio file we use ffplay: ffplay file01.flac Once all of the files have been recorded, to reduce the need for editing I would recommend recording several short segments, we need to put these together using: ffmpeg -i file01.flac -i file02.flac -i file03.flac -i file04.flac -i file05.flac -filter_complex "[0:a][1:a][2:a][3:a][4:a]concat=n=5:v=0:a=1" filedone.flac Not that the number of sets squared brackets [] should be the same as the number of files (these start at zero) and the number after n= should be the actual number of files you wish to combine. To remove any extended periods of silence then we can use: ffmpeg -i filedone.flac -af silenceremove=stop_periods=-1:stop_duration=1:stop_threshold=-45dB filefinished.flac Please note that this is a bit flaky at the time of recording (my results have been mixed) and it will re-encode the audio file so never do this with a lossy file such as ogg or mp3 as this will reduce the quality, keep this for lossless versions such as flac or wav files. If you want to spend a bit more time editing the files and getting a better final audio file then the most effective way (but not quick) is to trim the audio from the end and beginning. Listen to the audio files and note the times of any periods of silence. As these normally are at the beginning and end, especially when we are recording in short segments then I will limit the instructions to avoid these becoming silly in length. Firstly clip off the end silences first, if you start with the beginning then it will change the starting position of the end silences. To remove audio from the end of a track use: ffmpeg -i file01.flac -vn -acodec copy -to 00:01:30 file01cut.flac In this example anything after 1 minute and 30 seconds will be removed. The edited audio file will then be saved as file01cut.flac. This method does not re-encode the audio so there is no loss of quality. To remove audio from the start of a track use: ffmpeg -ss 30 -i file01.flac -c copy file01cut.flac In the above example the first 30 seconds of the file will be removed and saved as file01cut.flac Once you have edited each audio file then they will need to be merged together again to make a complete show. Provide feedback on this episode.

This show has been flagged as Clean by the host. 01 Introduction This is a follow up to my 4 part series on simple podcasting. In this episode I will discuss a number of experiments with audio filtering. These experiments were inspired by comments by listeners and by other discussions about audio on HPR. I am not an audio expert, so I am doing this partly in order to learn something, but mainly in order to have a bit of fun. I hope that you find this entertaining as well. In a comment on the first episode a listener mentioned something called Solocast and said that the method bore a resemblance to the method that I was using. Here is his comment -------------------- 02 Comment #3 posted on 2026-04-03 07:49:58 by Reto It reminds me about Solocast Hi Whiskeyjack, I really liked your podcast and the topic. I cannot remember about your last, but the sound quality of this one was good on my mobile speakers :) The concept reminded me about the program from Norrist (another host on HPR), while similar does it have some differences HPR 3496 https://hackerpublicradio.org/eps.php?id=3496 As I am not on the future feed, I look forward to your next episode. Cheers, Reto -------------------- 03 End of comment. I did not recall having heard the episode on Solocast, but this sounded very interesting. Solocast was in HPR episode 3496 and was released by norrist on the 27th of December 2021. I listened to that episode and does indeed use use the same basic concept of recording short segments of audio and combining them later instead of creating one big recording and editing it with an audio editor. 04 The main difference is that the work flow that I described involves a lot of manual steps, while Solocast is a short Python program that automates the entire process of presenting your script, recording the segments, combining the segments, and filtering and normalizing the result. I won't try to describe Solocast in detail, instead I would recommend just listening to HPR episode 3496 to get norrist's explanation directly. -------------------- 05 While I wanted to make sure that I credited norrist with having come up with this concept four years before I did, this won't be the focus of this episode. Instead I will talk about audio filtering and various experiments that I ran on several different methods. 06 While looking at the source code for Solocast I noticed that it used a filtering method that resembled one used by Jivetalk, a podcast production program that caught the attention of one of the HPR community news presenters. This method involves taking a sample of quiet audio where there is no speaking taking place, and then using this as input to a noise reduction filter which is applied to the voice recording. The filter subtracts the quiet sample from the voice audio, which should theoretically remove the ambient noise. 07 I decided to apply this method to a number of different audio test recordings which were recorded under different circumstances using different hardware. In this way I could see if the method worked equally well under all circumstances or if there were some sorts of noise which it was suited to and some sorts that were not. 08 While I was at it, I also picked several other filter methods to see how they worked as well. Potentially, some methods may be better under some conditions while other methods were better suited to others. -------------------- 09 I won't present all of my experiments, as that would be a bit dull to listen to. Instead I will describe each method and then present audio samples which illustrate my conclusions. There are two pieces of audio software involved, both of which were also used in my series on simple podcasting. 10 The first is Sox, spelled s o x , and which is short for Sound Exchange. Sox is a command line program for audio manipulation. Sox is Free Software, released under the GPLv2 or later. The other is FFMPEG, which is also a command line program. FFMPEG is also Free Software, released under the LGPL V 2.1 or later, and GPL v 2 or later. Sox actually uses FFMPEG for certain operations. -------------------- 11 Audio Hardware For recording hardware I used the following. 12 Maxwell Headset The first is a cheap Maxwell headset that has an electrical noise problem. Unfortunately I don't have a model number for this headset. I described this hardware, the noise problems that I had with it, and how I created filters to deal with the noise in my series on simple podcasting. Briefly though, this is a headset that has a build in microphone on a boom which allows the microphone to be positioned close to the mouth. It connects with a USB cable. 13 Borne Earpiece and In-line Microphone This is a set of earplugs that go in your ears and connected by wires and a very small microphone built into a small bulge in the cable. It connects using a 3.5mm jack. The model number seems to be BUD250-BL. 14 XTrike Headset This is a gaming headset similar to the Maxwell headset described above. The model number is GH-510 It uses a USB connection. 15 Yanmai Condenser Microphone This is a microphone that comes with a small tripod stand. The model number is SF-910 It uses a 3.5mm audio jack. -------------------- 16 This is not a review of the hardware. Rather, I was trying to create audio problems so that I could test ways to fix them. Therefore, do not take the above list as a recommendation of what to buy. However, you can see that I am not using any expensive audio hardware. If you want to make an HPR podcast, you do not need professional level hardware. -------------------- 17 Audio Samples The audio samples are as follows 18 Quiet This was recorded in a quiet environment at my desk. This is my normal podcasting environment and represents optimal conditions. The main reason for this method is to see how the various filter methods perform when dealing with the electrical noise from the Maxwell headset. 19 Small fan This is a small USB powered table fan approximately 10 cm in diameter. It was located roughly 40 cm or less to the left of the microphone, although this varies depending on the microphone. 20 Traffic This was along a busy street with traffic noise in the background. -------------------- 21 Filter Methods Sox noisered Filter with Audio Profile This method uses the Sox noisered filter. Here is a brief quote from the Sox documentation on this filter. Quote Reduce noise in the audio signal by profiling and filtering. This effect is moderately effective at removing consistent background noise such as hiss or hum. To use it, first run SoX with the noiseprof effect on a section of audio that ideally would contain silence but in fact contains noise - such sections are typically found at the beginning or the end of a recording. End of quote For these tests I recorded a separate noise profile to go with each test. -------------------- 22 Basic Manual Filter This is a basic high and low pass filter pair based on the work I had done in my previous series on simple podcasting. However, based on the tests that I have done for this episode, I decided to get a bit more aggressive in terms of filtering. I use a high pass filter of 120 Hz, and low pass filter of 8 kHz. The each filter is then applied twice to increase its effect. I also added band reject filters to deal specifically with 50 and 60 Hz line noise. -------------------- 23 Complex Manual Filter This uses the manually constructed filter described in my series on simple podcasting. This uses the basic manual filter plus a series of custom bandreject filters to fix specific noise problems with the Maxwell headset. -------------------- 24 FFMPEG afftdn Filter The documentation describes this as "Denoise audio samples with FFT." -------------------- 25 FFMPEG arnndn Filter The documentation describes this as "Reduce noise from speech using Recurrent Neural Networks." -------------------- 26 FFMPEG agate Filter I will pronounce this as "agate" for convenience. The documentation describes this as "A gate is mainly used to reduce lower parts of a signal. This kind of signal processing reduces disturbing noise between useful signals." -------------------- 27 Method The experimental method used was to take each noise sample and apply the different filter methods to it. Where there are parameters which can be adjusted, a script was used to generate a series of different sample files with different parameter values. Not all possible parameters were experimented with, as the goal is to see which method produces what sorts of results under different circumstances, not to get the best possible result for the samples that I happen to have. The method in each case was as follows 28 Step 1 Convert the audio file to FLAC if it is not already in that format. 29 Step 2 Apply a basic high and low pass filter described previously to each sample. The reason for this basic filtering is that it eliminates at least some undesired noise in a fairly fool proof manner, leaving less for the more advanced filter to deal with. This should allow for a better test of the filter under realistic conditions. 30 Step 3 Apply the noise reduction filter being tested. 31 Step 4 Normalize the filtered sample to 17 LUFS according to the EBU R128 standard. The EBU standard is described in my series on simple podcasting. Normalizing adjusts the audio signal to a desired loudness level. This allows for more more consistent sound levels and allows us to hear the results under realistic conditions. I normalize the audio individually for each sample as different recording hardware requires different amounts of loudness adjustment. This is different from the typical podcast process where normalizing takes place as the very last step in the process, but it was necessary in this case. 32 Step 5 Concatenate selected sample audio files to one another to allow for better review and comparing. -------------------- 33 Results The results are grouped according to the type of noise which is being mitigated. This allows for easier comparison of the effectiveness of each technique under different circumstances. I have only picked a few examples of interest out of the numerous experiments that I conducted. -------------------- 34 Quiet Recording Environment with Maxwell Headset This compares how well the various filtering methods work on the noise induced by the electronics in the Maxwell headset. This electronic noise consisted of a noise spike every 1 kHz. This should be representative of electronic noise caused by problems in recording hardware. 35 Manual Filter The manual filter applied a narrow band reject filter every 1 kHz from 1 kHz to 12 kHz. This completely removed the otherwise audible whine caused by the noise. 36 FFMPEG afftdn This method allows for setting a noise floor and then specifying how much the noise floor should be reduced by. The method is very sensitive to getting the noise floor correct for that recording. Set the floor too low and nothing happens. Set it too high, and some distortion results. However it seemed to be moderately effective, but it would seem to require checking it and possibly adjusting it each time it is used. 37 FFMPEG agate This method allows setting a noise floor and then suppressing all sound which falls below that level. This method is very sensitive to getting the noise floor correct for that recording. If set too low (or quiet), it is ineffective. If set too high (or loud), it distorts words which come after a pause, which would typically be between sentences. 38 When set correctly, it completely removes noise in the silences between sentences. However, the noise is still audible during speech. This is because the noise in this case is a higher frequency than normal speech, and so stands out more. It may not be a significant problem for noise which is closer to the main vocal frequency band. Overall, this method is not suitable for this particular problem. 39 FFMPEG arnndn This method used the standard model. A variety of different noise reduction models are available. I only tested it with one, std.rnnn It does not seem to introduce much distortion in the voice signal even with a high amount of mix parameter. 40 However, it is only slightly effective at removing the whine from the signal, even with a high amount of mix parameter. Overall, this method does not appear to be useful for this sort of noise problem. 41 Sox noisered Filter This was effective in removing noise between words, but noise can be heard while words are being spoken. It was better than agate however. 42 Overall Conclusion for the Maxwell Headset Noise When dealing with narrow noise bands that occur at known frequencies, the manual filter is leagues ahead of any of the other tested alternatives. 43 Sample Audio Here is a sample audio recording showing the best overall results The sample is repeated, first with only basic low and high pass filtering, and then with the manually constructed filtering. In the first sample you should hear a high pitched background whine. In the second sample, the high pitched whine is completely removed. 44 (Audio sample inserted here.) -------------------- 45 Traffic Noise This was recorded using the Borne in-line microphone connected to a mobile phone while walking along beside a busy street. This was in dry cool spring weather, and the road was paved with asphalt. This should be reasonably representative of podcasting while walking outdoors in a noisy environment. 46 Basic Manual Filter This used the basic manual filter with high and low pass filters. This did nothing very useful in this case as the signal was already filtered within those limits by the recording hardware anyway. The low sample rate of 8 kHz in the phone limited the upper frequency to 4 kHz. Recall that the sample rate has to be twice the highest frequency that you want to detect. Overall, this is not suitable for this sort of problem. 47 FFMPEG afftdn With a high noise floor, background noise is reduced, but not eliminated. There was not much distortion in the voice. This is only slightly useful for this sort of problem. 48 FFMPEG agate With a high threshhold, background noise is reduced, but not eliminated. There was some distortion in the voice. The background noise could also be heard when speaking, but because the frequency of the background signal was similar to the louder voice signal, it was not as noticeable as it would have been if the two were very different. This is moderately useful for this sort of problem. It may be more useful in situations where the background noise was not quite as loud. 49 FFMPEG arnndn With high amounts of noise reduction, much of the background noise is suppressed, but there is not a lot of distortion in the voice. The background traffic noise is still present, but is significantly less. This offers only a moderate improvement. 50 Sox noisered Filter With small amounts of noise reduction voice is clear but traffic noise is present as a very significant continuous warbling sound in the background. This is no improvement on the original and in fact could be seen as making it worse. With moderate amounts of noise reduction, traffic noise is mostly gone, but there are still various squeaks present. Voice is noticeably distorted. With large amounts of noise reduction, traffic noise is gone but voice is highly distorted. This is moderately useful for this sort of problem, but requires careful adjustment. 51 FFMPEG arnndn Followed by FFMPEG agate This combined two different filters. First, it used arnndn to suppress the background noise to a lower level without much voice distortion. Then it applied the agate filter to suppress the noise levels between words still further. This used the same amount of mix and threshold as was found to be most effective when each of these filters was used on its own. The background noise is almost completely gone while distortion of the voice signal is low. 52 Overall Conclusion for Traffic Noise The arnndn combined with agate filters was the most successful at suppressing background noise while limiting the amount of voice signal distortion. 53 Sample Audio Here is an audio sample for what I felt to be the best overall results, the arnndn filter combined with the agate filter. First is the original audio with basic filtering. This is followed with the same audio after being passed through the arnndn and agate filters. 54 (Insert arnndn plus agate audio sample here) 55 Another Sample Here is a second audio sample showing the Sox noisered profile based filter. I have included this to show how a profile based filter can make things worse if you are not careful how you use it. This repeats the test audio 4 times. The first is with basic filtering only. The second uses low amounts of noise reduction. The third uses moderate amounts of noise reduction. The fourth uses high amounts of noise reduction. 56 (Insert noisered audio sample here) -------------------- 57 Small Fan Noise with Yanmai Microphone This was recorded using the Yanmai condenser microphone. A small fan was set up behind and to the left of the microphone. This is intended to represent situations where someone may have a fan or air conditioner running in the background due to hot weather, or has a loud computer fan. 58 A condenser microphone was used for this test as they are more prone to picking up unwanted noise. However, for practical recording purposes, this sort of microphone is unsuitable for this type of environment. 59 Basic Manual Filter This used the basic manual filter with high and low pass filters. This did nothing useful as the fan noise was in the same frequency range as the voice signal. This may be of more help in cases where the noise is below the 120 Hz cut off used in the low pass filter. 60 FFMPEG afftdn With high amounts of noise reduction, much of the background noise is suppressed, but there is some distortion in the voice. The background fan noise is still present, but is significantly less. Overall this is moderately effective. 61 FFMPEG agate This was effective in removing noise between words, but noise can be heard while words are being spoken. However, this was a small voice sample and it is possible that more problems could occur. With less fan noise than was in this sample this technique may work much better. 62 FFMPEG arnndn With high amounts of noise reduction, much of the background noise is suppressed, but there is not a lot of distortion in the voice. The background fan noise is still present, but is significantly less. Overall this was fairly effective. 63 Sox noisered Filter With small amounts of noise reduction voice is clear but fan noise is present as a slight warbling sound in the background. With moderate amounts of noise reduction, fan noise is gone, but voice is somewhat distorted. With large amounts of noise reduction, fan noise is gone but voice is very distorted. 64 In general this method is fairly successful at dealing with this sort of problem. However, there is a trade off between background noise and voice quality. Getting that trade off correct takes experiment and judgment for each specific situation. 65 FFMPEG arnndn Followed by FFMPEG agate This combined two different filters. First, it used arnndn to suppress the background noise to a lower level without much voice distortion. Then it applied the agate filter to suppress the noise levels between words still further. This got rid of virtually all of the background noise between words. If you listen carefully however, there is a slight buzzing sound in the voice signal. 66 Overall Conclusion for Fan Noise with Yanmai Microphone. Of the methods tested, the arnndn followed by agate filter seemed to offer the most improvement for the least effort and least voice distortion. The arnndn filter on its own seemed the next most preferable to me despite leaving some fan noise in the background. 67 Audio Sample Here is an audio sample for what I felt to be the best overall results, the arnndn filter combined with the agate filter. First is the original audio with basic filtering. This is followed with the same audio after being passed through the arnndn and agate filters. 68 (Insert audio sample here) -------------------- 69 Small Fan Noise Recorded with Headset The following is an observation rather than a filtering technique. When a recording was made using the Maxwell headset and listened to on the headset later or with speakers, the fan was virtually inaudible. When the same recording was listened to with the XTrike headset, it was barely audible with careful listening and only identifiable as a fan because I knew it was there. 70 In situations where there is ambient noise, the best noise reduction technique is probably to move the microphone as close to your mouth as possible, although not directly in front of it, and reduce the gain if there is a gain adjustment in the microphone. This will work far better than trying to remove the noise later. If you are recording an HPR episode at a desk, then an inexpensive headset with boom mike may do the job just fine with minimal effort and expense. -------------------- 71 Conclusions I have tested three noise scenarios - Electronic noise in the audio hardware at specific frequencies. Recording outdoors with an inline microphone in a noisy traffic environment. A noisy fan creating background noise in an office. My conclusions on these are as follows. 72 Electronic Noise in the Audio Hardware at Specific Frequencies If you can use Audacity or some other means to find the frequencies which are causing the noise, the best solution, assuming you don't just replace the hardware, is to manually construct filters to remove those specific frequencies. This is the safest solution in terms of only doing what you tell it to and not producing unexpected surprises some time down the road when something changed in the environment. 73 If you are looking for a fairly automatic filtering method, the Sox noisered profile based filter seems to work fairly well. There is an equivalent filter in ffmpeg, but I did not include that in my experiments as it is harder to use in a script because it does not use a separate noise profile file. 74 Recording Outdoors with an Inline Microphone in a Noisy Traffic Environment. In this situation, the FFMPEG arnndn combined with agate filters seem to be the most successful. The Sox noisered filter may work, but at the cost of more distortion in the voice than is seen in the other methods. 75 An inherent problem with any profile based noise reduction method is that if the background noise is not constant, which it seldom is in that sort of environment, the profile may not represent the background noise which is present later on in the recording. This risks adding more distortion in the voice as the profile and later environments diverge. 76 However, for this application a different microphone that provided a better recording would appear to be advisable. A solution which brought the microphone much closer to the mouth and so resulted in a better ratio of voice signal compared to background noise would appear to be necessary, after which the question of what sort of noise reduction to use would need to be re-evaluated. 77 A Noisy Fan Creating Background Noise in an Office. The Sox noisered filter and the FFMPEG arnndn, afftdn, and agate methods all work to some degree. However, they all need correct selection of parameters to achieve the proper results. When I compared all four methods side by side, I found the arnndn combined with the agate filter to be preferable in terms of the trade off between background noise reduction and distortion of the voice signal. The arnndn filter on its own seemed the next most preferable to me despite leaving some fan noise in the background. 78 However, that is a subjective judgment of a specific noise sample when recorded using a specific microphone. Keep in mind though that many listeners will not be listening in an idea environment. They may be doing things where background noise is present rather than in a very quiet room and so may find a small amount of background noise in the recording to be less of a problem than distortion in the voice signal which may make some words harder to understand. 79 When I conducted the same experiment recorded with the XTrike headset I found that arnndn seemed to offer no noticeable improvement. This may be because the amount of audible fan noise was far less with the XTrike headset to begin with. In other words, there is no single best solution here, and you may have to be prepared to try different options to see which one works in your situation. The important thing is to avoid making things worse by applying filtering that is not appropriate for that situation. The best method may be to use a recording method that doesn't pick up the fan noise to begin with. This can include just using a gaming headset with boom mic. 80 I have one final observation on this point regarding headsets. The Maxwell headset has a foam cover over the microphone while the XTrike headset does not. There was some slight audible wind buffeting noise picked up by the XTrike headset that was not observed with the Maxwell. This seemed to cause particular problems with the Sox noisered profile based filter, as this noise was irregular and after filtering would show up as a warbling sound. If you use a headset and plan to use it in conjunction with a fan, it may be advisable to apply some sort of wind cover over it. 81 Combining Complex Filters In several cases I found that combining several complex filters offered better results than using any single one on its own. The basic strategy though is to first use a method which is good at reducing undesirable noise without introducing excessive voice distortion. Then apply a different filter which is good at reducing small levels of background noise to an even lower level while affecting the voice signal as little as possible. This uses the relative strengths of different filter types to compensate for the weaknesses of the other. 82 Different combinations of filters were most effective for different types of problems. I did not try all possible combinations however. Perhaps a further exploration of this would be worth doing in a later podcast. -------------------- 83 Case Study - Noise in Another HPR Episode Audio In the comments to my second episode on Simple Podcasting (which is HPR4618) where I discussed basic filtering, a couple of listeners brought up an interesting point. Antoine mentioned "declicking" in a post. -------------------- Vance replied 84 Antoine, thanks for mentioning the click removal capability in Audacity! While I already knew about its noise removal filter, I wasn't aware it also had click removal. It might have helped me for HPR4637, where some sort of electromagnetic signal was picked up by my microphone/recorder, a Zoom H2 (the tapping sound was *not* present in the room where I recorded). While click removal does seem to distort speech when applied to it (though to my ears, it doesn't sound as weird as when noise removal is done with speech), I could have applied the filter only to the pauses, where the "tapping" is most noticeable. I will consider doing this in the event that I'm not able to eliminate the source of interference in the future, which would be the best way to go. -------------------- 85 End of quote. I found this interesting as it sounded like another audio problem that could be experimented with. I found a sample of the episode which had the clicks and cut a copy of that segment out to experiment with. These sounds are a series of clicks, or "ticks" would be another way to describe them, in the quiet part of the audio between sentences or phrases. 86 Next I used Audacity to study the sound spectrum. I found a massive 60 Hz noise spike. However, my speakers won't reproduce sound that low, and filtering this out didn't reduce the clicks. The clicks turned out to be bursts of noise across the 100 to 800 Hz band, which is right where the main vocal band also is. This makes it difficult to filter based on frequency. The most promising approach would seem to be to filter based on sound level. 87 I tried all of the individual audio filter techniques mentioned in the other experiments above. None produced satisfactory results except for agate, which makes quiet audio quieter. This completely suppressed the clicks. However, when applied to the entire episode it also distorted the start of a few sentences which began with single short syllables. 88 The agate filter has a number of parameters which could be adjusted to try to deal with these cases, although I did not spend the time to do so. Another solution to this distortion problem is to simply not apply the filter to those parts of the audio which are affected. If you record the audio as a series of small individual files, it would be easy enough to filter before concatenating the files together while skipping those files which contain audio which is not suited to this method. Here are the results of the experiments. 89 FFMPEG afftdn This reduces the size of of the ticks, but they are still present. However, they may be reduced to a level which is considered acceptable. 90 FFMPEG agate This was very effective in removing ticks with the right parameters. However, it can introduce some voice distortion in the form of cutting out the start of a few sentences which began with single short syllables. This can be corrected with a very short "attack" parameter to turn off the filter when it detects sound above a set threshhold. 91 FFMPEG arnndn This was relatively ineffective. 92 Sox noisered This was effective in removing the sounds between phrases. However, it introduces some distortion in the voice signal. 93 I also tried combining filters. FFMPEG afftdn Followed by agate This combined two different filters. First, it used afftdn to suppress the background noise to a lower level without much voice distortion. Then it applied the agate filter to suppress the noise levels between words still further. This got rid of virtually all of the background noise between words. 94 Here is a short audio sample from HPR4637. First is the unfiltered audio. Second is the filtered audio using the combined afftdn plus agate filters. Since the "clicks" are very quiet, you may not hear them unless you are in quiet environment. Quite a few listeners would probably not be aware of the perceived audio problem in this episode if it had not been discussed here. None the less, it makes for an interesting experiment. Here it is: 95 (Insert sample audio here) 96 Overall Conclusion for Noise "Ticks" The afftdn combined with agate filters seemed to offer the best overall results when used with the right parameters. However, the author, Vance, speaks very clearly and evenly, and so his voice is ideally suited for use with this filter. Another author's voice may not be as suited to this filter. 97 The Sox noisered profile based filter offers various degrees of trade off between suppressing noise and distorting the voice signal. As to whether this is an acceptable trade off depends on the particular voice in question and how easily understood it is under normal circumstances with out additional distortion. The afftdn filter may be a fairly safe filter to use on its own while producing acceptable if not perfect output. -------------------- 98 Overall Conclusions I have presented only a few of the experiments that I conducted. My overall conclusion after all of this is that there is no universal audio filtering method that works best in all circumstances. There are instead a number of tools in the toolbox, and picking the right one for the job takes a bit of trial and error. 99 However, if you have a repeatable recording environment, then once you have decided what tool you need you should create a script for it so you can have a repeatable processing setup. These conclusions apply to voice podcasting. Music has a different set of criteria and techniques that work well with basic voice podcasting may produce poor results when applied to music which has a broader range of frequency and just as importantly, a broad range of loudness. 100 If you are used to using filters and effects in Audacity, many of the settings on those correspond to arguments in the command line version of ffmpeg. It is worth learning how to use ffmpeg directly to automate your recording process. 101 The experiments that I conducted were greatly assisted by writing scripts which created multiple versions of audio files with different settings, thereby allowing me to try many different alternatives relatively easily. It also allowed me to concatenate different audio samples into a single audio file and so listen to different versions in quick succession, making subjective listening judgments more reliable. 102 It is important to keep in mind in all this that I am playing with audio filtering mainly to have fun. It is not necessary to do any of this if you think your podcast episode sounds just fine without it. So, don't let any of what I have talked about in all this discourage you from simply recording a podcast and sending it in as is. I will include copies of the filters I have described here in the show notes. -------------------- 103 Related Matters Hardware Characterization Using Audio Signals I found it useful to characterize the hardware that I had in order to understand its limitations better before starting the experiments. This involved playing a signal out through a set of speakers and then recording it through a microphone. 104 I used two types of signal for this. One is type of signal is known as a "chirp" signal. This is a sine wave that steadily increases in frequency as it sweeps across the audio spectrum. The standard audio range is 20 Hz to 20 kHz, but for my purposes I limited the upper frequency to 15 kHz to save time as anything beyond that is not very useful for voice podcasts. 105 By recording the chirp signal with a microphone and analyzing it with a Fourier transform, I could quickly see what each device was capable of. See my previous series on simple podcasting for an explanation of what a Fourier transform is and what software to use to see the results of it. Here is a chirp signal. 106 (Insert Audio Sample Here) 107 In addition to a chirp signal, I also used a series of simple tones of specific frequencies. By using these tones of known frequency I could gain an understanding of the limitations of my speakers and headphones, and just as importantly, my own ears. By understanding these limitations I was able to narrow the range of frequencies that I need to deal with quite considerably and set the high and low pass filters accordingly. These tones are a series of flac files generated with ffmpeg. 108 Here is a a sample audio tone at a 2 kHz frequency. 109 (Insert Audio Sample Here) 110 Copies of the script to create the chirp signal and the tones are in the show notes. -------------------- 111 A "Not a Review" of some of the Hardware that I Used I said that I would not do a review of the hardware that I used. However, some of it deserves mention for either how good or bad it was. I will record each section using the hardware being described. 112 Maxwell Headset This is my original recording hardware. This is a headset with boom mic and USB connection. There is no model number on it, so I don't know the model. This probably cost somewhere between 10 and 25 dollars. The earpieces sit on the ears and do not fully enclose them. This makes it light weight and comfortable to wear for extended periods of time. It has a problem however with electronic noise consisting of a noise spike every 1 kHz. I was able to fix this with a series of filters using FFMPEG. Fixing this problem is what got me started in understanding audio. I will probably continue to use this headset to make podcasts. 113 XTrike Headset, Model GH-510 This is also a headset with boom mic and USB connection. I purchased this headset for the purposes of experimentation for this podcast episode. It cost $12.88. I found it to be surprisingly good for the price. It has fully enclosed ear pieces however, which may make it uncomfortable to wear in hot weather. I may try doing some of my future podcasting using this headset. 114 Borne Earpiece and In-line Microphone This is a set of earplugs that go in your ears and connected by wires and a very small microphone built into a small bulge in the cable. It connects using a 3.5mm jack. The model number seems to be BUD250-BL. It cost approximately $3.00. I bought several sets of these and use them for listening to podcasts from an MP3 player. The ear pieces are pretty good for listening with. The microphone works reasonably well when used in a quiet location. It is less good when in a noisy environment. It is very important however to secure the microphone to your lapel or other location reasonably near your mouth and to point the microphone (that is the small hole) outwards and not simply let it dangle freely. If you let it just hang, you will get poor quality and inconsistent audio. 115 Yanmai Condenser Microphone, Model SF-910 I purchased this microphone for the purposes of experimentation for this podcast episode. It cost $3.88. As it is a condenser microphone, it is prone to picking up background noise more and as such is probably not a good choice for podcasting by single person sitting at a desk. However, it is none the less a surprisingly good microphone for surprisingly little money. 116 iCan USB Microphone, Model M-306 I purchased this microphone for the purposes of experimentation for this podcast episode. This has a USB connection. This was also relatively inexpensive at $7.99, or roughly twice the price of the Yanmai microphone. Unlike the Yanmai however, it is absolutely wretched. There was such a high degree of distortion when recording through it that I found I could not use it in the fan experiments which I had bought it for. I ended up buying the Yanmai microphone for that instead. -------------------- 117 Easy Effects Software The techniques described so far all involve recording audio files and then processing them later to produce the desired result. This is probably the simplest and most straightforward way of doing things if you are making a typical podcast. However, there may be instances where you want to apply filtering or other effects on the "live" signal immediately and not after the fact. 118 There is audio software which can hook into your computer's audio system and do this with a live signal. For Linux, there is a package called "Easy Effects". This is Free Software and comes under a GPL V3 or later license. I installed it from the Debian repository under Ubuntu 24.04. 119 You can create various filters and even chain them together to combine them. I played with it a bit but do not know enough about it to discuss it seriously at this time. However, I thought it would be worth mentioning for the sake of those who may wish to try it out themselves. -------------------- 120 Episode Conclusion After having had some fun with audio and listening to other HPR members talk about audio, I thought I would have some more fun by playing with noise reduction filters. I have no intention of becoming an audio professional, but by doing some experiments I learned a few things and had some fun doing it. I hope that the rest of you found this interest as well. I will see you all again later in another episode of Hacker Public Radio. -------------------- Scripts Basic Filter This shows basic high and low pass filters ( 120 Hz and 8 kHz respectively) and band reject filters for 50 and 60 Hz. # The high and low pass filters. hlpfil="highpass=f=120, highpass=f=120, lowpass=f=8000, lowpass=f=8000" # Band reject filters filter for 60Hz and another for 50Hz. linefil="bandreject=f=60:width_type=h:w=20, bandreject=f=50:width_type=h:w=20" # Filter using ffmpeg. ffmpeg -i inputfile.flac -af "$hlpfil, $linefil" outputname.flac # ====================================================================== afftdn Filter # noisefloor should be between 20 and 80. noisefloor=$1 # Run the noise reduction. ffmpeg -i testrec-filtered.flac -af "afftdn=nr=10:nf=-""$noisefloor" tmptestrec.flac # ====================================================================== agate Filter # threshold shoud be between 10 and 80. threshold=$1 # Run the noise reduction. ffmpeg -i testrec-filtered.flac -af "agate=threshold=-"$threshold"dB:range=-60dB" tmptestrec.flac # ====================================================================== arnndn Filter # mix should be between 0 and 1. mix=$1 # Run the noise reduction. ffmpeg -i testrec-filtered.flac -af 'arnndn=model=std.rnnn:mix='"$mix" tmptestrec.flac # ====================================================================== sox noisered Filter # Generate the noise profile from a sample of background noise. sox silencefiltered.flac -n noiseprof noise.prof # nramount shoudl be between 0 and 1 sox testrec-filtered.flac noiseout-testrec.flac noisered noise.prof "$nramount" # ====================================================================== Manual Filter for Maxwell Headset Noise # Create a series of band reject filters, from 1 kHz to 11 kHz. ftemplate="bandreject=f=%s000:width_type=h:w=100" kilospikefil=$( seq 1 11 | xargs printf "$ftemplate," ) # Using ffmpeg ffmpeg -i testrec-filtered.flac -af "$kilospikefil" tmptestrec.flac # ====================================================================== Create a "chirp" signal # Start frequency. f0=20 # End frequency. f1=15000 # Duration of signal. duration=10 ffmpeg -f lavfi -i "aevalsrc=sin(2 * PI * (0.5 * ($f1 - $f0)/$duration * t^2 + ($f0 * t))):s=44100:d=$duration" -c:a flac -af "aformat=sample_fmts=s16" chirp.flac # ====================================================================== Generate Audio Tones toneout () { printf -v freqval "%05d" $1 ffmpeg -f lavfi -i "sine=frequency=$freqval:duration=3" tmptone.flac # Normalize ffmpeg -i tmptone.flac -af loudnorm=I=-17:TP=-2.0:LRA=4.0 -ar 44.1k -sample_fmt s16 tone$freqval.flac rm tmptone.flac } # List of frequencies in hertz. freqlist="50 60 100 120 130 140 150 160 170 200 500 1000 2000 3000 4000 5000 6000 7000 8000 9000" for freq in $( echo $freqlist ); do toneout $freq done # ====================================================================== Provide feedback on this episode.

This show has been flagged as Clean by the host. This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems. Most users of UNIX-like systems are probably familiar with the diff utility. It is widely used with source code to compare two files and see what the differences are between them. Non-programmers, like me, also use it to examine what has changed in different versions of scripts or configuration files. Quite a few pieces of newer software can compare different versions of data and express changes in a format either identical to or similar to diff output. However, there are two other long-standing tools for this purpose that are far less known and deserve in my view to be termed UNIX Curios. The first of these is cmp 1 . While diff is primarily intended to be used on text files and compares them line by line, cmp compares files byte by byte. In my experience, its main use is to see whether two binary files are in fact identical—if they are, cmp outputs nothing and returns an exit status of 0. Back when methods of transferring files were not as reliable as they are today, this was a tool I would reach for sometimes. For example, you could use it to confirm that the data on a CD-ROM you burned was the same as the original. If there is a difference between the files, cmp will return an exit status of 1. By default, it will also print the location (byte and line number) of the first differing byte. When used with the -l option, it will print the location and value of every byte that differs. There is one exception to these: if the files are the same except that one is shorter than the other, it will print a message to that effect. The exit status will still be 1 in that case. Using the -s option with cmp will cause it to be totally silent and output nothing. Only the exit status will indicate whether the files are the same, different, or if the exit status is greater than 1, that an error occurred. This makes it useful for scripting, for example in case you wanted to confirm that a file copied to another location arrived fully intact. It is worth noting that diff is also capable of comparing binary files—however, it is not required by POSIX to report what is actually different or where differences occur. The same exit status as in cmp is returned: 0 if the files are the same, 1 if they are different, or greater than 1 if an error occurred. While many implementations offer an option to suppress the output, this is not in the standard 2 so the most portable method would be to instead redirect output to /dev/null . On my system the diff utility is three times the size of cmp , so if you don't need its extra capabilities, it is a less efficient way of doing the job. The other UNIX Curio for today is comm , and this utility 3 is also intended to compare two files to see what is common between them. Ken Fallon briefly talked about it a few years ago in HPR episode 3889 . Compared to the others, it has a much more specific use case. The two files are expected to be text files that are already sorted. What comm will do is print a tab-separated list of all the lines appearing in either or both files. Lines only in the first file will appear in the first column, lines only in the second file will be in the second column, and lines in both files will be in the third column. Any combination of the options -1 , -2 , and -3 can be used with comm to suppress printing of the first, second, or third column respectively. Using all three options at the same time is supported but it results in no output, so that isn't very useful. Unlike the other utilities, the exit status of comm doesn't tell you anything about the two files. It will be 0 if the program ran successfully, and greater than 0 if it didn't. I'm not sure if I have ever actually used comm for anything practical. I find its default output a bit difficult to meaningfully interpret, plus you need to ensure the two files are already sorted. It seems to be best suited to comparing lists, and one use case that Ken Fallon mentioned would be comparing two lists of files to see if any are missing. The command comm -3 listA listB would print files that only appear in listA in the first column and those only in listB in the second column. This would let you ignore all the filenames that appear in both and focus on those that were absent from one or the other. If on the other hand you only wanted to see the filenames that are on both lists, comm -12 listA listB would give you that. Some more frivolous potential uses also come to mind. If for some reason the cat utility is broken on your system, you could use comm listA /dev/null to print the file listA instead. If you want to insert tab characters before every line of a file but have an aversion to using sed or awk , then comm /dev/null listA would output listA with one tab before each line, and comm listA listA would insert two tabs. A bit silly, but it would work. The GNU implementation of comm even lets you choose something other than a tab to separate the columns 4 , so you could go wild with that. According to the POSIX specifications for cmp and comm , one of the two filenames given as arguments, but not both, can be a " - ", in which case standard input will be used for that "file" in the comparison. Also, the results are undefined if both arguments are the same FIFO special, character special, or block special file. Some implementations might not have these limitations, but you shouldn't rely on that everywhere. All three of these were developed quite early. The cmp utility appeared in 1971's First Edition UNIX 5 , while comm and diff seem to have made their debut in Fourth Edition UNIX 6,7 from 1973. The original versions might not have behaved exactly like their modern counterparts, and newer implementations (especially of the diff utility) have acquired additional options and capabilities, but the basic operation of each has stayed the same. The next time you need to compare files against each other, consider whether cmp or comm might be appropriate before automatically reaching for diff . They all have their uses in different situations. References: Cmp specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/cmp.html Diff specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/diff.html Comm specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/comm.html GNU coreutils manual: comm https://www.gnu.org/software/coreutils/manual/html_node/comm-invocation.html First Edition UNIX cmp manual page http://man.cat-v.org/unix-1st/1/cmp Fourth Edition UNIX comm manual page https://www.tuhs.org/cgi-bin/utree.pl?file=V4/usr/man/man1/comm.1 Fourth Edition UNIX diff source https://www.tuhs.org/cgi-bin/utree.pl?file=V4/usr/source/s1/diff1.c Provide feedback on this episode.

This show has been flagged as Clean by the host. Sound quality, ease of use, comfort, battery life, price https://themusicozy.com/products/headband/index.htmlProvide feedback on this episode.

This show has been flagged as Clean by the host. HPR and Glasses HPR community members are nerds We know from movies that nerds wear glasses We can conclude that HPR community members wear glasses Cleaning Background I have very oily skin and am constantly smudging my glasses All lenses wipes are too small. I accidentally touch the lenses with my fingers when I am trying to clean them And smudge them up again I heard an eye doctor say, the best way to clean glasses is with dish soap and dry with a microfiber cloth I tried this method It is a great solution because you wash glasses and fingers tips at the same time Cleaning Method Hand soap is OK to use on glasses. get the soap on you fingertips Then hold your glasses one of the lenses Use your thumb and finger tips to clean the glasses the soap removes the excess oil from your fingertips no smudges when you dry with the microfiber cloth Progressives With regular progressives the top of the lenses is for distance vision and the bottom of the lenses is for up close vision There is a zone in the middle for medium distance, but it is very narrow Using a computer at work I kept having to move my head up and down to find the best position for working. Computer Glasses have more space for intermediate distances From https://iotlenses.com/blog/eight-types-of-progressive-lenses.html Computer progressive lenses, also known as occupational designs or enhanced readers, are optimized for intermediate and near vision, making them ideal for tasks like prolonged computer work or activities at mid-range distance Provide feedback on this episode.

This show has been flagged as Explicit by the host. Background This episode describes the contents of an old fishing tackle box I got as a birthday present very many years ago. I used it over the years to gather a pile of mostly useless electronic components and junk. I came across it recently when we were doing a clear out. Mrs X wanted to throw it out. I had a lot of sentimental value due to the long time I've had it. I am a bit of a hoarder and it took a bit of persuasion on my part to be able to keep it especially since I could not think of a single use for any of the contents! I've not had time to add links against all the items. Hopefully I've managed to add links to most of the more unusual items. Apologies if I've missed any. Contents of Box Top row left to right, back to front Compartment 1 Stereo Jack plug & socket Double pole double throw (2-way) switch 12V dc relay from RS 3 legged semiconductor (voltage regulator?) Compartment 2 Multiple fuses mostly for cars Compartment 3 Small silica jelly pouch Micro Switch Reed switches (Not reed relays as I call them in my show) Compartment 4 Potentiometers, Rheostat, variable resistors https://en.wikipedia.org/wiki/Potentiometer Control knobs Compartment 5 Large selection of miniature panel mounted toggle switches https://en.wikipedia.org/wiki/Switch#Toggle_switch Compartment 6 Two crocodile clips https://en.wikipedia.org/wiki/Crocodile_clip Half of a Hi Fi speaker plug https://en.wikipedia.org/wiki/File:Speaker_din_male_and_female.jpg Washer Compartment 7 Mostly chocolate block connectors https://en.wikipedia.org/wiki/File:Leuchterklemme_75dpi.jpg Small selection of glass cartridge Fuses Single 13 Amp mains fuse for UK plug Assortment of soldering iron tips Crocodile clip Air adapter for foot pump for blowing up an air bed or similar Spare nozzle for a de-soldering tool. Made from PTFE https://en.wikipedia.org/wiki/Desoldering#/media/File:Vacuum_plunge.jpg https://en.wikipedia.org/wiki/Polytetrafluoroethylene Ferrite ring https://en.wikipedia.org/wiki/Ferrite_bead Compartment 8 Rubber Grommets and a heatsink Middle row left to right, back to front Compartment 1 Large selection of glass cartridge fuses Compartment 2 Another large selection of glass cartridge fuses Small selection of diodes, some quite Large https://en.wikipedia.org/wiki/Diode Compartment 3 Two multi turn precision variable resistors https://en.wikipedia.org/wiki/Potentiometer#/media/File:Beckman_Helipot_potentiometer_SA1400A_2007.075.002.jpg Cheap variable resistor with in built switch Compartment 4 Bag of multi coloured LED's (Cheap for Pi add on board) 2 inch 8 ohm speaker Compartment 5 Panel mounted heavy duty toggle switch, two pole one way Compartment 6 Light Dependent Resistor (LDR) https://en.wikipedia.org/wiki/Photoresistor Large Tricolour LED Compartment 7 Variable capacitors PCB (Printed Circuit board) mounted Spare nozzle for de-soldering tool. Made from PTFE Light Dependent Resistor (LDR) Panel mounted heavy duty rotary switch, two pole one way Bag of various fuses Halogen 12V car headlight bulb Variable resistor with long plastic spindle, panel mount Heatsink possibly for heavy duty transistor Compartment 8 Variable resistor PCB (Printed circuit board) mounted Bottom row This is just a big jumble of stuff, listed in no particular order DC red and black power lead with inbuilt fuse Small solder-less breadboard for prototyping https://en.wikipedia.org/wiki/Breadboard Bag of various plugs & connectors mostly Jack plugs and sockets Bag of quartz crystals and a components receipt from 1985 I added crystals to an old PMR (Private mobile radio) boot mount radio. Possible a Pye Westminster Radio. I looked but couldn't find picture of the radio Several bits of Perfboard or strip board https://en.wikipedia.org/wiki/Perfboard 1 Watt audio amplifier kit with instructions not built! D-type serial port mounted to PCB with screw down connections Bag of resistors https://en.wikipedia.org/wiki/Resistor Large bag of LED panel mount holders Electronic Instruction guide XH79L (Maplin) price 25p Chocolate block connecting block https://en.wikipedia.org/wiki/File:Leuchterklemme_75dpi.jpg Telephone pickup for recording phone calls. (Never worked) https://www.amazon.co.uk/Black-Telephone-Pickup-Sensitive-Microphone/dp/B000L1OZG8 Multi coloured ribbon cable Plastic trimming tool screwdriver set. https://www.amazon.co.uk/C-K-T4857-Precision-Plastic-Trimmer/dp/B000WKIYTW Panel mounted BNC socket (Baby Neill Constant) https://en.wikipedia.org/wiki/BNC_connector Provide feedback on this episode.

This show has been flagged as Clean by the host. Openreads is a privacy oriented and open source cross-platform app written in Flutter available for Android and iOS for keeping track of your books. There are four lists provided so you won't get confused: books you finished, books you are currently reading, books you want to read later, books you didn't finish. You can use custom tags and filter through them. A book can be added by: looking it up in the Open Library database, scanning its barcode, adding its details manually. You can also view some cool statistics! Links: Battletech: Mercenary's Star by William H. Keith Permaculture for the Rest of Us by Jenni Blackmore Piranesi by Susanna Clarke Battletech: The Price of Glory by William H. Keith The Urban Homestead by Kelly Coyne Battletech: The Sword and the Dagger by Ardath Mayhar Battletch: Warrior En Garde by Michal A. Stackpole Sisterhood of Dune by Brian Herbert and Kevin J. Anderson Fundamentals of the Wudang Sword Method by Scott M. Rodell Battletech: Warrior Riposte by Michal A. Stackpole The Cardio Code by Kenneth Jay Mentats of Dune by Brian Herbert and Kevin J. Anderson How to Grow More Vegtables by John Jeavons Provide feedback on this episode.

This show has been flagged as Clean by the host. From https://en.wikipedia.org/wiki/Simon_(game) Simon is an electronic game of short-term memory skill invented by Ralph H. Baer and Howard J. Morrison, working for toy design firm Marvin Glass and Associates, with software programming by Lenny Cope. The device creates a series of tones and lights and requires a user to repeat the sequence. If the user succeeds, the series becomes progressively longer and more complex. Once the user fails or the time limit runs out, the game is over. The original version was manufactured and distributed by Milton Bradley and later by Hasbro after it took over Milton Bradley. Much of the assembly language code was written by Charles Kapps, who taught computer science at Temple University and also wrote one of the first books on the theory of computer programming. Simon was launched in 1978 at Studio 54 in New York City and was an immediate success, becoming a pop culture symbol of the 1970s and 1980s Provide feedback on this episode.

This show has been flagged as Explicit by the host. New hosts There were no new hosts this month. Last Month's Shows Id Day Date Title Host 4630 Fri 2026-05-01 Playing Civilization V, Part 11 Ahuka 4631 Mon 2026-05-04 HPR Community News for April 2026 HPR Volunteers 4632 Tue 2026-05-05 Hackerpublic Radio New Years Eve Show 2026 Episode 6 Honkeymagoo 4633 Wed 2026-05-06 Ham Radio Licence Lee 4634 Thu 2026-05-07 Upgrade Failsause operat0r 4635 Fri 2026-05-08 What did I do at work today? Part 3 Section 1 Lee 4636 Mon 2026-05-11 7 seconds memory Antoine 4637 Tue 2026-05-12 UNIX Curio #6 - at and batch Vance 4638 Wed 2026-05-13 Simple Podcasting - Episode 3 - Analyzing and Filtering Whiskeyjack 4639 Thu 2026-05-14 NLUUG Spring Conference 2026 Ken Fallon 4640 Fri 2026-05-15 Robert A. Heinlein Ahuka 4641 Mon 2026-05-18 Technical Dutch Open Source Event (T-DOSE) Ken Fallon 4642 Tue 2026-05-19 Hackerpublic Radio New Years Eve Show 2026 Episode 7 Honkeymagoo 4643 Wed 2026-05-20 HPR Beer Garden 13 - Triple IPA Kevie 4644 Thu 2026-05-21 Response to comments on HPR4424: Newsboat... Archer72 4645 Fri 2026-05-22 ZERO HOUR: FRIDAY AFTERNOON APK HACKING operat0r 4646 Mon 2026-05-25 Mobile Gaming Elsbeth 4647 Tue 2026-05-26 UNIX Curio #7 - Compression Vance 4648 Wed 2026-05-27 Simple Podcasting - Episode 4 - Audio Analysis Fun Whiskeyjack 4649 Thu 2026-05-28 What did I do at work today? Part 3 Section 2 Lee 4650 Fri 2026-05-29 Playing Civilization V, Part 12 Ahuka Comments this month Past shows hpr4424 (2025-07-17) "How I use Newsboat for Podcasts and Reddit" by Archer72. أحمد المحمودي said: "How did I find HPR" (2026-05-12 17:16:07) candycanearter07 said: "Re: How did I find HPR" (2026-05-12 19:36:04) hpr4502 (2025-11-04) "Cheap Yellow Display Project Part 3: Reverse beacon network " by Trey. Archer72 said: "Morse code" (2026-05-19 15:00:39) hpr4567 (2026-02-03) "Movie Recommendations for Hackers" by Deltaray. Antoine said: "Some watched!" (2026-05-27 03:33:12) hpr4587 (2026-03-03) "UNIX Curio #1 - Shell Archives" by Vance. Dave Morriss said: "Great reminder! I had forgotten shar" (2026-05-07 18:18:05) Vance said: "Color printing" (2026-05-08 22:02:23) hpr4607 (2026-03-31) "UNIX Curio #3 - basename and dirname" by Vance. Vance said: "Correction" (2026-05-12 01:03:12) hpr4618 (2026-04-15) "Simple Podcasting - Episode 2 - Basic Filtering" by Whiskeyjack. Henrik Hemrin said: "Generating sine wave" (2026-05-14 19:59:46) Whiskeyjack said: "Reply to Henrik Hemrin on Sine Waves in HPR4618" (2026-05-15 15:22:16) hpr4627 (2026-04-28) "UNIX Curio #5 - Faster, Pussycat! Kill! Kill!" by Vance. Steve Barnes said: "Thanks for the context!" (2026-05-21 05:28:05) hpr4628 (2026-04-29) "Nuclear Power Technology Follow Up" by Whiskeyjack. Antoine said: "o/" (2026-05-17 00:24:23) Whiskeyjack said: "Reply to Antoine on HPR4628" (2026-05-17 18:08:18) This month's shows hpr4631 (2026-05-04) "HPR Community News for April 2026" by HPR Volunteers. Whiskeyjack said: "Response to Ken Fallon in HPR4631 Community News" (2026-05-04 05:13:02) candycanearter07 said: "new episodes" (2026-05-04 18:47:39) Ken Fallon said: "new candycanearter07 episodes" (2026-05-05 09:00:00) أحمد المحمودي said: "Thanks for the encouragement" (2026-05-12 17:18:20) hpr4634 (2026-05-07) "Upgrade Failsause" by operat0r. YourName said: "Why is the audio so bad OMG sorry" (2026-05-06 20:05:43) Ken Fallon said: "Already reported and fixed" (2026-05-06 20:35:49) hpr4637 (2026-05-12) "UNIX Curio #6 - at and batch" by Vance. candycanearter07 said: "still useful!" (2026-05-13 03:43:37) norrist said: "at for scheduled reboots" (2026-05-16 19:41:04) Vance said: "Good points" (2026-05-17 03:03:47) Whiskeyjack said: "At and batch in HPR4637" (2026-05-17 18:35:11) hpr4640 (2026-05-15) "Robert A. Heinlein" by Ahuka. Antoine said: "Nice tips" (2026-05-17 00:22:01) Kevin O'Brien said: "I enjoyed doing it" (2026-05-17 19:14:19) hpr4646 (2026-05-25) "Mobile Gaming" by Elsbeth. candycanearter07 said: "my opinion of mobile gaming" (2026-05-27 00:24:58) hpr4647 (2026-05-26) "UNIX Curio #7 - Compression" by Vance. xmanmonk said: "Great Series" (2026-05-27 00:29:32) candycanearter07 said: "thoughts" (2026-05-27 13:20:13) Mailing List discussions Policy decisions surrounding HPR are taken by the community as a whole. This discussion takes place on the Mailing List which is open to all HPR listeners and contributors. The discussions are open and available on the HPR server under Mailman. The threaded discussions this month can be found here: https://lists.hackerpublicradio.com/pipermail/hpr/2026-May/thread.html Events Calendar With the kind permission of LWN.net we are linking to The LWN.net Community Calendar. Quoting the site: This is the LWN.net community event calendar, where we track events of interest to people using and developing Linux and free software. Clicking on individual events will take you to the appropriate web page. Provide feedback on this episode.

This show has been flagged as Clean by the host. In our sample game we look at playing as Austria and aiming for a Diplomatic Victory. And our focus is on puppeting Citty-States, but be I misunderstood and instead of making a Diplomatic Victory easier, it makes it harder. I still managed to get my Diplomatic Victory, but a Science or Domination Victory would definitely have been easier in this scenario. Playing Civilization V, Part 12 A Diplomatic Victory Strategy Civilization V introduced a new Victory type and I thought it might be fun to try this strategy for a sample game to see broadly how this would work. I decided I would play as Austria on Prince level, which means that all players are equal and no one is favored. For my map I chose Fractal, Map Size = Standard, Game speed = Standard. My only Advanced Option was Quick Combat, because I didn't need to see that drawn out, particularly since I plan to avoid combat as much as necessary. My leader is Maria Theresa, and her Austria has a unique ability called Diplomatic Marriage. This allows us to either Annex or Puppet any city-state that has been allied to us for 5 turns with the proper expenditure of Gold. My plan was to use this to Puppet the city-states to control their votes for the Diplomatic Victory, but that was a misunderstanding. I actually implemented one of the hardest ways to win a Diplomatic Victory. But that is what I did. Now to get them to ally with me the most effective way to do this is with cash and lots of it, and of course even more cash to actually effect to Puppeting of them. So my overriding objective in this game is to amass a large Treasury. But of course I cannot ignore my military either, since a weak military invites attacks form greedy neighbors. And I may need to “liberate” the occasional city-state if another player conquers them. Austria also has a Unique Unit, the Hussar, which replaces the Cavalry unit. It can move after attacking, has a flanking bonus, and has one extra movement. And the Unique Building is the Coffee House, which increases the generation of Great People in the city by 25%. So you can expect me to build these in every city as well. With that in mind, I started the game and settled in place, I had Mountains nearby, but also Sheep and Silver within my city, so some useful resources. I immediately started to produce a Scout as my first unit, and sent my Warrior out to explore. In the very early stage I focus on exploring the surrounding area and finding any Goody Huts, i.e. Ruins. My initial city site is not exactly ideal, as it is all hilly with Jungle nearby. After building my two Scouts, my next priority was to build a Worker unit to increase the productivity of my city. And for my first social policy I unlocked Tradition. When I got my second policy I picked Oligarchy, and plan to complete all of the Tradition tree. I cleared out a Barbarian encampment, and then discovered my first City-state, Vilnius. Then I needed to clear out another Barbarian encampment that was blocking me from finishing my exploration. Meanwhile I am focusing on getting techs for sailing the ocean blue, because that is how I plan to get trade routes, discover City-states I can puppet, and so on. Because money is key to my strategy I made a beeline for Currency in the my Science research. And while headed there I completed the Tradition tree. When I can get there I will work on the Commerce tree to maximize my cash, but until then the Patronage tree will let me improve my City-state relations, which is important for improving my relationships with City-states. After all, you need to be allies with them for 5 turns before you can puppet them. I now have three cities, and can probably squeeze out a few more, which should be sufficient to my needs. At Turn 141 I have 6 cities, which is all I will build in this game. I now have the technology to build Workshops, which are the first productivity boosters available, so I set all my cities to building them. For Research my immediate object was to get to Compass so I could build the Galleass, which would let me do more ocean exploration. But to go into deep Ocean I will need to go further to get Astronomy, which will let me build the Caravel which can enter deep ocean. Once I got that I switched to Banking, not just for the money, but as a prerequisite to building the Forbidden Palace, which grants two additional delegates in the World Congress/United Nations. When I get Banking I'll go back to Astronomy, and then Navigation, to advance my seagoing capabilities. By Turn 216 I had gotten Banking and started on the Forbidden Palace. And by luck, just as I got started I got a Great Engineer. I am holding him in reserve in case I need to hurry production, since Great Engineers are the only way to do that in Civ 5. I also picked up Astronomy, which will let me build Caravels to explore the whole ocean. My next research priority will be Gunpowder since it is time to beef up my defenses. At Turn 240 my Caravels started to come out, and I found several new City-States. And since my Treasury is healthy (I started with 6,000 gold, and I'm bringing in 100 each turn), I began the process of puppeting the City-states. Puppeting City-states as Austria Let's look at this in detail since it is important. The requirements are 2 things: Be allied with the City-state for 5 consecutive turns Have the cash needed. This amount is not too much early on, but it rises over time. So, how do you become allies with a City-state? There are a number of things you can do to improve your relationship. You can take on a quest that a City-state has published, which can be things like “Find another Natural Wonder” or “Create a Great Admiral”. These quests pop up continuously throughout the game, and you are free to ignore them, but fulfilling one will improve your relationship. Trade will also improve your relationship, so in this game all of my Trade routes were made with City-states. If you are in a position to have a successful war, you can find a former City-state that was conquered by one of the other Empires, liberate it, and then it will be your ally for the rest of the game. Giving them presents is how I usually do it, though. You can give them units or money. If your purpose is to get allies, money works best. But I do gift units in 2 circumstances. First, if I have obsolete units, giving them away might be better then deleting them. Second, if a City-state is under attack by another Empire. Gifting them units might help them hold out and make life difficult for a rival. The place where all of this is done is the City-state screen which opens up when you click on the bar above the City-state, which is where you handle all of your relationships. At the top of this screen you see your current status, which more often than not will be Neutral, which is how all City-state relationships start out. But you can get them angry by, for instance, moving a unit of your into their territory. If you only do it once, and give them time to get over it, they will go back to Neutral. You can also improve the relationship by pledging to protect them, but be careful. If they get attacked and you do not try to protect them, they will get very angry. Giving a gift opens a pop-up to say what kind of gift: 250 gold, 500 gold, 1000 gold, or a Unit. Note that a Unit is only worth 5 influence points, while 250 Gold is worth 20, so as I said Money is more powerful if your aim is to improve your status with them. So at Turn 242 I found the City-state of Singapore, and it appears that I was the first Empire to find them. So I immediately pledged to protect them. My Influence with them was 20, which is Neutral. We just met, and that is where things stand on first meeting. The be Friends you need to get to 30, and to be allies you need to get to 60. When I clicked Next Turn, I got a Quest from Singapore. They were worried about a Barbarian Encampment nearby, and if I cleared it out I would get additional influence with them. In this case, though, I let that go by. I want to move more quickly, and Singapore is across the Ocean from me. My influence with them at this point was 21, so only a modest increase. But I have 6038 Gold in my Treasury, and I am bringing in 102 per turn. So let's see what a gift of 500 Gold will do. It brings me to 65 influence, so we are now Allies. But when I mouse-over Singapore, the pop-up window reminds me that my Influence will decrease by 1.12 per turn. For an Empire other than Austria this would mean a regular infusion of cash to keep up your status. And I have won Diplomatic victories with other Empires by saving up a lot of cash and dumping it on City-states just before the United Nations vote. But for Austria you have special ability called Diplomatic Marriage that lets you turn the City-state into a Puppet, and that is permanent. But it also presents some obstacles as we will see. We are allies now, but my influence will drop by 1 each turn, and I might lose my allyship before I can puppet them. But I can gift a unit and get another 5, and I happen to have a very obsolete Warrior unit that will serve the purpose. However, it takes three turns for the Unit to arrive, so I lose few more points. At Turn 247 I can now use the diplomatic Marriage option to make a puppet of Singapore. Prior to doing this I was fourth in score with 604, while the leader had 729. My Happiness Level was +25, and I now had 5833 Gold in my Treasury. Then I made Singapore a puppet, and now I am third in the game with a score of 664. My Treasury has fallen to 5258, which means it cost me 575 Gold. But most significant is that my Happiness fell from +25 to +9, which is a huge loss. I have enough gold to puppet 4-5 more City-states at this time but if I did I would have rebellions breaking out and my Empire would eventually collapse. This is the obstacle that Austria has to face. We need to promote Happiness before we go much further with making puppets. Civ is always a game of balances. Links https://civilization.fandom.com/wiki/City-state_(Civ5) https://civilization.fandom.com/wiki/Austrian_(Civ5) https://www.palain.com/gaming/civilization-v/playing-civilization-v-part-12/ Provide feedback on this episode.

This show has been flagged as Explicit by the host. It is suggested reviewing the episode What did I do at work today? Part 3 Section 1 prior to listening Test driven development - a way of writing code that involves writing an automated unit-level test case that fails, then writing just enough code to make the test pass, then refactoring both the test code and the production code, then repeating with another new test case. - https://en.wikipedia.org/wiki/Test-driven_development Joplin - Joplin is an open source, cross platform note-taking app. - https://joplinapp.org/ PHP - A popular general-purpose scripting language that is especially suited to web development. Fast, flexible and pragmatic, PHP powers everything from your blog to the most popular websites in the world. - https://www.php.net/ MySQL - MySQL is an open-source relational database management system. MariaDB is a community developed fork of MySQL, often installing the MySQL package on a Linux distribution will actually install MariaDB. - https://en.wikipedia.org/wiki/MySQL - https://mariadb.org/ - https://www.mysql.com/ Sublime Text - Cross platform text editor - https://www.sublimetext.com/ Nmap - Network Mapper is a free and open source utility for network discovery and security auditing - https://nmap.org/ Markdown Fenced code blocks - "A code fence is a sequence of at least three consecutive backtick characters (`) or tildes (~). (Tildes and backticks cannot be mixed.) A fenced code block begins with a code fence, preceded by up to three spaces of indentation. The line with the opening code fence may optionally contain some text following the code fence; this is trimmed of leading and trailing spaces or tabs and called the info string. ... Although this spec doesn't mandate any particular treatment of the info string, the first word is typically used to specify the language of the code block." ```ruby def foo(x) return 3 end ``` from CommonMark Spec at https://commonmark.org/ (CommonMark is a standard, interoperable and testable version of Markdown.) Writing to a Database with PHP The following PHP method is implemented within a database access class: function create_with_id($id, $name) { $born = time(); $id = mysqli_real_escape_string($this->db, $id); $name = mysqli_real_escape_string($this->db, $name); $sql = "INSERT INTO object (object_id, display_name_text, born, died) VALUES ($id, '$name', $born, 0); "; db_run_sql($this->db, $sql); } Note db_run_sql is a helper function defined elsewhere, not a built in function, and the property db is a previously initialized mysqli object. Provide feedback on this episode.

This show has been flagged as Clean by the host. 01 This is the fourth episode in a four part series on simple podcasting. 02 Introduction In this episode we will discuss alternatives to Audacity when it comes to analyzing audio spectrums to find the sources of unwanted noise. I previously promised some gratuitous hackery, and we will get into that in this episode. 03 Recall that with Audacity you first import the audio file, then select the part of the audio you wish to analyze (or ctrl-A for all), and then select analyze > plot spectrum. This is in fact the only feature of Audacity that I know how to use. I am definitely not an audio expert. I do however have some background in processing and analyzing other signals, so some of the basics are familiar to me. 04 We can accomplish the same thing that Audacity does in this instance provided we can do the following. First, we need to get the data out of the audio file and into a form which we can import into other software. Second, we need to perform certain mathematical operations on this data. Finally, we need to be able to plot the results of these calculations on a chart. -------------------- 05 Fourier Transforms First though, we need a bit of mathematical background. What Audacity is doing when it shows a plot of frequency versus amplitude is that it is showing the results of a Fourier Transform. A Fourier Transforms is a mathematical operation that converts the time domain into the frequency domain. Any complex signal, audio or otherwise, can be broken down into a collection of sine waves of various frequencies. For example, a simple square wave signal of say 100 hertz can be represented as a sine wave of frequency 100 hertz plus a collection of higher frequency sine waves which add together to give the sharp corners. 06 A Fourier Transform finds these sine waves and sorts them out into separate bins, with each bin representing an individual frequency or a collection of closely related frequencies, depending on how fine grained the sorting is. 07 This is exactly what we want when we are trying to figure out how to filter out noise. Recall that earlier in this series we had to solve a problem with a high pitched background noise which was originating in my cheap microphone. Analyzing this audio by frequency showed that it was a series of individual tones at 1 kHz intervals. We were then able to use filters targeted at those frequencies to get rid of that noise. 08 There are several optimized versions of the Fourier Transform algorithm. A very common one is the Fast Fourier Transform, common abbreviated to just "FFT". This is so common that the term "FFT" is often used to simply mean any Fourier Transform even though this is not technically correct. 09 Typical FFT algorithms require that the number of data samples is exactly a power of two. So the number of samples we need may be something like 4096, 8192, or 65536, to give a few random examples. When we transform from the time domain to the frequency domain, each sample becomes a single frequency "bin". So the more samples we have, the finer the resolution we get in terms of frequency. 10 If we assume we are dealing with flac files recorded at a 44.1 kHz sample rate, that is, 44100 samples per second, then if we have 32768 samples, each "bin" represents slightly more than 1 hertz. If we have 65536 samples, then each "bin" represents a fraction of a hertz. For our purposes we will pick 65536 samples. That means we need 1.48 seconds of data. For simplicity's sake we will record at least 2 seconds of data and then just discard the samples that we don't need. 11 There is a further complication here. Fourier Transforms normally work with complex numbers. Recall from your school days that as well as integers and real numbers there are complex numbers. Each complex number consists of two parts, a real component and an imaginary component. I won't go into the details of this, just accept that each sample needs to have two components. Fortunately, if we don't have complex number data we can just set the imaginary component to zero and use that. This is enough talking about the theory, let's get into the practical details. -------------------- 12 Extracting Data from Audio Files First we will look at how to extract the data from the audio files. Fortunately, one of the programs which we have already been using can do this. To do this we will use Sox. I am not aware of an equivalent feature in ffmpeg. 13 Sox calls itself "SoX - Sound eXchange, the Swiss Army knife of audio manipulation" Sox is free software and is licensed under the GPLV2 or later. In this case we want to use a feature which allows us to convert a binary audio signal file to a text data file. To convert the file to text data we just give the output file a ".dat" file extension and Sox will do this for us. 14 Here is a command example. sox inputfile.flac tdata.dat 15 This gives us a file in the following format, assuming this is a mono audio recording. ; Sample Rate 44100 ; Channels 1 0 0.045471191406 2.2675737e-05 0.055023193359 4.5351474e-05 0.048217773438 6.8027211e-05 0.053192138672 etc. The first line states the sample frequency The second line states that the data is for channel 1. The data starts on the third line. Column 1 is the time in seconds. Column 2 is the waveform data point. 16 To analyze the data we want a subset of these samples. When we convert from the time domain to the frequency domain, our resolution will be determined by the number of samples. We would like therefore to have at least as many samples as the sampling rate. We also want the samples size to be an even multiple of two. The number of points we want to have is equal to the next even multiple of two above our chosen sampling rate, 44,100 Hz. This number would be 65536. 17 To extract this data from the file we can do the following. tail tdata.dat -n+3 | head -n65536 | awk '{printf "%sn", $2}' > tdata.csv 18 We use tail to skip over the first three lines. We use head to take the next 65536 lines and discard the rest. We use awk to extract the second column which we will use as the real component. We now have this data as a csv file in one column. -------------------- 19 Analyzing the Data To analyze the data we need software which can calculate FFTs. I will now show two examples of this, a very simple case using Libre Office Calc, and a more complex but more complete one using GNU Octave. 20 Using Libre Office We can do fourier analysis and plot charts using Libre Office. Take the csv file of data that we previously created. For this example I used data from a recording of silence so that I could see what internal noise was being generated by the headset. Open the csv file and import it into Libre Office Calc. 21 Now select all 65536 rows of column A. The Fourier function will automatically fill the imaginary component with zeros if we don't provide an column of imaginary numbers, so we don't need to provide a column of zeros. Then select Data > Statistics > Fourier Analysis. 22 A window will open allowing you to select various parameters. For Results to:, enter "D1". Grouped by Columns. Select OK. 23 New data should now appear starting in cell D1. The first line will say " Fourier Transform" The second line will state the input range. The third line will state "Real" in column D, and "Imaginary" in column E. The data will start in row 4. 24 For our simple example we will ignore the imaginary data and just use the real data, which will form our Y component when we plot it on a chart. We now need to create the X axis data. 25 Each cell is a "bin" of frequencies. Each cell therefore represents (sample frequency) / (Number of samples) Hz. 26 To create the X axis data showing frequency, enter the following formula in to column C to the left of each D column number. =((44100/65536) * (ROW() - 4) 27 We can now create an XY chart showing the frequency analysis. You may need to exclude the first couple of dozen rows as very low frequency components which cannot be heard may otherwise overwhelm the data we are interested in. Also, you only need the first half of the chart. The FFT mirrors the data from the first half of the array into the second half. 28 Because characterizing a sine wave requires a minimum of 2 points, although we have a sample frequency of 44.1 kHz, we really only have sound waves up to a maximum of half that, or 22.05 kHz. Create the chart with lines only. If you followed the above instructions, you should see something resembling what we saw in Audacity, except with each bin more sharply defined. 29 In the data that I had from a recording of unfiltered headset noise, I could see a distinct noise spike every 1000 hertz. 30 However, we have taken several shortcuts. First, the imaginary component of the data was ignored. Second, the magnitude (that is, Y axis) has both positive and negative peaks. Third, the data is not scaled to dB sound units, so we just have a relative measure. However, that by itself is enough to tell us where the frequencies are that we need to construct filters to deal with. 31 We could refine this spreadsheet a bit more to deal with the above issues, but I think we have demonstrated the basic principle, and working with a spreadsheet can be a bit awkward. However, if working with a spreadsheet is what you want to do, then you can add more columns and more formulae to improve on it. -------------------- 32 Other Analysis Software I will go on to GNU Octave in a moment, but I want to get a few other alternatives out of the way first. I won't go into any detail on them other than to point them out to people who want to have a go at trying these themselves. 33 Grace There is math and plotting software called Grace. This is free software, released under the GPL V2. According to the documentation, it seems to have the features we need, including an FFT function. However, I could not get it to work properly on Ubuntu 24.04. I could not get it to load a data file and plot data. 34 The error messages were vague and unhelpful. The file navigation system didn't work. There was no obvious path to success, and if it isn't easy to use then there is no point to it. This is fairly old software, designed for X Window and Motif. I gave up on it as not suitable for this series as I am looking for some fairly low effort things for people to try themselves. If someone else can get it to work on their PC, perhaps they could do an HPR episode on this themselves. 35 Command Line FFT Packages There are several command line FFT packages. They will read data from std in or from a file and output the FFT. However, these are not packaged for Ubuntu and appear to be distributed as C source code which you would download and compile. You can experiment with those if you wish, but I felt they were a bit out of scope for discussion here as I am looking at common tools that are ready to use. 36 Here are two examples. One is Command-line Fast Fourier Transform utility https://github.com/gregfjohnson/fft Another is cli-fft https://github.com/jonolafur/cli-fft 37 I have not tried these and cannot say whether they are any good or not. Similarly, there are a number of FFT packages that are libraries for languages such as Python. If you want to take the time to write a short program to go with them, you can create a dedicated FFT command line program. However, I felt that this too was out of scope for what I was trying to do here. 38 Doing it the Hard Way Hypothetically, it may be possible to write an FFT function in bash bc, which is the arbitrary precision calculator language which is part of the standard shell package. I say hypothetically, because I have not tried it. I think it would be an interesting challenge, but I don't have the time at the moment to try it. If anyone feels motivated to give it a try, they're welcome to give it a go and then do a podcast episode on it. -------------------- 39 GNU Octave We have seen that as well as using features built into Audacity to analyze the audio spectrum to see the frequencies of undesired noises, we were able to do the same using a Libre Office spreadsheet. 40 Now we'll look at another bit of software, GNU Octave. GNU Octave is free software, licensed under the GPL V3 or later. It is a mathematical scripting language, very similar to Matlab. People use it for mathematical, engineering, and scientific work. It can be found in most Linux distros and is available for some other operating systems as well. 41 Octave has two features built in that we need for our purposes. It does FFTs, and it has a plotting system built in to produce graphs. -------------------- 42 We will take the same audio test file that we used with Audacity and Libre Office and use it here as well. The bash script to convert the flac file to text data is essentially the same, with the exception that file extension on the output file as is ".txt" instead of ".csv". This latter change was an arbitrary decision on my part. 43 As a quick review, this bash script uses sox to convert a flac file to a text ".dat" file. Then it uses tail, head, and awk to extract the first 65536 rows of data, skipping over the header information and ignoring the first column of time data. This script will be in the show notes. -------------------- #!/bin/bash # This version is for use with the GNU Octave script. sox hsnoisemono.flac hsnoisemono.dat tail hsnoisemono.dat -n+3 | head -n65536 | awk '{printf "%sn", $2}' > hsnoisemono.txt -------------------- 44 We now have a 1.1 MB file containing 65536 samples of data in text format. Now the next thing we need to do is to create a short Octave script file. I will just give a brief overview of the script here, the full script will be in the show notes. 45 I put the script in a file called "octavespectrum.m". I have never used Octave before now, but the convention seems to be to give the script a ".m" ending. The "she-bang" line is "#!/usr/bin/env octave". If you make the file executable you can run it like any other script, or you can type "octave" and then the name of the script to run. 46 I won't read out the script in detail, as that would be too hard to following along in a podcast. However, I pass several arguments to the script including the name of the data file, and then two integers that I use to limit the display area in the Y and X axes so I can have the chart focus on the areas of interest that I want to see. I also pass a string containing the name of the graphic file that I want the chart exported to. This was an arbitrary decision on my part and you can just hard code these values in if that is what you want to do. 47 The arguments are accessed by calling the "args()" function, which returns an array of strings. Next, it reads in the specified file using the "dlmread()" function. This reads all of the data into an array. 48 Next, it performs a hamming windowing function on the data. I'll explain that briefly. It is standard practice when doing FFT signal processing to "window" the signal. Since the signal sample is of finite length, it will stop at each end of the array. 49 Unless you were lucky enough for this to happen exactly at a zero crossing, this would produced an abrupt transition in the data which looks like "noise" to the FFT. The solution is to taper the signal off gradually towards the ends so that when it gets cut off the signal is fairly small at that point anyway. There are a variety of different windowing functions, but "hamming" seems to be the most commonly used. 50 Next, it does an FFT using the "fft()" function. 51 This gives us real and imaginary outputs. These are combined by summing the squares of each corresponding real and imaginary element and then taking the square root of each and storing that in a new array. This gives a single array of the same length as the originals, but combining the two output components. If anyone wants to tell me that this isn't how things are done in the audio world, they're welcome to make an HPR episode telling us all the right way to do things. 52 Then it does some scaling and selection of subsets of data so we get the X axis in hertz and just the number of samples that we wish to look at. If you are looking at the script, the thing to keep in mind is that Octave will work on entire arrays of data in a single operation. You don't need to write explicit loops for this. The looping is handled implicitly as part of the syntax. 53 It also does various other things that make the chart easier to read. The comments in the script describe these in more detail. Since this is a script it's easier to add these sorts of refinements than is the case for a spreadsheet so I have made the effort to add them. Finally it calls the "plot()" function. If an output graphics file name was provided, it also creates a PNG file containing the same image using the "saveas" function. 54 We now see the chart, and it looks more or less as expected. However, this chart is interactive. You can zoom and pan the data, something that you can't do with either Audacity or Libre Office. The chart window doesn't have a function for exporting the resulting chart to a "png" file, it will only save to an ".ofig" file. The ofig file is not a standard graphics file, it is a serialization of the chart data that can only be looked at using the Octave chart viewer. 55 Alternatively, you can just take a screenshot of the chart after you have interactively zoomed and panned to a point of interest. At the bottom left of the chart window is a pair of x-y coordinates which tell you the current position of the mouse pointer in chart units. This is very handy as it can be used to get the exact (or close to exact) frequency of each noise spike. 56 The Y axis is not scaled in any particular units such as dB, as I'm not sure how to do that according to audio industry conventions. On the other hand, I'm not sure that it's really necessary, as I don't know what dB means in tangible terms anyway. It does show relative sizes, so it helps to determine whether you have one noise frequency or multiple frequencies to worry about. 57 If anyone is familiar with how to scale the raw data from a flac file as exported by Sox into dB units according to audio industry convention, then they are welcome to create an HPR episode telling us how to do it. -------------------- 58 Comments on GNU Octave I had never used GNU Octave before this, although I had heard of it and it is quite a significant piece of software for a specific segment of users. 59 The syntax is a bit odd especially in how it deals with array operations, but I was able to google various examples and answers to eventually get this working. A few other peculiarities are that it uses the percent "%" character to denote a comment, and leaving out the semi-colon at the end of the line causes it to print the answer to the console after executing the statement. 60 The GNU Octave solution was harder to get working than the Libre Office method. However, once it was working it is easier to use repeatedly. If I were to want to automatically generate audio files with different filtering or other options and wanted to script the creation of a large number of images showing the results, this would be the way to do it. 61 When your run the Octave script you may get a warning which says something like "QSocketNotifier: Can only be used with threads started with QThread". This is apparently a routine warning message from the Qt graphics system which has no real significance in this context and can be ignored for our purposes. -------------------- 62 We now have a bash script which will use sox to extract the data from a flac file, and a GNU Octave script which can be used to display the resulting frequency spectrum. This does more or less the same thing as "Plot Spectrum" does in Audacity, but allows for zooming and panning to get a more detailed look at the data. 63 However it doesn't give you an absolute reading of the sound levels in dB, something that Audacity does provide. What I wanted it for though was to find the frequencies of the audible noise in the signal, something that it does quite well. -------------------- #!/usr/bin/env octave % Perform an FFT on the data in a file and plot the results. % ====================================================================== % The sampling frequency. This must be changed to accommodate the % actual sampling frequency if it was something else. samplefreq = 44100; % Thickness of line on plot. linewidth = 2; % ====================================================================== % The name of the data file is passed as a argument. args = argv(); if length(args) < 3 quit endif % File name. fname = args{1}; % Clip the peak values. peakclip = str2double(args{2}); % How much data to show, in kHz. rbound = str2double(args{3}) * 1000; % The optional file name to save a chart image to. if length(args) > 3 chartfile = args{4}; else chartfile = ""; endif % ====================================================================== % Read the data in from the file. sampledata = dlmread(fname); % Number of samples. samplecount = length(sampledata); % ====================================================================== % Window the data. This helps deal with the discontinuity of data at % each end of the array and the effects this has on introducing apparent % noise into the signal. windoweddata = (hamming(samplecount) .* sampledata); % ====================================================================== % Do the actual FFT. fftresults = fft(windoweddata); % Get real component. r = real(fftresults); % Get the imaginary component. i = imag(fftresults); % Combine the real and imaginary. In order to square each element of each % array, we must use the ".^" operator, not just "^". rfft = sqrt(r.^2 + i.^2); realfft = rfft(1:samplecount); % ====================================================================== % Scale factor for frequency. fscale = samplefreq / samplecount; % X axis scale, scaled to frequency. f = (0:samplefreq/2) * fscale; % Take a subset of the data if specified. rbound has to be re-scaled % from kHz to array increments. freq = f(1:min(rbound / fscale,length(f))); % y axis. We take the absolute value and then limit (clip) the peaks % so that a few large peaks don't obscure the smaller ones. mag = min(abs(realfft(1: length(freq))), peakclip); % Plot the results. figure; whandle = plot(freq, mag, 'LineWidth', linewidth); title(["Audio Spectrum of ", fname]); xlabel("Frequency (Hz)"); ylabel("Unscaled Magnitude"); grid on; % If the appropriate optional argument was specified, save the chart % to a file of that name. if length(chartfile) > 4 saveas(gcf, chartfile, "png"); endif % Need this so the plot window stays open. waitfor(whandle); % ====================================================================== -------------------- This is the shell script used with the above Octave script. The arguments are 1 - the file name for the input data file. 2 - The value to clip the peaks at. 3 - The upper frequency bound in kHz. 4 - The output graphics file name. #!/bin/bash octave octavespectrum.m hsnoisemono.txt 10 12 hsnoisemono.png -------------------- 64 Episode Conclusion In this episode we covered the following topics. What Fourier transforms are. Extracting data from audio files using Sox. Analyzing the data using Libre Office. Analyzing the data using GNU Octave. And, several alternative analysis methods. 65 Series Conclusion This is the end of a four part series on simple podcasting. In the first episode, we covered a simple podcast recording method. This first episode is all you really need to make a podcast. 66 In the second episode we covered basic filtering and a few other simple topics. The methods discussed in that episode provide basic improvements to your audio if you feel the need for it. 67 In the third episode we covered how to analyze audio noise problems using Audacity and additional filtering techniques to deal with specific problems that we may find. We also covered command line recording, playback, and getting information about an audio recording. 68 In the fourth episode we engaged in a bit of gratuitous hackery for the fun of it and showed how to use alternative software methods to analyze audio signals. 69 I hope that this series has been both useful and entertaining and that you will use the knowledge gained here to create and submit your own HPR podcast episodes. -------------------- -------------------- Provide feedback on this episode.

This show has been flagged as Clean by the host. This series is dedicated to exploring little-known—and occasionally useful—trinkets lurking in the dusty corners of UNIX-like operating systems. In UNIX Curio #4 ( HPR episode 4617 ), I teased the subject of file compression. Today I'm circling back to that. The history of data compression goes back at least to the 1970s, and in contexts outside UNIX and computers, probably even earlier. Somehow, it is refreshing to learn that humans have always struggled to have enough storage space to keep all the data they want to hang on to. One way around this limitation is to use some form of compression. I am only going to dive into lossless compression for this episode—that is, a compression method that can be reversed and will spit out the original data bit for bit. Lossy compression methods also have their places: you might be familiar with their use for audio (such as Ogg Vorbis or MP3); it's also used for images (such as JPEG). Lossy compression allows some of the original data to be thrown away, resulting in a smaller file than is possible with lossless compression, but the intent is for the result to still sound or look "good enough" to a human observer. Also, I am going to limit my discussion to generic methods used for many types of data; while FLAC does lossless compression, it is specifically designed just for audio. I should make clear that I have never studied computer science or information theory, so this episode will not get into the science behind various types of compression algorithms and how they differ. But in general, these methods take advantage of the fact that many types of data have recurring patterns. English text mostly consists of words that often re-appear many times—source code similarly has keywords and variable names that recur. Compression is accomplished by representing a piece of data that occurs multiple times with a symbol that is shorter in length. The first compression program in the UNIX world I could find is called pack , from 1978 1 . It was shortly followed in 1979 by a similar program called compact 2 . Both of these used a technique called Huffman coding, but with some differences between them. Files compressed with pack were given a .z extension and compact gave filenames a .C extension. Roughly every five or ten years after this, a new program would come along and achieve lasting popularity. There were, and still are, two opposing forces facing any new form of compression. Working in favor was the advantages it provided—first among these was achieving a better compression ratio, but performance improvements such as speed or reduced memory usage could also be compelling. The force against any new method was the fact that it was not yet widely supported—it doesn't much help to have a smaller file if the people you share it with cannot decompress it. The next major advance in compression arose out of three scientific papers: two in 1977 and 1978 by Abraham Lempel and Jacob Ziv (called LZ77 and LZ78), and one by Terry Welch in 1984 which built on LZ78. This last method is typically referred to as LZW. Our UNIX Curio for today is a program called compress 3 that implements the LZW method. Files compressed this way are named with the extension .Z . I had always assumed that this was to honor Jacob Ziv, but now that I've researched the history, it seems more likely to be a follow-on from how files compressed by pack were named. Since pack did not use any of the Lempel-Ziv methods, I would guess that it used .z because that wasn't already taken by anything else, but that's pure speculation. I do recall encountering .Z files in the wild, but feel certain that hasn't happened in the last 25 years, maybe longer. If you need to expand one of these, uncompress 4 is the program to use ( GNU's gunzip can also handle them 5 ). However, there was a serious problem that arose with the LZ78 and LZW compression methods. Both of them were patented, and the owner became aggressive in seeking payment from developers and users. The compress utility was developed within two months of the publication of Welch's 1984 paper and was included in Bell Laboratories' Eighth Edition UNIX before these shakedowns started. The paper did not disclose that a patent had been filed, and apparently Spencer Thomas and the other developers of compress were unaware of it. The utility became popular for a while, and was even standardized by POSIX, but people moved away from LZW once the legal threats started. Another important advance came in 1991 and was called the DEFLATE compression method. It combined the un-patented LZ77 method with Huffman coding to achieve a similar level of compression as LZW (actually, often better) without the legal trouble. DEFLATE was developed for PKZIP and was soon adopted by the GNU project's gzip compressor. While Phil Katz (the "PK" in PKZIP ) patented one way of implementing the DEFLATE method, it was possible to write a compressor and decompressor without infringing 6 ; also, he apparently never tried to enforce the patent 7 . As I mentioned in UNIX Curio #4, .zip is both an archive and a compression format. Each archive member can be compressed with one of several possible methods (or stored without compression). Unlike a tar file where compression can be applied to the entire archive, in .zip each archive member is compressed individually. This often means a .zip file will be slightly bigger than a tar file with the same contents compressed with gzip , because the .zip format cannot take advantage of duplication that occurs among more than one member of the archive. The vast majority of .zip files use only the DEFLATE and uncompressed storage methods and these are the only options if you want to follow the profile standardized in ISO/IEC 21320-1. Actually, since they both use DEFLATE, gzip is able to extract a .zip file in the special case where it only holds one member compressed with that method. From the 1990s onward, people paid significant attention to avoiding patent landmines, so only methods that didn't have that problem became broadly popular. While the patents on LZ78 and LZW have since expired, I feel like their most successful legacy was in discouraging people from using those methods, leading to DEFLATE taking the popularity crown. The next step came in 1996 and 1997 with the development of bzip and bzip2 by Julian Seward. The original method was quickly followed by bzip2 , which was the version that achieved true popularity. They use the Burrows-Wheeler transform, which does not itself compress data but re-arranges it to make it more compressible; this is combined with other techniques 8 . (At least, that's my understanding. I told you, I'm not up on information theory.) This provides a significant reduction in the compressed size of the data compared to earlier methods—however, it is slower than DEFLATE both during compression and decompression. Separate projects have developed parallel versions of gzip and bzip2 that can take advantage of multi-processor machines, but the original utilities run single-threaded. Another five years later, in 2001, Igor Pavlov added the Lempel-Ziv-Markov chain algorithm (LZMA), an enhancement to LZ77, to his 7-Zip compression tool. This was followed a few years later by LZMA2, a container format that allowed for LZMA compression to be split between multiple threads. Broad LZMA2 support came to the UNIX world in 2009 with the xz utility 9 . It offers roughly similar compression ratios to bzip2 , though it can be better or worse depending on the data to be compressed. While compression generally takes even longer than bzip2 , decompression is significantly faster (though still not as fast as gzip ). The Linux kernel relatively quickly supported booting from xz-compressed images 10 because it was a good match for that use case—compression, the time-consuming activity, only has to be done once while the more frequent decompression during boot happens relatively fast. The last method I will cover is Zstandard 11 , often written as zstd . This came about in 2015, and is another variation on LZ77 that uses finite-state entropy (which means nothing to me, but you might understand it). It performs about as well as DEFLATE in terms of compression ratios, but is much faster both when compressing and decompressing data. I should say that these statements are true with the typical default settings—depending on the compression level selected, it can compress more slowly, but compress the data smaller. However, decompression is always speedier than DEFLATE. This makes it attractive for some uses, and it is heavily promoted by Meta/Facebook, where Yann Collet developed it. For example, shipping large amounts of actively-used data between machines in a data center can go more quickly when the size is reduced; however, if the compression and decompression steps take too long that benefit is lost. A speedy method can be valuable even if it doesn't result in the greatest reduction in size. This use case stands in contrast to, say, a compressed backup file which might only be accessed in a disaster recovery scenario or never accessed at all, making size more important than speed. Both the xz and zstd utilities have some built-in support for multi-threading, but the default is to run in a single thread. While xz can use multiple threads for decompression (but only if the file was compressed in multi-thread mode), the reference zstd utility can only use more than one thread for compression, not decompression. There are many other methods of lossless compression that have been developed over the decades, but I believe these are the ones you are most likely to encounter in the world of UNIX-like systems. This is a personal opinion, and others might choose a different set. As mentioned, it can be tough for a new method to gain popularity and 35-year-old DEFLATE is still probably the most commonly used despite not being the fastest or offering the greatest reduction in size. Even systems like FreeBSD, NetBSD, and OpenBSD that do not like to include GNU tools supported it by developing their own version of gzip based on the permissively-licensed zlib library. Technically, the LZW method used by the compress utility is still standardized by POSIX, so one might expect it to have the widest support. However, aggressive patent enforcement discouraged adoption, especially by Free and Open Source Software systems—even though the patent has expired, it is still out of favor compared to DEFLATE. For this reason, I feel justified in calling it a curio. References: Eighth Edition UNIX pack.c https://www.tuhs.org/cgi-bin/utree.pl?file=V8/usr/src/cmd/pack/pack.c 2.9BSD compact.c https://www.tuhs.org/cgi-bin/utree.pl?file=2.9BSD/usr/src/ucb/compact/compact.c Compress specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/compress.html Uncompress specification https://pubs.opengroup.org/onlinepubs/009695399/utilities/uncompress.html GNU Gzip manual https://www.gnu.org/software/gzip/manual/gzip.html RFC 1951: DEFLATE Compressed Data Format Specification version 1.3 https://tools.ietf.org/html/rfc1951 History of Lossless Data Compression Algorithms: The Rise of Deflate https://ethw.org/History_of_Lossless_Data_Compression_Algorithms#The_Rise_of_Deflate bzip2 https://en.wikipedia.org/wiki/Bzip2 XZ Utils https://en.wikipedia.org/wiki/XZ_Utils 2.6.38 merge window part 2 https://lwn.net/Articles/423541/ zstd https://en.wikipedia.org/wiki/Zstd Appendix The table below demonstrates the results of compressing different types of data using tools described in this episode. While not totally rigorous, I did run each compression and decompression multiple times to ensure I was getting consistent results. The laptop I used has an Intel Core i5-6200U CPU running at 2.30GHz, and the system had at least 5 GB of free memory for each run. While this processor has two cores and can run four simultaneous threads, all utilities were run single-threaded. The term "best" means the highest level of compression available (the exact level used is shown). For bzip2 , the default is the best. For zstd , "best" is -19, which is the highest "normal" level, but "ultra" levels that are even higher also exist. Ratios are the percentage of the original size that the file was reduced to (other sources might instead express the compression ratio as the reduction in size achieved). In all results, smaller numbers are better. ┌────────────────────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┐ │ │ gzip │ gzip │ bzip2 │ xz │ xz │ zstd │ zstd │ │ │(default -6) │ (best -9) │ (-9) │(default -6) │ (best -9) │(default -3) │ (best -19) │ ├──────────────┬─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Size (ratio) │ 22,036,508 │ 21,891,623 │ 15,795,698 │ 13,487,768 │ 12,938,464 │ 20,454,657 │ 13,709,078 │ │ │ │ (24%) │ (24%) │ (17%) │ (15%) │ (14%) │ (23%) │ (15%) │ │English Text ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │(90,532,092 │Compression │ 4.8s │ 7.6s │ 8.5s │ 49.8s │ 58.8s │ 0.6s │ 65.2s │ │bytes │time │ │ │ │ │ │ │ │ │uncompressed) ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Decompression│ 0.7s │ 0.8s │ 3.7s │ 1.2s │ 1.2s │ 0.4s │ 0.4s │ │ │time │ │ │ │ │ │ │ │ ├──────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Size (ratio) │ 125,291,122 │ 124,189,544 │ 98,016,512 │ 84,882,492 │ 81,954,344 │ 120,604,855 │ 87,298,645 │ │ │ │ (21%) │ (21%) │ (17%) │ (14%) │ (14%) │ (20%) │ (15%) │ │Source Code ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │(590,008,320 │Compression │ 22.0s │ 39.3s │ 54.8s │ 241s │ 298s │ 3.7s │ 348s │ │bytes │time │ │ │ │ │ │ │ │ │uncompressed) ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Decompression│ 5.1s │ 5.1s │ 20.3s │ 8.1s │ 7.8s │ 2.4s │ 2.4s │ │ │time │ │ │ │ │ │ │ │ ├──────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Size (ratio) │ 32,830,905 │ 32,371,241 │ 26,856,579 │ 20,717,288 │ 20,352,880 │ 28,538,810 │ 23,154,582 │ │ │ │ (19%) │ (19%) │ (16%) │ (12%) │ (12%) │ (17%) │ (13%) │ │Binary Program├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │(171,972,264 │Compression │ 6.4s │ 22.4s │ 18.6s │ 62.2s │ 67.8s │ 0.8s │ 111s │ │bytes │time │ │ │ │ │ │ │ │ │uncompressed) ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Decompression│ 1.5s │ 1.5s │ 5.6s │ 2.3s │ 2.3s │ 0.7s │ 0.7s │ │ │time │ │ │ │ │ │ │ │ ├──────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Size (ratio) │ 146,397,772 │ 146,397,757 │ 144,485,451 │ 131,950,232 │ 130,926,780 │ 147,154,979 │ 145,703,840 │ │ │ │ (89%) │ (89%) │ (88%) │ (80%) │ (80%) │ (90%) │ (89%) │ │WAVE Audio ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │(164,396,302 │Compression │ 9.2s │ 9.2s │ 25.1s │ 70.4s │ 97.7s │ 0.7s │ 58.3s │ │bytes │time │ │ │ │ │ │ │ │ │uncompressed) ├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │Decompression│ 2.0s │ 2.0s │ 13.5s │ 12.2s │ 12.1s │ 0.6s │ 0.8s │ │ │time │ │ │ │ │ │ │ │ ├──────────────┴─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ │ │ gzip │ gzip │ bzip2 │ xz │ xz │ zstd │ zstd │ │ │(default -6) │ (best -9) │ (-9) │(default -6) │ (best -9) │(default -3) │ (best -19) │ └────────────────────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┘ English text consists of Titles 1 through 10 of the 2020 U.S. Code of Federal Regulations . Source code consists of a tar file containing the Linux kernel source, version 4.0. Binary program consists of an ELF-format executable of the pandoc application, version 2.17.1.1 found on Debian 12. Audio consists of a 24-bit Signed Integer PCM WAVE file with 2 channels at 44.1kHz, about 10:21 in length. For comparison, the audio-specific flac lossless compression utility reduced this file to 97,962,711 bytes (60%) in 2.6 seconds at the default (-5) level and to 97,714,876 bytes (59%) in 5.4 seconds at the highest (-8) level. Provide feedback on this episode.