Podcast appearances and mentions of ned batchelder

Topics covered in this episode: Solara UI Framework Coverage at a crossroads “Virtual” methods in Python classes Extras Joke Watch on YouTube About the show Sponsored by ScoutAPM: pythonbytes.fm/scout Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 10am PT. Older video versions available there too. Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it. Michael #1: Solara UI Framework via Florian A Pure Python, React-style Framework for Scaling Your Jupyter and Web Apps Solara lets you build web apps from pure Python using ipywidgets or a React-like API on top of ipywidgets. These apps work both inside the Jupyter Notebook and as standalone web apps with frameworks like FastAPI. See the Examples page. Based on Reacton By building on top of ipywidgets, Solara automatically leverage an existing ecosystem of widgets and run on many platforms, including JupyterLab, Jupyter Notebook, Voilà, Google Colab, DataBricks, JetBrains Datalore, and more. Brian #2: Coverage at a crossroads Ned Batchelder is working on making coverage.py faster. Includes a nice, quick explanation of roughly how coverage.py works with trace function and arcs used for branch coverage. And how trace slows things down for lines we know are already covered. There are cool ideas from SlipCover that could be applicable. There's also sys.monitoring from Python 3.12 that helps with line coverage, since you can disable it for lines you already have info on. It doesn't quite complete the picture for branch coverage, though. Summary: jump in and help if you can read it anyway for a great mental model of how coverage.py works. Michael #3: “Virtual” methods in Python classes via Brian Skinn PEP 698 just got accepted, defining an @override decorator for type hinting, to help avoid errors in subclasses that override methods. Only affects type checkers but allows you to declare a “link” between the base method and derived class method with the intent of overriding it using OOP. If there is a mismatch, it's an error. Python 3.12's documentation Makes Python a bit more like C# and other more formal languages Brian #4: Parsing Python ASTs 20x Faster with Rust Evan Doyle Tach is “a CLI tool that lets you define and enforce import boundaries between Python modules in your project.” we covered it in episode 384 When used to analyze Sentry's ~3k Python file codebase, it took about 10 seconds. Profiling analysis using py-spy and speedscope pointed to a function that spends about 2/3 of the time parsing the AST, and about 1/3 traversing it. That portion was then rewritten in Rust, resulting in 10x speedup, ending in about 1 second. This is a cool example of not just throwing Rust at a speed problem right away, but doing the profiling homework first, and focusing the Rust rewrite on the bottleneck. Extras Brian: I brought up pkgutil.resolve_name() last week on episode 388 Brett Cannon says don't use that, it's deprecated Thanks astroboy for letting me know Will we get CalVer for Python? it was talked about at the language summit There's also pep 2026, in draft, with a nice nod in the number of when it might happen. 3.13 already in the works for 2024 3.14 slated for 2025, and we gotta have a pi release So the earliest is then 2026, with maybe a 3.26 version ? Saying thanks to open source maintainers Great write-up by Brett Cannon about how to show your appreciation for OSS maintainers. Be nice Be an advocate Produce your own open source Say thanks Fiscal support On topic Thanks Brett for pyproject.toml. I love it. Michael: The Shiny for Python course is out! Plus, it's free so come and get it. Joke: Tao of Programming: Book 1: Into the Silent Void, Part 1

Topics covered in this episode: pycountry Does Python have pointers? ingestr Make your terminal nice Extras Joke Watch on YouTube About the show Sponsored by ScoutAPM: pythonbytes.fm/scout Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: pycountry A Python library to access ISO country, subdivision, language, currency and script definitions and their translations. pycountry provides the ISO databases for the standards: 639-3 Languages 3166 Codes for representation of names of countries and their subdivisions 3166-1 Countries 3166-3 Deleted countries 3166-2 Subdivisions of countries 4217 Currencies 15924 Scripts Brian #2: Does Python have pointers? Ned Batchelder Turns out, this is really the description of “what's a variable in Python?” that helps to make sense of the “variables as names” model in Python, especially for people coming from languages that use pointers a lot. You can use id() to find out what a variable points to You just can't do the reverse of access it given an id. There's no “dereference” operator. See also Python Names and Values, also by Ned Should be required reading/viewing for all Python curriculum. Michael #3: ingestr ingestr is a command-line application that allows ingesting or copying data from any source into any destination database. Works on both MongoDB and Postgres and many more. incremental loading: append, merge or delete+insert Brian #4: Make your terminal nice David Lord David's switched to Fish and Starship I tried switching to Fish several times, and I guess I'm good with zsh. Although I admire the brave comic sans motto: “Finally, a command line shell for the 90s” But I'm finally ready for Starship, and it takes almost no time to set up Plus it's fast. (Has it always been Rust?) Extras Brian: Doing some groundwork for a SaaS project, using SaaS Pegasus I just talked with Cory from Pegasus for an upcoming PythonTest episode I haven't decided whether to save up SaaS episodes for one big series, or spread them out. But mostly I'm excited to get my project started. Michael: Excellent video about “cloud exit” uv - The Next Evolution in Python Packages? Python 3.13 a5 Target's Open Source Fund via Pat Decker Joke: Anti-social engineer

Topics covered in this episode: AppleCrate One way to package Python code right now Flask8 but why? Extra, Extra, Extra Extras Joke Watch on YouTube About the show Sponsored by ScoutAPM: pythonbytes.fm/scout Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: AppleCrate By Rhet Turnbull (from Building macOS Apps episode) AppleCrate is a tool for creating native macOS installers for your command line tools. It's useful for creating installers for command line tools written in any language. Tools written in interpreted languages like Python will need to be first processed with a tool like pyinstaller to create a standalone executable. AppleCrate uses Jinja2 templates to generate the files required for the installer. This allows you to use template variables in your files or command line parameters to customize the installer. Brian #2: One way to package Python code right now Ned Batchelder An example repo with all the parts for package A lot of discussion and what to think about in the README (unfortunately rst and not md, but we can't have everything) Includes pyproject.toml dev-requirements.txt README.rst Makefile LICENSE.txt .bitignore .editorconfig see https://editorconfig.org Shout out to to Packaging Python Projects on python.org, which is pretty good Michael #3: Flask8 but why? Ihor Kalnytskyi: Something I really like about #ruff, a new tool for both linting and formatting in the #python ecosystem. You can literally pick any lint rule it supports and see both reasoning and examples. Ruff supports over 700 lint rules, many of which are inspired by popular tools like Flake8, isort, pyupgrade, and others. Brian #4: Extra, Extra, Extra Flat.app kinda like trello, etc. but a very simple interface that makes it pretty easy to use tosdr.org Terms of Service; Didn't Read Kind of a wikipeda way to summarize the terms of service of different web services, and give them ratings/grades Why I write I talked about blogging more last episode. Here's a cool write-up by Sheena O'Connell reasons to remember to refine my thinking to impact to get through hard times to connect Three pytest Features You Will Love Helen Scott at JetBrains/PyCharm Fixtures, Markers, Parametrize Plus shoutouts to my course and book Extras Brian: Wikipedia List of common misconceptions - just for fun Ear Trumpet Labs (a Potland Company) Edwina mic - just something on my wish list Michael Mozilla Monitor Python 3.12.2 Upgraded all the Python apps (11 of them) in about 2 minutes and one command Got a Vision Pro? Try the Talk Python Courses app Great video event: Data Doodling with Martina Pugliese Joke: Free Tier

Susan Senator is an author, blogger and journalist living in the Boston area with her husband Ned Batchelder. She has three sons, the oldest of whom is 33 and has severe autism. Ms. Senator is the author of Making Peace With Autism as well as The Autism Mom's Survival Guide and Autism Adulthood: Insights and Creative Strategies for a Fulfilling Life. A journalist since 1997, she has a blog in Psychology Today, and she has published many pieces on disability, parenting, and living happily, in places like the New York Times, the Washington Post, the Boston Globe, and NPR. Senator has appeared as a guest on the Today Show, MSNBC, ABC News, PBS, NPR and CNN. Her writings on Special Olympics took her to the White House in 2006, to a state dinner for Eunice Kennedy Shriver. Ms. Senator's blog, publications, and events can be found on www.susansenator.com Person Centered Planning http://personcenteredplanning.com/index.php/team/cheryl-ryan-chan/ Susan Senator https://susansenator.com/blog/ Autism Adulthood: Strategies and Insights for a Fulfilling Life by Susan Senator https://www.amazon.com/Autism-Adulthood-Strategies-Insights-Fulfilling-ebook/dp/B01CN2N6RG?ref=d6k_applink_bb_dls&dplnkId=a2ba04a3-8a49-4c40-a894-b8c3f5946d26 _______________________________________________________ Become a JOWMA Member! www.jowma.org  Follow us on Instagram! www.instagram.com/JOWMA_org  Follow us on Twitter! www.twitter.com/JOWMA_med  Follow us on Facebook! https://www.facebook.com/JOWMAorg/ Stay up-to-date with JOWMA news! Sign up for the JOWMA newsletter! https://jowma.us6.list-manage.com/subscribe?u=9b4e9beb287874f9dc7f80289&id=ea3ef44644&mc_cid=dfb442d2a7&mc_eid=e9eee6e41e

Topics covered in this episode: A Python/Django Advent calendar Dropbase helps you build internal web apps with Python Real-world match/case Extra, extra, extra, so many extras! Extras Joke Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training The Complete pytest Course Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: A Python/Django Advent calendar James Bennett's take on an Advent Calendar “I'm going to try to publish one short blog post each day of Advent 2023, each covering a small but hopefully useful tip or bit of information for Python and/or Django developers” First post also discusses using enums A couple cool testing posts Don't mock Python's HTTPX I didn't know HTTPX had built in transport mocking, very cool Test your documentation doctest discussion Michael #2: Dropbase helps you build internal web apps with Python Build fullstack web apps for your internal teams. Import existing Python scripts Quickly layer UIs and granular permissions on top. Turn any SQL SELECT into an admin panel with Smart Tables. Watch the video for the zen of it. Freemium model Brian #3: Real-world match/case Ned Batchelder Structural pattern matching example taken from a GitHub bot Matching nested dictionaries, pulling out bits of data The examples of not just matching but using case [structure] if [test on component] are neat. Michael #4: Extra, extra, extra, so many extras! WAY better DNS with Bunny.net DNS Terminal Secrets essay Meet the Supporting Developer in Residence (via Pycoders) Songs in Python code BohemianRhapsody.py MoneyForNothing.py PyCascades 2024 Project names blocked on PyPI to avoid name collision for downstream free-threaded Python distributions An Open Letter to the Python Software Foundation PSF's official mission https://discuss.python.org/t/python-3-12-1-now-available/40603 https://discuss.python.org/t/python-3-11-7-is-available/40778 Obfuscated Python winning (via Johannes Lippmann) Extras Brian: Python for VSCode, Dec 2023 release, rolls out better test discovery to everyone. Forcing pip to use virtualenv Advent of Code Joke: Too many open tabs

In this episode, we dive into The Myth of the Myth of Learning Styles by Ned Batchelder. This is a short but thought provoking article on how to find the right way to learn for you. We go through a series of questions you can ask yourself to find out how to best learn. We're still shaking off the rust after a few months away but we know you listen for the flubs as much as the content. Head over to runtimerundown.com and let us know what you thought of the show! Music by ⁠Hina⁠ and ⁠Kevin MacLeod⁠

Topics covered in this episode: Django 5.0 beta 1 released git bash, terminals, and Windows Mastering Integration Testing with FastAPI Reuven Learner has been banned for trading in rare animals (Pythons and Pandas) Extras Joke Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training The Complete pytest Course Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: Django 5.0 beta 1 released Django 5.0 release notes supports Python 3.10, 3.11, 3.12 Facet filters in the admin Simplified templates for form field rendering Database-computed default values Database generated model field More options for declaring field choices More Django news Djangonaut Space now accepting applications for our next contributor mentorship cohort Take the Django Developers Survey 2023 Michael #2: git bash, terminals, and Windows See the screenshot Requires Windows 10 Install the Windows Terminal from the Microsoft Store Brian #3: Mastering Integration Testing with FastAPI Alex Jacobs Some great integration testing techniques Focused on FastAPI, but relevant to many frameworks. Mocking authentication Mocking external APIs Fun use of parametrize and indirect fixtures for mocking responses. Mocking MongoDB Mocking AWS S3 Michael #4: Reuven Learner has been banned for trading in rare animals (Pythons and Pandas) via Pat Decker Reuven, like us, teaches Python and Data Sci Tried to advertise his courses (Python and Pandas courses) on Meta Got permanently (life-time) banned for selling rare and endangered animals. Sometimes I really hate these big tech companies My recent beefs have been with app store reviewers and surveillance-based capitalism Extras Brian: Where did everyone go? - Ned Batchelder I do feel like we're more fragmented than before, but I am feeling like we have a community on Mastodon. reminder that Mastodon has text search now On Sunday, I released Ch9, Coverage, as part of The Complete pytest course, specifically part of pytest Working with Projects. It was super fun. I've used coverage a lot since writing the book, for example, I demonstrate branch coverage. It's so much more effective to teach in video than in printed screenshots. Michael: Autin shell enhancer by Ellie Huxtable recommended by recommended by Nik JupyterCon 2023 videos are out More shells follow up from Teemu Hukkanen for “editor like” features Zsh and Bash ruff format and strings, aka format.quote-style = "single" Glyph's programming your computer talk is up. Joke: this is what the experts do

ElectricSQL is a project that offers a local-first sync layer for web and mobile apps, Ned Batchelder writes about the myth of the myth of “learning styles”, Carl Johnson thinks XML is better than YAML, Berkan Sasmaz defines and describes “idempotency” & HyperDX is an open source alternative Datadog or New Relic.

ElectricSQL is a project that offers a local-first sync layer for web and mobile apps, Ned Batchelder writes about the myth of the myth of “learning styles”, Carl Johnson thinks XML is better than YAML, Berkan Sasmaz defines and describes “idempotency” & HyperDX is an open source alternative Datadog or New Relic.

ElectricSQL is a project that offers a local-first sync layer for web and mobile apps, Ned Batchelder writes about the myth of the myth of “learning styles”, Carl Johnson thinks XML is better than YAML, Berkan Sasmaz defines and describes “idempotency” & HyperDX is an open source alternative Datadog or New Relic.

Hynek joins the show to discuss towncrier. At the top of the towncrier documentation, it says "towncrier is a utility to produce useful, summarized news files (also known as changelogs) for your project." Towncrier is used by "Twisted, pytest, pip, BuildBot, and attrs, among others." This is the last of 3 episodes focused on keeping a CHANGELOG. Episode 200 (https://testandcode.com/200) kicked off the series with keepachangelog.com and Olivier Lacan In 201 (https://testandcode.com/201) we had Ned Batchelder discussing scriv. Special Guest: Hynek Schlawack.

Last week we talked about the importance of keeping a changelog. This week we talk with Ned Batchelder about scriv, a tool to help maintain that changelog. Scriv "is a command-line tool for helping developers maintain useful changelogs. It manages a directory of changelog fragments. It aggregates them into entries in a CHANGELOG file." Special Guest: Ned Batchelder.

Watch on YouTube About the show Sponsored by Influxdb Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Brian #1: Pydantic V2 Pre Release Terrence Dorsey & Samuel Colvin Alpha release available to everyone: pip install --pre -U "pydantic>=2.0a1" Headlines: pydantic-core - all validation logic rewritten in Rust and moved to separate package, pytest-core 5-50x faster separation will aid safety and maintainability Lots ready for experimentation BaseModel, Dataclasses, Serialization, … Much still under construction Docs, BaseSettings→ pydantic-settings, … Michael #2: microdot The impossibly small web framework for Python and MicroPython Microdot is a minimalistic Python web framework inspired by Flask, and designed to run on systems with limited resources such as microcontrollers. It runs on standard Python and on MicroPython. Support for async, websockets, tls, even ASGI servers. Less mem usage by a big margin. Brian #3: GitHub Actions Tools: watchgha, build and inspect, and pytest annotate failures watchgha Ned Batchelder Watch GH Actions progress on the command line build-and-inspect-python-package Hynek Test the build of wheels, check contents, lint README print sdist contents, wheel contents, and metadata pytest-github-actions-annotate-failures utgwkk Nice traceback annotations for pytest Michael #4: PEP 709 – Inlined comprehensions by Carl Meyer Comprehensions are currently compiled as nested functions, which provides isolation of the comprehension's iteration variable, but is inefficient at runtime. This PEP proposes to inline list, dictionary, and set comprehensions into the code where they are defined, and provide the expected isolation by pushing/popping clashing locals on the stack. This change makes comprehensions much faster: up to 2x faster for a microbenchmark of a comprehension alone. Extras Michael: Python Web Apps that Fly with CDNs Course Joke: Can't watch movies

Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/stream/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Michael #1: Secure maintainer workflow by Ned Batchelder We are the magicians, but also the gatekeepers for our users Terminal sessions with implicit access to credentials first is unlikely: a bad guy gets onto my computer and uses the credentials to cause havoc second way is a more serious concern: I could unknowingly run evil or buggy code that uses my credentials in bad ways. Mitigations 1Password: where possible, I store credentials in 1Password, and use tooling to get them into environment variables. Side bar: Do not use lastpass, see end segment I can have the credentials in the environment for just long enough to use them. This works well for things like PyPI credentials, which are used rarely and could cause significant damage. Docker: To really isolate unknown code, I use a Docker container. Brian #2: Tools for parsing HTML and JSON Learned these from A Year of Writing about Web Scraping in Review Parsel - extract and remove data from HTML using XPath and CSS selectors jmespath - “James Path” - declaratively specify how to extract elements from a JSON document Michael #3: git-sizer Compute various size metrics for a Git repository, flagging those that might cause problems. Tip, partial clone: git clone --filter=blob:none URL # Stats for training.talkpython.fm # Full: git clone repo Receiving objects: 100% (118820/118820), 514.31 MiB | 28.83 MiB/s, done. Resolving deltas: 100% (71763/71763), done. Updating files: 100% (10792/10792), done. 1.01 GB on disk # Partial: git clone --filter=blob:none repo Receiving objects: 100% (10120/10120), 220.25 MiB | 24.92 MiB/s, done. Resolving deltas: 100% (1454/1454), done. Updating files: 100% (10792/10792), done. 694.4 MB on disk Partial clone is a performance optimization that “allows Git to function without having a complete copy of the repository. The goal of this work is to allow Git better handle extremely large repositories.” When changing branches, Git may download more missing files. Not the same as shallow clones or sparse checkouts Consider shallow clones for CI/CD/deployment Sparse checkouts for a slice of a monorepo Brian #4: Dataclasses without type annotations Probably file this under “don't try this at home”. Or maybe “try this at home, but not at work”. Or just “that Brian fella is a bad influence”. What! It's not me. It's Adrian, the dude that wrote the article. Unless you're using a type checker, for dataclasses, “… use any type you want. If you're not using a static type checker, no one is going to care what type you use.” @dataclass class Literally: anything: ("can go", "in here") as_long_as: lambda: "it can be evaluated" # Now, I've noticed a tendency for this program to get rather silly. hell: with_("from __future__ import annotations") it_s: not even.evaluated it: just.has(to=be) * syntactically[valid] # Right! Stop that! It's SILLY! Extras Michael: LastPass story just keeps getting worse We will see problems in supply chains because of this too A whole 2 hour discussion diving into what I touched on: twit.tv/shows/security-now Got your new mac mini yet? Or MacBook Pro? Joke: Developer/maker, what's my purpose?

If you maintain projects on places like GitHub, you know that having a classy readme is important and that maintaining a change log can be helpful for you and consumers of the project. It can also be a pain. That's why I'm excited to welcome back Ned Batchelder to the show. He has a lot of tools to help here as well as some opinions we're looking forward to hearing. We cover his tools and a bunch of others he and I found along the way. Links from the show Ned on Mastodon: @nedbat@hachyderm.io Ned's website: nedbatchelder.com Readme as a Service: readme.so hatch-fancy-pypi-readme: github.com Shields.io badges: shields.io All Contributors: allcontributors.org Keep a changelog: keepachangelog.com Scriv: Changelog management tool: github.com changelog_manager: github.com executablebooks' github activity: github.com dinghy: A GitHub activity digest tool: github.com cpython's blurb: github.com release drafter: github.com Towncrier: github.com mktestdocs testing code samples in readmes: github.com shed: github.com blacken-docs: github.com Cog: github.com Awesome tools for readme: github.com coverage.py: coverage.readthedocs.io Tailwind CSS "Landing page": tailwindcss.com Poetry "Landing page": python-poetry.org Textual: textualize.io Rich: github.com Join Mastodon Page: joinmastodon.org Watch this episode on YouTube: youtube.com Episode transcripts: talkpython.fm --- Stay in touch with us --- Subscribe to us on YouTube: youtube.com Follow Talk Python on Mastodon: talkpython Follow Michael on Mastodon: mkennedy Sponsors Local Maximum Podcast Sentry Error Monitoring, Code TALKPYTHON AssemblyAI Talk Python Training

Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Michael #0: Scorpions concert Check out this short video if you're interested. Michael #1: Pyscript 2022.09.1 is out Beware of the breaking changes Brian #2: Decorator shortcuts by Ned Batchelder Michael #3: Panel (of Holoviz) on Pyscript via Marc Skov Madsen Panel can now run on pyodide and pyscript with no backend server. Brian #4: auto-walrus Marco Gorelli Also, a reminder about pyupgrade, mentioned on the show back in 2018 Especially, look at the --py38-plus through --py39-plus flags. There are flags for 3.10 and 3.11, but limited testing for them. Joke: I need a new shirt

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Michael #1: auto-py-to-exe Converts .py to .exe using a simple graphical interface A good candidate to install via pipx For me, just point it at the top level app.py file and click go Can add icons, etc. Got a .app version and CLI version (I think

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Brian #1:distinctipy “distinctipy is a lightweight python package providing functions to generate colours that are visually distinct from one another.” Small, focused tool, but really cool. Say you need to plot a dynamic number of lines. Why not let distinctipy pick colors for you that will be distinct? Also can display the color swatches. Some example palettes here: https://github.com/alan-turing-institute/distinctipy/tree/main/examples from distinctipy import distinctipy # number of colours to generate N = 36 # generate N visually distinct colours colors = distinctipy.get_colors(N) # display the colours distinctipy.color_swatch(colors) Michael #2: Soda SQL Soda SQL is a free, open-source command-line tool. It utilizes user-defined input to prepare SQL queries that run tests on dataset in a data source to find invalid, missing, or unexpected data. Looks good for data pipelines and other CI/CD work! Daniel #3: Python in Nature There's a review article from Sept 2020 on array programming with NumPy in the research journal Nature. For reference, in grad school we had a fancy paper on quantum entanglement that got rejected from Nature Communications, a sub-journal to Nature. Nature is hard to get into. List of authors includes Travis Oliphant who started NumPy. Covers NumPy as the foundation, building up to specialized libraries like QuTiP for quantum computing. If you search “Python” on their site, many papers come up. Interesting to see their take on publishing software work. Brian #4: Supercharging GitHub Actions with Job Summaries From a tweet by Simon Willison and an article: GH Actions job summaries Also, Ned Batchelder is using it for Coverage reports “You can now output and group custom Markdown content on the Actions run summary page.” “Custom Markdown content can be used for a variety of creative purposes, such as: Aggregating and displaying test results Generating reports Custom output independent of logs” Coverage.py example: - name: "Create summary" run: | echo '### Total coverage: ${{ env.total }}%' >> $GITHUB_STEP_SUMMARY echo '[${{ env.url }}](${{ env.url }})' >> $GITHUB_STEP_SUMMARY Michael #5:Language Summit is write up out via Itamar, by Alex Waygood Python without the GIL: A talk by Sam Gross Reaching a per-interpreter GIL: A talk by Eric Snow The "Faster CPython" project: 3.12 and beyond: A talk by Mark Shannon WebAssembly: Python in the browser and beyond: A talk by Christian Heimes F-strings in the grammar: A talk by Pablo Galindo Salgado Cinder Async Optimisations: A talk by Itamar Ostricher The issue and PR backlog: A talk by Irit Katriel The path forward for immortal objects: A talk by Eddie Elizondo and Eric Snow Lightning talks, featuring short presentations by Carl Meyer, Thomas Wouters, Kevin Modzelewski, Samuel Colvin and Larry Hastings Daniel #6:AllSpice is Git for EEs Software engineers have Git/SVN/Mercurial/etc None of the other engineering disciplines (mechanical, electrical, optical, etc), have it nearly as good. Altium has their Vault and “365,” but there's nothing with a Git-like UX. Supports version history, diffs, all the things you expect. Even self-hosting and a Gov Cloud version. “Bring your workflow to the 21st century, finally.” Extras Brian: Will McGugan talks about Rich, Textual, and Textualize on Test & Code 188 Also 3 other episodes since last week. (I have a backlog I'm working through.) Michael: Power On-Xbox Documentary | Full Movie The 4 Reasons To Branch with Git - Illustrated Examples with Python A Python spotting - via Jason Pecor 2022 StackOverflow Developer Survey is live, via Brian TextSniper macOS App PandasTutor on webassembly Daniel: I know Adafruit's a household name, shout-out to Sparkfun, Seeed Studio, OpenMV, and other companies in the field. Joke: A little awkward

Watch the live stream: Watch on YouTube About the show Sponsor: Brought to you by FusionAuth - check them out at pythonbytes.fm/fusionauth Special guest: Calvin Hendryx-Parker Brian #1: Why your mock still doesn't work Ned Batchelder Some back story: Why your mock doesn't work a quick tour of Python name assignment The short version of Python Names and Values talk importing difference between from foo import bar and import foo w.r.t mocking punchline: “Mock it where it's used” Now, Why your mock still doesn't work talks about using @patch decorator (also applies to @patch.object decorator) and utilizing mock_thing1, mock_thing2 parameters to test you can change the return value or an attribute or whatever. normal mock stuff. But…. the punchline…. be careful about the order of patches. It needs to be @patch("foo.thing2") @patch("foo.thing1") def test_(mock_thing1, mock_thing2): ... Further reading: https://docs.python.org/3/library/unittest.mock.html#patch https://docs.python.org/3/library/unittest.mock.html#patch-object Michael #2: pls via Chris May Are you a developer who uses the terminal? (likely!) ls/l are not super helpful. There are replacements and alternatives But if you're a dev, you might want the most relevant info for you, so enter pls See images in Michael's tweets [1, 2]. You must install nerdfonts and set your terminal's font to them Calvin #3: Kitty Cross platform GPU accelerated terminal (written in Python Extended with Kittens written in Python Super fast rendering Has a rich set of plugins available for things like searching the buffer with fzf Brian #4: Futures and easy parallelisation Jaime Buelta Code example for quick scripts to perform slow tasks in parallel. Uses concurrent.futures and ThreadPoolExecutor. Starts with a small toy example, then goes on to a requests example to grab a bunch of pages in parallel. The call to executor.submit() sets up the job. This is done in a list comprehension, generating a list of futures. The call to futures.result() on each future within the list is where the blocking happens. Since we want to wait for all results, it's ok to block on the first, second, etc. Nice small snippet for easy parallel work. Example: from concurrent.futures import ThreadPoolExecutor import time import requests from urllib.parse import urljoin NUM_WORKERS = 2 executor = ThreadPoolExecutor(NUM_WORKERS) def retrieve(root_url, path): url = urljoin(root_url, path) print(f'{time.time()} Retrieving {url}') result = requests.get(url) return result arguments = [('https://google.com/', 'search'), ('https://www.facebook.com/', 'login'), ('https://nyt.com/', 'international')] futures_array = [executor.submit(retrieve, *arg) for arg in arguments] result = [future.result() for future in futures_array] print(result) Michael #5: pgMustard So you have a crappy web app that is slow but don't know why. Is it an N+1 problem with an ORM? Is it a lack of indexes? If you're using postgres, check out pgMustard: A simple yet powerful tool to help you speed up queries This is a paid product but might be worthwhile if you live deeply in postgres. Calvin #6: bpytop Great way to see what is going on in your system/server Shows nice graphs in the terminal for system performance such as CPU and Network traffic Support themes and is fast and easy to install with pipx Michael uses Glances which is fun too. Calvin used to be a heavy Glances user until he saw the light

Developers hate wasting effort on manual processes when we can write code to do it instead. Cog is a tool to manage the work of automating the creation of text inside another file by executing arbitrary Python code. In this episode Ned Batchelder shares the story of why he created Cog in the first place, some of the interesting ways that he uses it in his daily work, and the unique challenges of maintaining a project with a small audience and a well defined scope.

Watch the live stream: Watch on YouTube About the show Sponsored by Shortcut - Get started at shortcut.com/pythonbytes Special guest: Chris Patti Brian #1: Using cog to update --help in a Markdown README file Simon Willison I've wanted to have a use case for Ned Batchelder's cog Cog is a utility that looks for specially blocks [[[cog some code ]]] and [[[end]]] These block can be in comments, [HTML_REMOVED] for markdown. When you run cog on a file, it runs the “some code” and puts the output after the middle ]]] and before the [[[end]]]. Simon has come up with an excellent use, running --help and capturing the output for a README.md file for a CLI project. He even wrote a test, pytest of course, to check if the README.md needs updated. Michael #2: An oral history of Bank Python Bank Python implementations are effectively proprietary forks of the entire Python ecosystem which are in use at many (but not all) of the biggest investment banks. The first thing to know about Minerva is that it is built on a global database of Python objects. Barbara is a simple key value store with a hierarchical key space. It's brutally simple: made just from pickle and zip. Applications also commonly store their internal state in Barbara - writing dataclasses straight in and out with only very simple locking and transactions (if any). There is no filesystem available to Minerva scripts and the little bits of data that scripts pick up has to be put into Barbara. Barbara also has some "overlay" features: # connect to multiple rings: keys are 'overlaid' in order of # the provided ring names db = barbara.open("middleoffice;ficc;default") # get /Etc/Something from the 'middleoffice' ring if it exists there, # otherwise try 'ficc' and finally the default ring some_obj = db["/Etc/Something"] Lots of info about modeling with classes (instruments, books, etc) If you understand excel you will be starting to recognize similarities. In Excel, spreadsheets cells are also updated based on their dependencies, also as a directed acyclic graph. Dagger allows people to put their Excel-style modelling calculations into Python, write tests for them, control their versioning without having to mess around with files like CDS-OF-CDS EURO DESK 20180103 Final (final) (2).xlsx. Dagger is a key technology to get financial models out of Excel, into a programming language and under tests and version control. Time to drop a bit of a bombshell: the source code is in Barbara too, not on disk. Remain composed. It's kept in a special Barbara ring called sourcecode. Interesting table structures, like Pandas, but closer to a DB (MnTable) Over time the divergence between Bank Python and Open Source Python grows. Technology churns on both sides, much faster outside than in of course, but they do not get closer. Minerva has its own IDE - no other IDEs work if you keep your source files in a giant global database. What I can't understand is why it contains its own web framework. Investment banks have a one-way approach to open source software: (some of) it can come in, but none of it can go out BTW, I “read” this with naturalreaders app Chris #3: Pyxel Pyxel is a ‘retro gaming console' written in Python! This might seem old and un-shiny, but the restrictions imposed by the environment gift simplicity Vastly decreased learning time and effort compared to something like Unity or even Pygame Straight forward simple commands, just like it was for micro-computers in the 80s cls(), line(), rect(), circ() etc. Pyxel is somewhat more Python and less console than others like PICO-8 or TIC-80 but this is a feature! Use your regular development environment to build. Brian #4: How to Ditch Codecov for Python Projects Hynek Schlawack Codecov is a third party service that checks your coverage output and fails a build if coverage dropped. It's not without issues. Hynek is using coverage.py --fail-under flag in place of this in GitHub actions. It's not as simple as just adding a flag if you are using --parallel to combine coverage for multiple test runs into one report. Hynek is utilizing the coverage output as an artifact for each test, then pulling them all together in a coverage stage combine and check coverage. He provides the snippet of GH Action, and even links to a working workflow file using this process. Nice! Michael #5: tiptop (like glances) via Zach Villers tiptop is a command-line system monitoring tool in the spirit of top. It displays various interesting system stats, graphs it, and works on all operating systems. Really nice visualization for your servers Good candidate for pipx install tiptop Chris #6: pyc64 A Commodore 64 emulator written in pure Python! Not 100% complete - screen drawing is PETSCII character mode only This still allows for a lot of interesting apps & exploration Actual machine emulation using py65 - a pure Python 6502 chip emulator! You can pop to a Python REPL from inside the emulator and examine data structures like memory, registers, etc! An incredible example of what Python is capable of 0.6 Mhz with CPython and over 2Mhz with pypy! Extras Michael: Michael's FlaskCon 2021 HTMX Talk Chris: Amazon OpsTech IT is hiring! (If deemed appropriate :) Joke: I hate how the screens get bright so early this time of year

ZENKEI AI ポッドキャスト、シーズン１６は２０２１年４月２８日に開催した ZOOMライブの模様です。(YouTube) この回はいちきの一人しゃべり、テーマは「ゴールデンウィークも AI」です。（ZOOM に Matt さん、ホンダさん、chan-mori さん、大島さん、米田さんも来てくれました。ありがとうございます。） エピソード２０はメインの話題その２、『コンピュータ会話教室』第２回の（６）で、python 流の for loop の活用として pytorch での DataLoader の使い方です。 当日の市來の発表資料 youtube: Loop like a native (Ned Batchelder)

ZENKEI AI ポッドキャスト、シーズン１６は２０２１年４月２８日に開催した ZOOMライブの模様です。(YouTube) この回はいちきの一人しゃべり、テーマは「ゴールデンウィークも AI」です。（ZOOM に Matt さん、ホンダさん、chan-mori さん、大島さん、米田さんも来てくれました。ありがとうございます。） エピソード１９はメインの話題その２、『コンピュータ会話教室』第２回の（５）で、Ned さんのビデオから感じたポイントの（自分なりの）まとめです。 当日の市來の発表資料 youtube: Loop like a native (Ned Batchelder)

ZENKEI AI ポッドキャスト、シーズン１６は２０２１年４月２８日に開催した ZOOMライブの模様です。(YouTube) この回はいちきの一人しゃべり、テーマは「ゴールデンウィークも AI」です。（ZOOM に Matt さん、ホンダさん、chan-mori さん、大島さん、米田さんも来てくれました。ありがとうございます。） エピソード１８はメインの話題その２、『コンピュータ会話教室』第２回の（４）で、ぼくが Ned さんのビデオで一番感動したポイント、多重ループからの break についての話です。 当日の市來の発表資料 youtube: Loop like a native (Ned Batchelder)

ZENKEI AI ポッドキャスト、シーズン１６は２０２１年４月２８日に開催した ZOOMライブの模様です。(YouTube) この回はいちきの一人しゃべり、テーマは「ゴールデンウィークも AI」です。（ZOOM に Matt さん、ホンダさん、chan-mori さん、大島さん、米田さんも来てくれました。ありがとうございます。） エピソード１７はメインの話題その２、『コンピュータ会話教室』第２回の（３）で、python 流に for loop を書く話です。 当日の市來の発表資料 youtube: Loop like a native (Ned Batchelder)

ZENKEI AI ポッドキャスト、シーズン１６は２０２１年４月２８日に開催した ZOOMライブの模様です。(YouTube) この回はいちきの一人しゃべり、テーマは「ゴールデンウィークも AI」です。（ZOOM に Matt さん、ホンダさん、chan-mori さん、大島さん、米田さんも来てくれました。ありがとうございます。） エピソード１６はメインの話題その２、『コンピュータ会話教室』第２回の（２）で、python の for loop では index は敵だ、という話です。 当日の市來の発表資料 youtube: Loop like a native (Ned Batchelder)

ZENKEI AI ポッドキャスト、シーズン１６は２０２１年４月２８日に開催した ZOOMライブの模様です。(YouTube) この回はいちきの一人しゃべり、テーマは「ゴールデンウィークも AI」です。（ZOOM に Matt さん、ホンダさん、chan-mori さん、大島さん、米田さんも来てくれました。ありがとうございます。） エピソード１５はメインの話題その２、『コンピュータ会話教室』第２回の（１）で、前回紹介したNet Batchelder のビデオを解説しようと頑張ります。 当日の市來の発表資料 youtube: Loop like a native (Ned Batchelder)

Watch the live stream: Watch on YouTube About the show Sponsored by Shortcut Special guest: Morleh So-kargbo Michael #1: Django 4.0 beta 1 released Django 4.0 beta 1 is now available. Django 4.0 has an abundance of new features The new *expressions positional argument of UniqueConstraint() enables creating functional unique constraints on expressions and database functions. The new scrypt password hasher is more secure and recommended over PBKDF2. The new django.core.cache.backends.redis.RedisCache cache backend provides built-in support for caching with Redis. To enhance customization of Forms, Formsets, and ErrorList they are now rendered using the template engine. Brian #2: py - The Python launcher py has been bundled with Python for Windows only since Python 3.3, as py.exe See Python Launcher for Windows I've mostly ignored it since I use Python on Windows, MacOS, and Linux and don't want to have different workflows on different systems. But now Brett Cannon has developed python-launcher which brings py to MacOS and various other Unix-y systems or any OS which supports Rust. Now py is everywhere I need it to be, and I've switched my workflow to use it. Usage py : Run the latest Python version on your system py -3 : Run the latest Python 3 version py -3.9 : Run latest 3.9 version py -2.7 : Even run 2.x versions py --``list : list all versions (with py-launcher, it also lists paths) py --``list-paths : py.exe only - list all versions with path Why is this cool? - I never have to care where Python is installed or where it is in my search path. - I can always run any version of Python installed without setting up symbolic links. - Same workflow works on Windows, MacOS, and Linux Old workfow Make sure latest Python is found first in search path, then call python3 -m venv venv For a specific version, make sure python3.8, for example, or python38 or something is in my Path. If not, create it somewhere. New workflow. py -m venv venv - Create a virtual environment with the latest Python installed. After activation, everything happens in the virtual env. Create a specific venv to test something on an older version: py -3.8 venv venv --``prompt '``3.8``' Or even just run a script with an old version py -3.8 script_name.py Of course, you can run it with the latest version also py script_name.py Note: if you use py within a virtual environment, the default version is the one from the virtual env, not the latest. Morleh #3: Transformers As General-Purpose Architecture The Attention Is All You Need paper first proposed Transformers in June 2017. The Hugging Face (

Ned's personal site@nedbat on Twitter, Discord, and IRCInteractive Walks post coverage.py django-coverage-pluginSpeed Up Your Django Tests by Adam Johnson bookedx-platform CogSupport the ShowThis podcast does not have any ads or sponsors. To support the show, please consider visiting LearnDjango.com, Button, or Django News.

Somewhere in the midst of an infinitely recursing café, Ned Batchelder joins Bojan and Jason to talk about the wiles of testing and code coverage, and how to answer questions online without scaring off the new kids.

Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Michael #1: Awkward arrays via Simon Thor Awkward Array is a library for nested, variable-sized data, including arbitrary-length lists, records, mixed types, and missing data, using NumPy-like idioms. This makes it better than numpy at handling data where e.g. the rows in a 2D array have different lengths. It can even be used together with numba to jit-compile the code to make it even faster. Arrays are dynamically typed, but operations on them are compiled and fast. Their behavior coincides with NumPy when array dimensions are regular and generalizes when they’re not. Recently rewritten in C++ for the 1.0 release and can even be used from C++ as well as Python. Careful on installation: pip install awkward1 ← Notice the 1. Brian #2: Ordered dict surprises Ned Batchelder “Since Python 3.6, regular dictionaries retain their insertion order: when you iterate over a dict, you get the items in the same order they were added to the dict. Before 3.6, dicts were unordered: the iteration order was seemingly random.” The surprises: You can’t get the first item, like d[0], since that’s just the value matching key 0, if key 0 exists. (I’m not actually surprised by this.) equality and order (this I am surprised by) Python 3.6+ dicts ignores order when testing for equality {"a": 1, "b": 2} == {"b": 2, "a": 1} OrderdDicts care about order when testing for equality OrderedDict([("a", 1), ("b", 2)]) != OrderedDict([("b", 2), ("a", 1)]) Michael #3: jupyter lab autocomplete and more via Anders Källmar Examples show Python code, but most features also work in R, bash, typescript, and many other languages. Hover: Hover over any piece of code; if an underline appears, you can press Ctrl to get a tooltip with function/class signature, module documentation or any other piece of information that the language server provides Diagnostics: Critical errors have red underline, warnings are orange, etc. Hover over the underlined code to see a more detailed message Jump to Definition: Use the context menu entries to jump to definitions Highlight References: Place your cursor on a variable, function, etc and all the usages will be highlighted Automatic Completion: Certain characters, for example '.' (dot) in Python, will automatically trigger completion Automatic Signature Suggestions: Function signatures will automatically be displayed Rename: Rename variables, functions and more, in both: notebooks and the file editor. Brian #4: Open Source Tools & Data for Music Source Separation An online “book” powered by Jupyter Book By Ethan Manilow, Prem Seetharaman, and Justin Salamon A tutorial intended to guide people “through modern, open-source tooling and datasets for running, evaluating, researching, and deploying source separation approaches. We will pay special attention to musical source separation, though we will note where certain approaches are applicable to a wider array of source types.” Uses Python and interactive demos with visualizations. Section “basics of source separation” that includes a primer on digitizing audio signals, a look time frequency representations, what phase is, and some evaluations and measurements. Includes use of a library called nussl deep learning approaches datasets training deep networks Brian’s comments: Very cool this is an open source book Even if you don’t care about source separation, the primer on waveform digitization is amazing. The interactive features are great. Michael #5: Pass by Reference in Python: Background and Best Practices Does Python have pointers? Some languages handle function arguments as references to existing variables, which is known as pass by reference. Other languages handle them as independent values, an approach known as pass by value. Python uses pass by assignment, very similar to pass by ref. In languages that default to passing by value, you may find performance benefits from passing the variable by reference instead If you actually want to change the value, consider Returning multiple values with tuple unpacking A mutable data type Returning optional “value” types For example, how would we recreate this in Python? public static bool TryParse (string s, out int result); Tuple unpacking def tryparse(string, base=10): try: return True, int(string, base=base) except ValueError: return False, None success, result = tryparse("123") Optional types: def tryparse(string, base=10) -> Optional[int]: try: return int(string, base=base) except ValueError: return None if (n := tryparse("123")) is not None: print(n) Best Practice: Return and Reassign Brian #6: Visualizing Git Concepts by onlywei Wei Wang Git Basics is good, and important, but hard to get all these concepts to sink in well until you play with it. Visualizing Git Concepts with D3 solidifies the concepts Practice using git commands without any code, just visualizing the changes to the repository (and sometimes the remote origin repository) while typing commands. commit, branch, checkout, checkout -b reset, revert merge, rebase tag fetch, pull, push Incredibly powerful to be able to play around with these concepts without using any code or possibly mucking up your repo. Extras: Brian: micro:bit now has a speaker and a microphone - available in November Michael: Firefox containers Twitch! Joke: Q: Where do developers drink? A: The Foo bar - Knock Knock! - An async function - Who's there?

Sponsored by us! Support our work through: Our courses at Talk Python Training Python Testing with pytest Brian #1: How to be helpful online Ned Batchelder When answering questions. Lots of great advice. We’ll focus on just a few here. Answer the question first. There may be other problems with their code that they are not asking about that you want to point out. But keep that for after you’ve helped them and built up trust. No third rails. “It should be OK for someone to ask for help with a program using sockets, and not have to defend using sockets, especially if the specific question has nothing to do with sockets.” Same for pickle, threads, globals, singletons, etc. Don’t let your strong opinions derail the conversation. The goal is to help people. Strong reactions can make the asker feel attacked. No dog-piling. Meet their level. “Try to determine what they know, and give them a reasonable next step, not the ultimate solution. A suboptimal solution they understand is better than a gold standard they can’t make use of.” Say yes. Avoid absolutes. Step back. Take some blame. Use more words. “IRC and other online mediums encourage quick short responses, which are exactly the kinds of responses that will be easy to misinterpret. Try to use more words, especially encouraging optimistic words.” Understand your motivations. Humility. Make connections. Finally: It’s hard. All of Ned’s advice is great. Good meditations for when you read a question and your mouth drops open and your eyes stare in shock. Michael #2: blackcellmagic IPython magic command to format python code in cell using black. Has a great animated gif ;) Just do: %load_ext blackcellmagic Then in any cell %%black and magic! Accepts “arguments” like %%black -l 79 Tobin Jones has been kind enough to develop a NPM package over blackcellmagic to format all cells at once which can be found here. But it’s archived so no idea whether it’s current. Brian #3: Test smarter, not harder Luke Plant There’s lots of great advice in here, but I want to highlight two parts that are often overlooked. “Write your test code with the functions/methods/classes you wish existed, not the ones you’ve been given.” “If the API you want to use doesn’t exist yet, you still use it, and then make it exist.” This is huge. People tend to think like this while coding, but forget to do it while testing. Also. Your tests are often the first client for your API, so if the API in question is under your control and you need an easier API for testing, consider adding it to the real API. If it’s easier for testing, it may be easier for other clients of the API as well. “Only write necessary tests — specifically, tests whose estimated value is greater than their estimated cost. This is a hard judgement call, of course, but it does mean that at least some of the time you should be saying “it’s not worth it”.” Michael #4: US: The Greatest Package in the World by Jeremy Carbaugh A package for easily working with US and state metadata: all US states and territories postal abbreviations Associated Press style abbreviations FIPS codes capitals years of statehood time zones phonetic state name lookup is contiguous or continental URLs to shapefiles for state, census, congressional districts, counties, and census tracts The state lookup method allows matching by FIPS code, abbreviation, and name Even a CLI: $ states md Brian #5: Think Like A Coder Part of TED-Ed “… a 10-episode series that will challenge viewers with programming puzzles as the main characters— a girl and her robot companion— attempt to save a world that has been plunged into turmoil.” Although, I only count 9 episodes, I was 4 episodes in and hooked. Main cool thing, I think, is introducing terms and topics so they will be familiar when someone really does start coding: loops, for loops, until loops, while loops conditionals variables path logic permutations searches tables recursion Big O Also highly recommended for getting excited about coding: Girls Who Code: Learn to Code and Change the World TED-Ed has tons of other cool series on lots of subjects. CodeCombat Michael #6: Costs of running a Python web app for 55k monthly users How much does running a web app in production actually cost? KeepTheScore is an online software for scorekeeping. Create your own scoreboard for up to 150 players and start tracking points. It's mostly free and requires no user account. Keepthescore.co is a Python flask application running on DigitalOcean and Firebase. It currently has around 55k unique visitors per month, per day it’s around 3.4k. Servers and database on DigitalOcean: Costs per month: $95, the servers are oversized for the load they’re currently seeing. Amazon Web Services: Costs per month: $60, use a reporting tool called Metabase to generate insights and reports from the database Google Cloud, costs per month: $1.32, for Firebase DNS hosting, costs per month: $5 Disqus, costs per month: $10 Is it worth it? Is there revenue? In total that’s around $171 USD per month. If you’re running a company with employees that would be peanuts, but in this case the cost is being borne by a single indie-developer out of his own pocket. The bigger issue is that on the revenue side there’s a big fat zero. This is the reason why we are currently working on monetization. Some Talk Python stats: Maybe 40k monthly visitors, but oh, the podcast clients 3M requests / month just RSS, resulting in 320 GB / mo of XML traffic. We run on two prod servers: $10 & $5 as well as a dedicated MongoDB server @ $10. Total $25/mo. On the other hand, Talk Python Training's AWS bill last month was over $1,000 USD. You can hear a bunch about this on Talk Python 215. Joke: From twitter, originally from Netlify: "Oh no! We lost the hackers! Where did they go?" "I don't know! They just ransomware!” Number of days since I have encountered an array index error: -1.

Sponsored by us! Support our work through: Our courses at Talk Python Training Brian’s pytest book Brian #1: Improving Python exception chaining with raise-from Ram Rachum Python3 has a change called PEP 3134: Exception Chaining and Embedded Tracebacks It should be used more than it is. If an exception is raised from an except clause, it could be because: something unexpected happened “An exception was raised, and we decided to replace it with a different exception that will make more sense to whoever called this code. Maybe the new exception will make more sense because we’re giving a more helpful error message. Or maybe we’re using an exception class that’s more relevant to the problem domain, and whoever’s calling our code could wrap the call with an except clause that’s tailored for this failure mode.” If it’s the second case, you should change your code to something like this: try: [HTML_REMOVED] except ExpectedExceptionType as e: raise BetterException('Better explanation') from e It’s the from e that does the magic. And now instead of getting During handling of the above exception, another exception occurred: You get: The above exception was the direct cause of the following exception: “That’s how you know you have a case of a friendly wrapping of an exception.” Michael #2: Create and publish interactive reports in Python via Tim Pogue Datapane is an open source framework which makes it easy to turn scripts and notebooks into interactive reports. Free for individuals, paid(?) for teams Build reports in Python and deploy scripts and notebooks as self-service reporting tools. Analyze data in your own tools: Write code and analyze data in your own editor or environment, whether its Jupyter, Colab, or Airflow. Build reports in code: Datapane's framework makes it easy to create rich reports from DataFrames, Markdown, and visualization libraries, such as Altair. Publish and share: Export as standalone HTML files, or publish reports to Datapane.com for free, where they can be shared and embedded. Add forms to filter / drive the report Everything in Datapane is an API. Deploy scripts and generate reports from your server, GitHub, Airflow, or CI system. Check out the gallery. Brian #3: Pickle’s nine flaws Ned Batchelder Instead of “never use pickle”, Ned says “only use pickle if you are OK with it’s nine flaws” The flaws Insecure : Malicious pickles can get the unpickler to run bad code Old pickles look like old code : Any changes to your data structures might break your ability to read old pickles Implicit: All data is serialized as class objects, even if that’s not what you want. Over-serializes: Serializes everything in your objects, even things like cached computation __init__ isn’t called : during unpickling, even if it really should be for your situation Python only : for the most part, it’s not something you can use with other languages Unreadable: binary, so good luck debugging problems Appears to pickle code: but doesn’t really. Keeping a list of functions or classes? It’ll get pickled as names and get bound to a function/class matching the name during unpickling. Slow Some of it you can work around, but then, why? Alternatives: JSON, marshmallow, cattrs, protocol buffers, … Michael #4: PEP 602 -- Annual Release Cycle for Python by Łukasz Langa Status accepted This PEP proposes that Python 3.X.0 will be developed for around 17 months: The first five months overlap with Python 3.(X-1).0's beta and release candidate stages and are thus unversioned. The next seven months are spent on versioned alpha releases where both new features are incrementally added and bug fixes are included. The following three months are spent on four versioned beta releases where no new features can be added but bug fixes are still included. The final two months are spent on two release candidates (or more, if necessary) and conclude with the release of the final release of Python 3.X.0. Annual release cadence: Feature development of Python 3.(X+1).0 starts as soon as Python 3.X.0 Beta 1 is released. This creates a twelve month delta between major Python versions. This change provides the following advantages: makes releases smaller: since doubling the cadence doesn't double our available development resources, consecutive releases are going to be smaller in terms of features; puts features and bug fixes in hands of users sooner; creates a more gradual upgrade path for users, by decreasing the surface of change in any single release; creates a predictable calendar for releases where the final release is always in October (so after the annual core sprint), and the beta phase starts in late May (so after PyCon US sprints), which is especially important for core developers who need to plan to include Python involvement in their calendar; Brian #5: More git Resources: On episode 187 we talked about Oh Sh*t, Git!, a zine by Julia Evans I mentioned that I was concerned about buying it for a team due to the mild swearing. John Place reached out to tell us there’s a non-swearing version: Dangit, git!, the zine. Also both of these are inspired by two websites by Katie Sylor-Miller: dangitgit.com ohshitgit.com These are free websites with “something went wrong, how to I fix it” solutions. All issues have titles that are links/anchors, so you can send someone a link if they ask you a question of how to fix something with git, and hopefully they can figure it out themselves sometime. Also Git Cheatsheet Not just a pdf An interactive single page site that is, for one, beautifully designed. There’s 5 columns: Stash, Workspace, Index, Local Repo, Upstream Repo Hover over a column and it shows you git commands that affect that part and flows to other columns. Hover over a command and the description pops up at the bottom. The visual is great for reinforcing how actions move data between different parts of a repository, and a great way to teach people to have that mental model that git is not just your repo, it’s all of these components working together. Lastly, git-pretty Similar goals as the dangit and ohsh*t offerings, this is a single page png flowchart that starts with “so you have a mess on your hands” and asks a bunch of questions to funnel you to how to fix it. A fun thing to print out and pin to your wall. Michael #6: PEP 616 -- String methods to remove prefixes and suffixes Dennis Sweeney Status: Accepted Question: What does this return? “saturday is the 1st".strip('st') Answer: aturday is the 1 If you expected it to remove the string st, well, no. That’s PEP 616. Add two new methods, removeprefix() and removesuffix(), to the APIs of Python's various string objects. These methods would remove a prefix or suffix (respectively) from a string, if present, and would be added to Unicode str objects, binary bytes and bytearray objects, and collections.UserString. Extras: Michael: Manning conference Python Bytes event Michael's 10 tips for web dev PyCon recording out Learn Python Humble Bundle Telegram bots by Abhishek Pednekar Python Bytes https://t.me/TalkPythonBot Joke: Karen Chee (@karencheee): you: A famous engineer / inventor is coming over for dinner tonight, want to join us? me: Sure, who is it? you: His name is Rube Goldberg me: That name rings a bell, which sets off a trap that undoes a buckle and releases a ball that rolls down a pipe and …

Sponsored by Datadog: pythonbytes.fm/datadog Michael #1: PSF / JetBrains Survey via Jose Nario Let’s talk results: 84% of people who use Python do so as their primary language [unchanged] Other languages: JavaScript (down), Bash (down), HTML (down), C++ (down) Web vs Data Science languages: More C++ / Java / R / C# on Data Science side More SQL / JavaScript / HTML Why do you mainly use Python? 58% work and personal What do you use Python for? Average answers was 3.9 Data analysis [59% / 59% — now vs. last year] Web Development [51% / 55%] ML [40% / 39%] DevOps [39% / 43%] What do you use Python for the most? Web [28% / 29%] Data analysis [18% / 17%] Machine Learning [13% / 11%] Python 3 vs Python 2: 90% Python 3, 10% Python 2 Widest disparity of versions (pro 3) is in data science. Web Frameworks: Flask [48%] Django [44%] Data Science NumPy 63% Pandas 55% Matplotlib 46% Testing pytest 49% unittest 30% none 34% Cloud AWS 55% Google 33% DigitalOcean 22% Heroku 20% Azure 19% How do you run code in the cloud (in the production environment) Containers 47% VMs 46% PAAS 25% Editors PyCharm 33% VS Code 24% Vim 9% tool use version control 90% write tests 80% code linting 80% use type hints 65% code coverage 52% Brian #2: Hypermodern Python Claudio Jolowicz, @cjolowicz An opinionated and fun tour of Python development practices. Chapter 1: Setup Setup a project with pyenv and Poetry, src layout, virtual environments, dependency management, click for CLI, using requests for a REST API. Chapter 2: Testing Unit testing with pytest, using coverage.py, nox for automation, pytest-mock. Plus refactoring, handling exceptions, fakes, end-to-end testing opinions. Chapter 3: Linting Flake8, Black, import-order, bugbear, bandit, Safety. Plus more on managing dependencies, and using pre-commit for git hooks. Chapter 4: Typing mypy and pytype, adding annotations, data validation with Desert & Marshmallow, Typeguard, flake8-annotations, adding checks to test suite Chapter 5: Documentation docstrings, linting docstrings, docstrings in nox sessions and test suites, darglint, xdoctest, Sphinx, reStructuredText, and autodoc Chapter 6: CI/CD CI with GithHub Actions, reporting coverage with Codecov, uploading to PyPI, Release Drafter for release documentation, single-sourcing the package version, using TestPyPI, docs on RTD The series is worth it even for just the artwork. Lots of fun tools to try, lots to learn. Michael #3: Open AI Jukebox via Dan Bader Listen to the songs under “Curated samples.” A neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. Code is available on github. Dataset: To train this model, we crawled the web to curate a new dataset of 1.2 million songs (600,000 of which are in English), paired with the corresponding lyrics and metadata from LyricWiki. The top-level transformer is trained on the task of predicting compressed audio tokens. We can provide additional information, such as the artist and genre for each song. Two advantages: first, it reduces the entropy of the audio prediction, so the model is able to achieve better quality in any particular style; second, at generation time, we are able to steer the model to generate in a style of our choosing. Brian #4: The Curious Case of Python's Context Manager Redowan Delowar, @rednafi A quick tour of context managers that goes deeper than most introducitons. Writing custom context managers with __init__, __enter__, __exit__. Using the decorator contextlib.contextmanager Then it gets even more fun Context managers as decorators Nesting contexts within one with statement. Combining context managers into new ones Examples Context managers for SQLAlchemy sessions Context managers for exception handling Persistent parameters across http requests Michael #5: nbstripout via Clément Robert In the latest episode, you praised NBDev for having a git hook that strips out notebook outputs. strip output from Jupyter and IPython notebooks Opens a notebook, strips its output, and writes the outputless version to the original file. Useful mainly as a git filter or pre-commit hook for users who don’t want to track output in VCS. This does mostly the same thing as the Clear All Output command in the notebook UI. Has a nice youtube tutorial right in the pypi listing Just do nbstripout --``install in a git repo! Brian #6: Write ups for The 2020 Python Language Summit Guido talked about this in episode 179 But these write-ups are excellent and really interesting. Should All Strings Become f-strings?, Eric V. Smith Replacing CPython’s Parser with a PEG-based parser, Pablo Galindo, Lysandros Nikolaou, Guido van Rossum A Formal Specification for the (C)Python Virtual Machine, Mark Shannon HPy: a Future-Proof Way of Extending Python?, Antonio Cuni CPython Documentation: The Next 5 Years, Carol Willing, Ned Batchelder Lightning talks (pre-selected) What do you need from pip, PyPI, and packaging?, Sumana Harihareswara A Retrospective on My "Multi-Core Python" Project, Eric Snow The Path Forward for Typing, Guido van Rossum Property-Based Testing for Python Builtins and the Standard Library, Zac Hatfield-Dodds Core Workflow Updates, Mariatta Wijaya CPython on Mobile Platforms, Russell Keith-Magee Wanted to bring this up because Python is a living language and it’s important to pay attention and get involved, or at least pay attention to where Python might be going. Also, another way to get involved is to become a member of the PSF board of directors What’s a PSF board of directors member do? video There are some open seats, Nominations are open until May 31 Extras: Michael: Updated search engine for better result ranking Windel Bouwman wrote a nice little script for speedscope https://github.com/windelbouwman/pyspeedscope (follow up from Austin profiler) Jokes: “Due to social distancing, I wonder how many projects are migrating to UDP and away from TLS to avoid all the handshakes?” - From Sviatoslav Sydorenko “A chef and a vagrant walk into a bar. Within a few seconds, it was identical to the last bar they went to.” - From Benjamin Jones, crediting @lufcraft Understanding both of these jokes is left as an exercise for the reader.

This episode is sponsored by DigitalOcean: pythonbytes.fm/digitalocean Michael #1: pydantic via Colin Sullivan Data validation and settings management using python type annotations. (We covered Cerberus, this is similar) pydantic enforces type hints at runtime, and provides user friendly errors when data is invalid. class User(pydantic.BaseModel): id: int name = 'John Doe' signup_ts: datetime = None friends: List[int] = [] external_data = { 'id': '123', 'signup_ts': '2019-06-01 12:22', 'friends': [1, 2, '3'] } user = User(**external_data) id is of type int; the annotation-only declaration tells pydantic that this field is required. Strings, bytes or floats will be coerced to ints if possible; otherwise an exception will be raised. name is inferred as a string from the provided default; because it has a default, it is not required. signup_ts is a datetime field which is not required (and takes the value None if it's not supplied). Why use it? There's no new schema definition micro-language to learn. In benchmarks pydantic is faster than all other tested libraries. Use of recursive pydantic models, typing's standard types (e.g. List, Tuple, Dict etc.) and validators allow complex data schemas to be clearly and easily defined, validated, and parsed. As well as BaseModel, pydantic provides a [dataclass](https://pydantic-docs.helpmanual.io/usage/dataclasses/) decorator which creates (almost) vanilla python dataclasses with input data parsing and validation. Brian #2: Coverage.py 5.0 beta 1 adds context support Please try out the beta, even without trying contexts, as it helps Ned Batchelder to make sure it’s as backwards compatible as possible while still adding this super cool functionality. Coverage 5.0 beta 1 announcement The changes. Measurement contexts in depth. Trying out contexts with pytest and pytest-cov: (venv) $ pip install coverage==5.0b1 (venv) $ pip install pytest-cov (venv) $ pytest --cov=foo --cov-context=test test_foo.py (venv) $ coverage html --show-contexts (venv) $ open htmlcov/index.html results in coverage report that has little dropdowns on the right for lines that are covered, and what context they were covered. For the example above, with pytest-cov, it shows what test caused each line to be hit. Contexts can do way more than this. One example, split up different levels of tests, to see which lines are only hit by unit tests, indicating missing higher level tests, or the opposite. The stored db could also possibly be mined to see how much overlap there is between tests, and maybe help with higher level tools to predict the harm or benefit from removing some tests. I’m excited about the future, with contexts in place. Even if you ignore contexts, please go try out the beta ASAP to make sure your old use model still works. Michael #3: PSF is seeking developers for paid contract improving pip via Brian Rutledge The Python Software Foundation Packaging Working Group is receiving funding to work on the design, implementation, and rollout of pip's next-generation dependency resolver. This project aims to complete the design, implementation, and rollout of pip's next-generation dependency resolver. Lower the barriers to installing Python software, empowering users to get a version of a package that works. It will also lower the barriers to distributing Python software, empowering developers to make their work available in an easily reusable form. Because of the size of the project, funding has been allocated to secure two contractors, a senior developer and an intermediate developer, to work on development, testing and building test infrastructure, code review, bug triage, and assisting in the rollout of necessary features. Total pay: Stage 1: $116,375, Stage 2: $103,700 Brian #4: dovpanda - Directions OVer PANDAs Dean Langsam “Directions are hints and tips for using pandas in an analysis environment. dovpanda is an overlay for working with pandas in an analysis environment. "If you think your task is common enough, it probably is, and Pandas probably has a built-in solution. dovpanda is an overlay module that tries to understand what you are trying to do with your data, and help you find easier ways to write your code.” “The main usage of dovpanda is its hints mechanism, which is very easy and works out-of-the-box. Just import it after you import pandas, whether inside a notebook or in a console.” It’s like training wheels for pandas to help you get the most out of pandas and learn while you are doing your work. Very cool. Michael #5: removestar via PyCoders newsletter Tool to automatically replace 'import *' in Python files with explicit imports Report only mode and modify in place mode. Brian #6: pytest-quarantine : Save the list of failing tests, so that they can be automatically marked as expected failures on future test runs. Brian Rutlage Really nice email from Brian: >"Hi Brian! We've met a couple times at PyCon in Cleveland. Thanks for your podcasts, and your book. I've gone from being a complete pytest newbie, to helping my company adopt it, to writing a plugin. The plugin was something I developed at work, and they let me open-source it. I wanted to share it with you as a way of saying "thank you", and because you seem to be a bit of connoisseur of pytest plugins. ;)" Here it is: https://github.com/EnergySage/pytest-quarantine/” pytest has a cool feature called xfail, to allow you to mark tests you know fail. pytest-quarantine allows you to run your suite and generate a file of all failures, then use that to mark the xfails. Then you or your team can chip away at these failures until you get rid of them. But in the meantime, your suite can still be useful for finding new failures. And, the use of an external file to mark failures makes it so you don’t have to edit your test files to mark the tests that are xfail. Extras: MK: Our infrastructure is fully carbon neutral! Joke: A cop pulls Dr. Heisenberg over for speeding. The officer asks, "Do you know how fast you were going?" Heisenberg pauses for a moment, then answers, "No, but I know where I am.” [1] See Uncertainty principle, also called Heisenberg uncertainty principle or indeterminacy principle, statement, articulated (1927) by the German physicist Werner Heisenberg, that the position and the velocity of an object cannot both be measured exactly, at the same time, even in theory.

In this episode we’re shinning our maintainer spotlight on Ned Batchelder. Ned is one of the lucky ones out there that gets to double-dip — his day job is working on open source at edX, working on the Open edX community team. Ned is also a “single maintainer” of coverage.py - a tool for measuring code coverage of Python programs. This episode with Ned kicks off the first of many in our maintainer spotlight series where we dig deep into the life of an open source software maintainer. We’re producing this series in partnership with Tidelift. Huge thanks to Tidelift for making this series possible.

In this episode we’re shinning our maintainer spotlight on Ned Batchelder. Ned is one of the lucky ones out there that gets to double-dip — his day job is working on open source at edX, working on the Open edX community team. Ned is also a “single maintainer” of coverage.py - a tool for measuring code coverage of Python programs. This episode with Ned kicks off the first of many in our maintainer spotlight series where we dig deep into the life of an open source software maintainer. We’re producing this series in partnership with Tidelift. Huge thanks to Tidelift for making this series possible.

We write tests to make sure that our code is correct, but how do you make sure the tests are correct? This week Ned Batchelder explains how coverage.py fills that need, how he became the maintainer, and how it works under the hood.

In this episode I interview Ned Batchelder. I know that coverage.py is very important to a lot of people to understand how much of their code is being covered by their test suites. Since I'm far from an expert on coverage, I asked Ned to discuss it on the show. I'm also quite a fan of Ned's 2014 PyCon talk "Getting Started Testing", so I definitely asked him about that. We also discuss edX, Python user groups, PyCon talks, and more. Some of what's covered (pun intended) in this episode: coverage.py types of coverage Line coverage branch coverage Behavior coverage Data coverage How Ned became the owner of coverage.py Running tests from coverage.py vs running coverage from test runner. edX what is it what Ned's role is Ned's blog Ned's PyCon 2014 talk "Getting Started Testing" Teaching testing and the difficulty of the classes being part of unittest fixtures package some of the difficulties of teaching unittest because of it's class based system. the history of classes in unittest coming from java's jUnit implementation Boston's Python Group PyCon in Portland Ned to do a talk here "Machete mode debugging". Practicing PyCon talks at local group meetings. At the very least, practice it in front of a live audience. Links: Ned Batchelder (http://nedbatchelder.com/) Coverage (https://pypi.python.org/pypi/coverage) Coverage documentation (https://coverage.readthedocs.org) django-nose (https://pypi.python.org/pypi/django-nose) pytest-django (https://pypi.python.org/pypi/pytest-django) edX (https://www.edx.org/) open edX (https://open.edx.org/) Boston Python User Group (http://www.meetup.com/bostonpython/) Portland Python User Group (http://www.meetup.com/pdxpython/) - I need to go to these PyCon 2016 (https://us.pycon.org/2016/) - Planning on attending, it's in Portland. Yay! Getting Started Testing (http://nedbatchelder.com/text/test0.html) - Ned's 2014 Pycon talk

Episode 5 - Ned Batchelder