Podcasts about Matplotlib

Heute wird's bunt, spannend und ein bisschen investigativ! In dieser Folge von Female TechTalk dreht sich alles um Datenvisualisierungen. Wir zeigen euch, wie Diagramme, Infografiken & Heatmaps uns helfen, riesige Datenmengen verständlich zu machen und trotzdem mit Vorsicht zu genießen sind.Was hat es mit Scatterplots auf sich, und wie hilft uns die lineare Regression, aus Daten echte Zusammenhänge zu erkennen? Außerdem stellen wir euch einige der mächtigsten Tools vor – von Matplotlib & Seaborn bis hin zu Plotly & D3.js.Und natürlich darf die große Frage nicht fehlen: Korrelation oder Kausalität? Macht Schokolade essen wirklich schlauer? Oder gibt es vielleicht einen versteckten dritten Faktor?

Topics covered in this episode: NumFOCUS concerns leaping pytest debugger llm Extra, Extra, Extra, PyPI has completed its first security audit Extras Joke Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training The Complete pytest Course Patreon Supporters Connect with the hosts Michael: @mkennedy@fosstodon.org Brian: @brianokken@fosstodon.org Show: @pythonbytes@fosstodon.org Join us on YouTube at pythonbytes.fm/live to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too. Finally, if you want an artisanal, hand-crafted digest of every week of the show notes in email form? Add your name and email to our friends of the show list, we'll never share it. Brian #1: NumFOCUS concerns Suggested by Pamphile Roy Write up of the current challenges faced by NumFOCUS, by Paul Ivanov (one of the OG of Scientific Python: Jupyter, Matplotlib, etc.) Struggling to meet the needs of sponsored and affiliated projects. In February, NumFOCUS announced it is moving in a new direction. NumFOCUS initiated an effort to run an election for open board seats and proposed changing its governance structure. Some projects are considering and actively pursuing alternative venues for fiscal sponsorship. Quite a bit more detail and discussion in the article. NumFOCUS covers a lot of projects NumPy, Matplotlib, pandas, Jupyter, SciPy, Astropy, Bokeh, Dask, Conda, and so many more. Michael #2: leaping pytest debugger llm You can ask Leaping questions like: Why am I not hitting function x? Why was variable y set to this value? What was the value of variable x at this point? What changes can I make to this code to make this test pass? Brian #3: Extra, Extra, Extra, 2024 Developer Summit Also suggested by Pamphile, related to Scientific Python The Second Scientific Python Developer Summit , June 3-5, Seattle, WA Lots of great work came out of the First Summit in 2023 pytest-regex - Use regexs to specify tests to run Came out of the '23 summit I'm not sure if I'm super happy about this or a little afraid that I probably could use this. Still, cool that it's here. Cool short example of using __init__ and __call__ to hand-roll a decorator. ruff got faster Michael #4: PyPI has completed its first security audit Trail of Bits spent a total of 10 engineer-weeks of effort identifying issues, presenting those findings to the PyPI team, and assisting us as we remediated the findings. Scope: The audit was focused on "Warehouse", the open-source codebase that powers pypi.org As a result of the audit, Trail of Bits detailed 29 different advisories discovered across both codebases. When evaluating severity level of each advisory, 14 were categorized as "informational", 6 as "low", 8 as "medium" and zero as "high". Extras Brian: pytest course community to try out Podia Communities. Anyone have a podia community running strong now? If so, let me know through Mastodon: @brianokken@fosstodon.org Want to join the community when it's up and running? Same. Or join our our friends of the show list, and read our newsletter. I'll be sure to drop a note in there when it's ready. Michael: VS Code AMA @ Talk Python [video] Gunicorn CVE Talk submissions are now open for both remote and in-person talks at the 2024 PyConZA? The conference will be held on 3 and 4 October 2024 in Cape Town, South Africa. Details are on za.pycon.org. FlaskCon 2024 will be happening Friday, May 17 inside PyCon US 2024. Call for proposals are now live! Joke: Debugging with your eyes

Can you build a Space Invaders clone using Python's built-in turtle module? What advantages does the Seaborn data visualization library provide compared to Matplotlib? Christopher Trudeau is back on the show this week, along with special guest Real Python core team member Bartosz Zaczyński. We're sharing another batch of PyCoder's Weekly articles and projects.

Happy leap year day everyone, very excited to bring you a special once-in-a-4 year edition of ThursdAI

COMO ANALIZAR LOS DATOS DE UN PODCAST? En esta ocasión, te comparto algunas claves para realizar un análisis de datos básicos utilizando Python, Jupyter Lab y librerías como: Pandas, Matplotlib, Seaborn, etc.Por razones obvias no te voy a mostrar todos los entresijos de mi podcast, pero si que te mostraré pasos muy interesantes como: cambio de formatos, descomposición de fechas y un interesantísimo bucle anidado para crear una nueva “feature”.Recuerda que esto aún NO es Inteligencia Artificial o Machine Learning, pero nos da pistas de cuáles serían los siguientes pasos e incluso nos indica si puede merecer la pena.Pero cuidado, no infravalores un análisis de datos ya que aún nos sigue dando información súper valiosa, sin necesidad de añadir ML/AI. De nuevo, disculpa por no mostrarte los últimos resultados, pero como he dicho, es muy potente así que me lo guardo “pa mi” ;)Página oficial del podcast:https://www.podcasteleconomista.com Quien es Jose Garcia:https://www.economistajosegarcia.com Apoyar el podcast:https://ko-fi.com/economista *Tambien puedes ver el video de este episodio:https://www.podcasteleconomista.com/videos/como-hacer-un-analisis-de-datos-basico-data-analysis-marketing-python-pandas-jupyter/Support the Show.

⬇⬇⬇APRIMI⬇⬇ Python e Streamlit: creiamo un app. Auto Elettrica VS Auto a Combustione Link app: https://confronto-auto-combustione-contro-auto-elettrica.streamlit.app/ Link github: https://github.com/DrElegantia/BEV-VS-TRAD/blob/main/BEV-VS-TRAD.py Abbonati qui: https://www.youtube.com/economiaitalia/join https://www.patreon.com/join/EconomiaItalia? Questo codice esegue un'analisi comparativa dei costi tra un'auto elettrica e un'auto a combustione, tenendo conto di vari fattori come il costo iniziale, i costi di ricarica/combustibile, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri costi. Ecco come funziona il codice: Vengono acquisite le informazioni relative ai costi e ai parametri delle auto, come il costo iniziale, i chilometri annuali, i costi di rifornimento/ricarica, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri fattori. Vengono calcolati i costi annuali fissi e variabili per entrambe le auto elettrica e a combustione, tenendo conto dei chilometri annuali, i costi di rifornimento/ricarica, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri fattori. Il calcolo dei costi totali include sia i costi fissi che quelli variabili. Viene calcolato il "Break Even Point" (BEP), che rappresenta il numero di anni necessari affinché i costi totali dell'auto elettrica diventino inferiori rispetto a quelli dell'auto a combustione. Viene determinato quale tipo di auto (elettrica o a combustione) è più conveniente in base al confronto tra i costi variabili annuali. Vengono creati due istogrammi in pila per rappresentare la suddivisione dei costi totali (fissi + variabili) delle auto elettrica e a combustione. Uno degli istogrammi mostra i costi fissi e variabili separatamente, mentre l'altro mostra i costi totali. Viene creato un grafico a linee che mostra l'evoluzione dei costi totali delle due auto nel corso degli anni. Tutti i risultati vengono stampati in output, incluso il confronto tra i costi variabili annuali, il BEP, i costi totali delle due auto e la conclusione su quale auto è più conveniente in base alle impostazioni. Alla fine, i grafici vengono mostrati utilizzando la libreria Matplotlib. In sintesi, il codice ti consente di valutare in modo dettagliato e visivo i costi totali delle auto elettrica e a combustione nel corso degli anni, aiutandoti a prendere decisioni informate sulla scelta del tipo di veicolo da acquistare. Qui per segnalare temi: https://tellonym.me/dr.elegantia Podcast (su tutte le piattaforme): https://www.spreaker.com/show/dr-elegantia-podcast COME SOSTENERCI: Il nostro nuovo libro sull'economia: Guida Terrestre per Autoeconomisti https://www.poliniani.com/product-page/guida-terrestre link acquisto Amazon: https://amzn.to/36XTXs8 Acquistando le nostre T-shirt dedicate ai dati stampate in Serigrafia Artigianale con passione e orgoglio dai detenuti del Carcere Lorusso e Cutugno di Torino https://bit.ly/3zNsdkd e HTTPS://urly.it/3nga1 Guida al VOTO 2022: https://amzn.to/3KflXHd DonazionI Paypal: https://paypal.me/appuntiUAB Vuoi sostenermi ma non sborsare nemmeno un euro? Usa questo link per per il tuo prossimo acquisto su Amazon: https://amzn.to/2JGRyGT Qui trovi i libri che consiglio per iniziare a capirne di più sull'economia: https://www.youtube.com/watch?v=uEaIk8wQ3z8 Dove ci trovi: https://www.umbertobertonelli.it/info/ https://linktr.ee/economiaitalia La mia postazione: Logitech streamcam https://amzn.to/3HR6xq0 Luci https://amzn.to/3n6qtgP Shure MV7https://amzn.to/3HRh7k1 Asta https://amzn.to/3HSRvzY #economiaitalia #drelegantia #economia

⬇⬇⬇APRIMI⬇⬇ Python e Streamlit: creiamo un app. Auto Elettrica VS Auto a Combustione Link app: https://confronto-auto-combustione-contro-auto-elettrica.streamlit.app/ Link github: https://github.com/DrElegantia/BEV-VS-TRAD/blob/main/BEV-VS-TRAD.py Abbonati qui: https://www.youtube.com/economiaitalia/join https://www.patreon.com/join/EconomiaItalia? Questo codice esegue un'analisi comparativa dei costi tra un'auto elettrica e un'auto a combustione, tenendo conto di vari fattori come il costo iniziale, i costi di ricarica/combustibile, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri costi. Ecco come funziona il codice: Vengono acquisite le informazioni relative ai costi e ai parametri delle auto, come il costo iniziale, i chilometri annuali, i costi di rifornimento/ricarica, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri fattori. Vengono calcolati i costi annuali fissi e variabili per entrambe le auto elettrica e a combustione, tenendo conto dei chilometri annuali, i costi di rifornimento/ricarica, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri fattori. Il calcolo dei costi totali include sia i costi fissi che quelli variabili. Viene calcolato il "Break Even Point" (BEP), che rappresenta il numero di anni necessari affinché i costi totali dell'auto elettrica diventino inferiori rispetto a quelli dell'auto a combustione. Viene determinato quale tipo di auto (elettrica o a combustione) è più conveniente in base al confronto tra i costi variabili annuali. Vengono creati due istogrammi in pila per rappresentare la suddivisione dei costi totali (fissi + variabili) delle auto elettrica e a combustione. Uno degli istogrammi mostra i costi fissi e variabili separatamente, mentre l'altro mostra i costi totali. Viene creato un grafico a linee che mostra l'evoluzione dei costi totali delle due auto nel corso degli anni. Tutti i risultati vengono stampati in output, incluso il confronto tra i costi variabili annuali, il BEP, i costi totali delle due auto e la conclusione su quale auto è più conveniente in base alle impostazioni. Alla fine, i grafici vengono mostrati utilizzando la libreria Matplotlib. In sintesi, il codice ti consente di valutare in modo dettagliato e visivo i costi totali delle auto elettrica e a combustione nel corso degli anni, aiutandoti a prendere decisioni informate sulla scelta del tipo di veicolo da acquistare. Qui per segnalare temi: https://tellonym.me/dr.elegantia Podcast (su tutte le piattaforme): https://www.spreaker.com/show/dr-elegantia-podcast COME SOSTENERCI: Il nostro nuovo libro sull'economia: Guida Terrestre per Autoeconomisti https://www.poliniani.com/product-page/guida-terrestre link acquisto Amazon: https://amzn.to/36XTXs8 Acquistando le nostre T-shirt dedicate ai dati stampate in Serigrafia Artigianale con passione e orgoglio dai detenuti del Carcere Lorusso e Cutugno di Torino https://bit.ly/3zNsdkd e HTTPS://urly.it/3nga1 Guida al VOTO 2022: https://amzn.to/3KflXHd DonazionI Paypal: https://paypal.me/appuntiUAB Vuoi sostenermi ma non sborsare nemmeno un euro? Usa questo link per per il tuo prossimo acquisto su Amazon: https://amzn.to/2JGRyGT Qui trovi i libri che consiglio per iniziare a capirne di più sull'economia: https://www.youtube.com/watch?v=uEaIk8wQ3z8 Dove ci trovi: https://www.umbertobertonelli.it/info/ https://linktr.ee/economiaitalia La mia postazione: Logitech streamcam https://amzn.to/3HR6xq0 Luci https://amzn.to/3n6qtgP Shure MV7https://amzn.to/3HRh7k1 Asta https://amzn.to/3HSRvzY #economiaitalia #drelegantia #economiaDiventa un supporter di questo podcast: https://www.spreaker.com/podcast/dr-elegantia-podcast--5692498/support.

Hey ya'll, welcome to yet another ThursdAI, this is Alex coming at you every ThursdAI, including a live recording this time! Which was incredible, we chatted about Falcon 180B,had a great interview in the end with 3 authors of the YaRN scaling paper and LLongMa 128K context, had 3 breaking news! in the middle, MOJO

Python en Excel / Traductor (casi) universal de Facebook / Cadena perpetua por estafa cripto / Airbus 380 reconquista el cielo / Caos de basura espacial Patrocinador: El 23 de agosto llega a Disney Plus la nueva serie original de Star Wars: Ahsoka (tráiler). Volvemos a ver en pantalla a la rebelde Ahsoka Tano en una épica aventura llena de acción, intriga y emociones desbordantes, que solo podrás disfrutar en Disney Plus. Python en Excel / Traductor (casi) universal de Facebook / Cadena perpetua por estafa cripto / Airbus 380 reconquista el cielo / Caos de basura espacial

⬇⬇⬇APRIMI⬇⬇ Python per Confrontare i Costi: Analisi Dettagliata tra Auto Elettriche e Auto a Combustione https://www.mexem.com/ https://colab.research.google.com/drive/17JYoZdrgeJ-oyC3sv7-aMGFG72mLgbxF?usp=sharing Abbonati qui: https://www.youtube.com/economiaitalia/join https://www.patreon.com/join/EconomiaItalia? Questo codice esegue un'analisi comparativa dei costi tra un'auto elettrica e un'auto a combustione, tenendo conto di vari fattori come il costo iniziale, i costi di ricarica/combustibile, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri costi. Ecco come funziona il codice: Vengono acquisite le informazioni relative ai costi e ai parametri delle auto, come il costo iniziale, i chilometri annuali, i costi di rifornimento/ricarica, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri fattori. Vengono calcolati i costi annuali fissi e variabili per entrambe le auto elettrica e a combustione, tenendo conto dei chilometri annuali, i costi di rifornimento/ricarica, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri fattori. Il calcolo dei costi totali include sia i costi fissi che quelli variabili. Viene calcolato il "Break Even Point" (BEP), che rappresenta il numero di anni necessari affinché i costi totali dell'auto elettrica diventino inferiori rispetto a quelli dell'auto a combustione. Viene determinato quale tipo di auto (elettrica o a combustione) è più conveniente in base al confronto tra i costi variabili annuali. Vengono creati due istogrammi in pila per rappresentare la suddivisione dei costi totali (fissi + variabili) delle auto elettrica e a combustione. Uno degli istogrammi mostra i costi fissi e variabili separatamente, mentre l'altro mostra i costi totali. Viene creato un grafico a linee che mostra l'evoluzione dei costi totali delle due auto nel corso degli anni. Tutti i risultati vengono stampati in output, incluso il confronto tra i costi variabili annuali, il BEP, i costi totali delle due auto e la conclusione su quale auto è più conveniente in base alle impostazioni. Alla fine, i grafici vengono mostrati utilizzando la libreria Matplotlib. In sintesi, il codice ti consente di valutare in modo dettagliato e visivo i costi totali delle auto elettrica e a combustione nel corso degli anni, aiutandoti a prendere decisioni informate sulla scelta del tipo di veicolo da acquistare. 00:00 Confrontiamo il costo di un auto elettrica con il costo di un auto a combustione 01:10 Collaborazione con Mexem: scelte di portafoglio 03:04 Usiamo Python per costruire un tool di confronto fra Auto Elettrica e Auto a combustione 09:35 è più conveniente una Golf VIII o una Tesla Model 3? Qui per segnalare temi: https://tellonym.me/dr.elegantia Podcast (su tutte le piattaforme): https://www.spreaker.com/show/dr-elegantia-podcast COME SOSTENERCI: Il nostro nuovo libro sull'economia: Guida Terrestre per Autoeconomisti https://www.poliniani.com/product-page/guida-terrestre link acquisto Amazon: https://amzn.to/36XTXs8 Acquistando le nostre T-shirt dedicate ai dati stampate in Serigrafia Artigianale con passione e orgoglio dai detenuti del Carcere Lorusso e Cutugno di Torino https://bit.ly/3zNsdkd e HTTPS://urly.it/3nga1 Guida al VOTO 2022: https://amzn.to/3KflXHd DonazionI Paypal: https://paypal.me/appuntiUAB Vuoi sostenermi ma non sborsare nemmeno un euro? Usa questo link per per il tuo prossimo acquisto su Amazon: https://amzn.to/2JGRyGT Qui trovi i libri che consiglio per iniziare a capirne di più sull'economia: https://www.youtube.com/watch?v=uEaIk8wQ3z8 Dove ci trovi: https://www.umbertobertonelli.it/info/ https://linktr.ee/economiaitalia La mia postazione: Logitech streamcam https://amzn.to/3HR6xq0 Luci https://amzn.to/3n6qtgP Shure MV7https://amzn.to/3HRh7k1 Asta https://amzn.to/3HSRvzY #economiaitalia #drelegantia #economia

⬇⬇⬇APRIMI⬇⬇ Python per Confrontare i Costi: Analisi Dettagliata tra Auto Elettriche e Auto a Combustione https://www.mexem.com/ https://colab.research.google.com/drive/17JYoZdrgeJ-oyC3sv7-aMGFG72mLgbxF?usp=sharing Abbonati qui: https://www.youtube.com/economiaitalia/join https://www.patreon.com/join/EconomiaItalia? Questo codice esegue un'analisi comparativa dei costi tra un'auto elettrica e un'auto a combustione, tenendo conto di vari fattori come il costo iniziale, i costi di ricarica/combustibile, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri costi. Ecco come funziona il codice: Vengono acquisite le informazioni relative ai costi e ai parametri delle auto, come il costo iniziale, i chilometri annuali, i costi di rifornimento/ricarica, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri fattori. Vengono calcolati i costi annuali fissi e variabili per entrambe le auto elettrica e a combustione, tenendo conto dei chilometri annuali, i costi di rifornimento/ricarica, l'assicurazione, il bollo, la manutenzione, i tagliandi e altri fattori. Il calcolo dei costi totali include sia i costi fissi che quelli variabili. Viene calcolato il "Break Even Point" (BEP), che rappresenta il numero di anni necessari affinché i costi totali dell'auto elettrica diventino inferiori rispetto a quelli dell'auto a combustione. Viene determinato quale tipo di auto (elettrica o a combustione) è più conveniente in base al confronto tra i costi variabili annuali. Vengono creati due istogrammi in pila per rappresentare la suddivisione dei costi totali (fissi + variabili) delle auto elettrica e a combustione. Uno degli istogrammi mostra i costi fissi e variabili separatamente, mentre l'altro mostra i costi totali. Viene creato un grafico a linee che mostra l'evoluzione dei costi totali delle due auto nel corso degli anni. Tutti i risultati vengono stampati in output, incluso il confronto tra i costi variabili annuali, il BEP, i costi totali delle due auto e la conclusione su quale auto è più conveniente in base alle impostazioni. Alla fine, i grafici vengono mostrati utilizzando la libreria Matplotlib. In sintesi, il codice ti consente di valutare in modo dettagliato e visivo i costi totali delle auto elettrica e a combustione nel corso degli anni, aiutandoti a prendere decisioni informate sulla scelta del tipo di veicolo da acquistare. 00:00 Confrontiamo il costo di un auto elettrica con il costo di un auto a combustione 01:10 Collaborazione con Mexem: scelte di portafoglio 03:04 Usiamo Python per costruire un tool di confronto fra Auto Elettrica e Auto a combustione 09:35 è più conveniente una Golf VIII o una Tesla Model 3? Qui per segnalare temi: https://tellonym.me/dr.elegantia Podcast (su tutte le piattaforme): https://www.spreaker.com/show/dr-elegantia-podcast COME SOSTENERCI: Il nostro nuovo libro sull'economia: Guida Terrestre per Autoeconomisti https://www.poliniani.com/product-page/guida-terrestre link acquisto Amazon: https://amzn.to/36XTXs8 Acquistando le nostre T-shirt dedicate ai dati stampate in Serigrafia Artigianale con passione e orgoglio dai detenuti del Carcere Lorusso e Cutugno di Torino https://bit.ly/3zNsdkd e HTTPS://urly.it/3nga1 Guida al VOTO 2022: https://amzn.to/3KflXHd DonazionI Paypal: https://paypal.me/appuntiUAB Vuoi sostenermi ma non sborsare nemmeno un euro? Usa questo link per per il tuo prossimo acquisto su Amazon: https://amzn.to/2JGRyGT Qui trovi i libri che consiglio per iniziare a capirne di più sull'economia: https://www.youtube.com/watch?v=uEaIk8wQ3z8 Dove ci trovi: https://www.umbertobertonelli.it/info/ https://linktr.ee/economiaitalia La mia postazione: Logitech streamcam https://amzn.to/3HR6xq0 Luci https://amzn.to/3n6qtgP Shure MV7https://amzn.to/3HRh7k1 Asta https://amzn.to/3HSRvzY #economiaitalia #drelegantia #economiaDiventa un supporter di questo podcast: https://www.spreaker.com/podcast/dr-elegantia-podcast--5692498/support.

Regardless of which side of Python, software developer or data scientist, you sit on, you surely know that data scientists and software devs seem to have different styles and priorities. But why? And what are the benefits as well as the pitfalls of this separation. That's the topic of conversation with our guest, Dr. Jodie Burchell, data science developer advocate at JetBrains. Links from the show Jodie on Twitter: @t_redactyl Jodie's PyCon Talk: youtube.com Deep Learning with Python book: manning.com Keras: keras.io scikit-learn: scikit-learn.org Matplotlib: matplotlib.org XKCD Matplotlib: matplotlib.org Pandas: pandas.pydata.org Polars: pola.rs Polars on Talk Python: talkpython.fm Jupyter: jupyter.org Ponder: ponder.io Dask: dask.org Explosion AI's Prodigy discount code: get a personal license for 25% off using the discount code TALKPYTHON. Watch this episode on YouTube: youtube.com Episode transcripts: talkpython.fm --- Stay in touch with us --- Subscribe to us on YouTube: youtube.com Follow Talk Python on Mastodon: talkpython Follow Michael on Mastodon: mkennedy Sponsors PyCharm Prodigy Talk Python Training

Wrangling data in Pandas, when to use Pandas, Matplotlib or Seaborn, and why you should learn to create Python packages: Jon Krohn speaks with guest Stefanie Molin, author of Hands-On Data Analysis with Pandas. This episode is brought to you by Posit, the open-source data science company (https://posit.co), and by AWS Inferentia (https://go.aws/3zWS0au). Interested in sponsoring a SuperDataScience Podcast episode? Visit JonKrohn.com/podcast for sponsorship information. In this episode you will learn: • The advantages of using pandas over other libraries [07:55] • Why data wrangling in pandas is so helpful [12:05] • Stefanie's Data Morph library [24:27] • When to use pandas, matplotlib, or seaborn [33:45] • Understanding the ticker module in matplotlib [36:48] • Where data analysts should start their learning journey [40:08] • What it's like being a software engineer at Bloomberg [51:19] Additional materials: www.superdatascience.com/675

EP#66 Willis Nana : Data Ingenieur En savoir plus sur moi : https://espresso-jobs.com/conseils-carriere/les-geeks-du-web-willis-nana/ "Creates simple solutions to complex problems" - Mes Skills Data Engineering

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing Squigglepy, a Python package for Squiggle, published by Peter Wildeford on October 19, 2022 on The Effective Altruism Forum. Squiggle is a "simple programming language for intuitive probabilistic estimation". It serves as its own standalone programming language with its own syntax, but it is implemented in JavaScript. I like the features of Squiggle and intend to use it frequently, but I also frequently want to use similar functionalities in Python, especially alongside other Python statistical programming packages like Numpy, Pandas, and Matplotlib. The squigglepy package here implements many Squiggle-like functionalities in Python. The package also has useful utility functions for Bayesian networks (using rejection sampling), pooling forecasts (via weighted geometric mean of odds and others), laplace (including the time-invariant version), and kelly betting. The package and documentation are available on GitHub. The package can be downloaded from Pypi using pip install squigglepy. This package is unofficial and supported by myself and Rethink Priorities. It is not affiliated with or associated with the Quantified Uncertainty Research Institute, which maintains the Squiggle language (in JavaScript). This package is also new and not yet in a stable production version, so you may encounter bugs and other errors. Please report those so they can be fixed. It's also possible that future versions of the package may introduce breaking changes. This package is available under an MIT license. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Guest Melissa Mendonça Panelists Richard Littauer | Amanda Casari Show Notes Hello and welcome to Sustain! The podcast where we talk about sustaining open source for the long haul. Today, we are so excited to have a wonderful guest, Melissa Mendonça, joining us. Melissa is a Senior Developer Experience Engineer at Quansight, where she focuses more on developer experience and contributor experience. Today, we'll hear all about Quansight and the focus for Melissa's role as a Developer Experience Engineer. Melissa tells us about a grant they are working on with CZI that focuses on NumPy, SciPy, Matplotlib, and pandas, she shares several ideas on what can be done to make people feel seen and heard, and we hear her thoughts on what the future of community management and community development looks like for people entering the role of these projects. Go ahead and download this episode now to hear more! [00:01:25] Melissa tells us her background and her role at Quansight. [00:03:41] When Melissa made the decision to switch from one role to another, Amanda asks if that was her plan or if she learned that the skills that she needed to get things done changed over time. [00:06:10] We find out what the focus is for Melissa's role as a Developer Experience Engineer and what she does on a day-to-day basis. [00:08:43] As Melissa was talking about her projects that they work on at Quansight, Amanda wonders if that's the majority of her portfolio, or if she works across different kinds of projects. We learn about the current grant they are working on with CZI that focuses on NumPy, SciPy, Matplotlib, and pandas. [00:13:18] We learn about the funding model and how sustainable it is. [00:16:20] Melissa shares some great ideas on how we can put more effort into making people feel seen and heard. [00:19:26] Melissa details some things she learned with the open source projects and things she recommends for others with large established projects. [00:22:44] Amanda talks about a 2020 paper that was released in nature called “Array programming with NumPy,” and Melissa gives us her perspective on what happened with the community in 2020, if things have changed, and what needs to be addressed. [00:27:09] Find out how CZI got involved with Melissa's work, what their goals are, and how she's changing in order to adapt towards those goals. [00:31:32] Melissa shares her thoughts on what the future of community management and community development looks like for people who are entering the role for those projects. [00:36:40] We hear more about Python Brasil 2022 that's coming up. [00:38:05] Find out where you can follow Melissa online and learn more about her work. Quotes [00:02:49] “Since Quansight is a company very focused on sustaining and helping maintain open source projects, we are trying to help new contributors, people who want to do the move from contributor to maintainer, understanding what that means, and how we can help them get there, and how we can help improve leadership in our open source projects.” [00:11:53] “This is one of the barriers that we want to break, is that making sure that people understand that these are important, they are core projects in the scientific Python ecosystem, but at the same time they are projects just like any other.” [00:12:17] “I think experience of working with projects that are so old and big has taught me a lot about the dynamics of how people work and how new people try to join these projects and how we can improve on that.” [00:16:41] “We need to make sure that people who do contribution outside of code are credited and that they are valued inside open source projects.” [00:18:20] “I think we should think about diversifying these paths for contribution, but for that we need to go beyond GitHub. We need to go beyond the current metrics that we have for open source, we need to go beyond the current credit system and reputation system that we have for open source contributions.” [00:30:38] “Community managers are not second-class citizens.” Spotlight [00:39:21 Amanda's spotlight is a 2014 paper from MSR called, “The Promises and Perils of Mining GitHub.” [00:40:48] Richard's spotlight is the book, Don't Sleep, There Are Snakes, by Daniel Everett. [00:41:52] Melissa's spotlights are Ralf Gommers and Scientific Python initiative. Links SustainOSS (https://sustainoss.org/) SustainOSS Twitter (https://twitter.com/SustainOSS?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) SustainOSS Discourse (https://discourse.sustainoss.org/) podcast@sustainoss.org (mailto:podcast@sustainoss.org) Richard Littauer Twitter (https://twitter.com/richlitt?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Amanda Casari Twitter (https://twitter.com/amcasari?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor) Melissa Mendonça Twitter (https://twitter.com/melissawm) Melissa Mendonça LinkedIn (https://br.linkedin.com/in/axequalsb) Melissa Mendonça GitHub (https://melissawm.github.io/) Quansight (https://quansight.com/) Quansight Labs (https://labs.quansight.org/) Quansight Lab Projects (https://labs.quansight.org/projects) Quansight Labs Team (https://labs.quansight.org/team) Sustain Podcast-Episode 57: Mikeal Rogers on Building Communities, the Early Days of Node.js, and How to Stay a Coder for Life (https://podcast.sustainoss.org/guests/mikeal) Sustain Podcast-Episode 85: Geoffrey Huntley and Sustaining OSS with Gitpod (https://podcast.sustainoss.org/85) Advancing an inclusive culture in the scientific Python ecosystem (CZI grant for NumPy, SciPy, Matplotlib, and Pandas (https://figshare.com/articles/online_resource/Advancing_an_inclusive_culture_in_the_scientific_Python_ecosystem/16548063) Sustain Podcast-Episode 79: Leah Silen on how NumFocus helps makes scientific code more sustainable (https://podcast.sustainoss.org/79) NumPy (https://numpy.org/) SciPy (https://scipy.org/) Matplotlib (https://matplotlib.org/) pandas (https://pandas.pydata.org/) Sustain Podcast-Episode 64: Travis Oliphant and Russell Pekrul on NumPy, Anaconda, and giving back with FairOSS (https://podcast.sustainoss.org/guests/oliphant) Tania Allard Twitter (https://twitter.com/ixek?lang=en) Array programming with NumPy (nature) (https://www.nature.com/articles/s41586-020-2649-2) Python Brasil 2022 (https://2022.pythonbrasil.org.br/) “The Promises and Perils of Mining GitHub,” by Eirini Kalliamvakou, Georgios Gousios, Kelly Blincoe, Leif Singer, Daniel M. German, Daniela Damian (https://kblincoe.github.io/publications/2014_MSR_Promises_Perils.pdf) “The Promises and Perils of Mining GitHub,” by Eirini Kalliamvakou, Georgios Gousios, Kelly Blincoe, Leif Singer, Daniel M. German, Daniela Damian (ACM Digital Library) (https://dl.acm.org/doi/10.1145/2597073.2597074) Daniel Everett (Wikipedia) (https://en.wikipedia.org/wiki/Daniel_Everett#Don't_Sleep,_There_Are_Snakes:_Life_and_Language_in_the_Amazonian_Jungle) Excerpt: ‘Don't Sleep, There Are Snakes' (npr) (https://www.npr.org/2009/12/23/121515579/excerpt-dont-sleep-there-are-snakes?t=1661871384424) Ralf Gommers (GitHub) (https://github.com/rgommers) Scientific Python (https://scientific-python.org/) Credits Produced by Richard Littauer (https://www.burntfen.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr Peachtree Sound (https://www.peachtreesound.com/) Special Guest: Melissa Mendonça.

Do you struggle to know where to start in the wide range of Python's visualization frameworks? Not sure when to use Plotly vs. Matplotlib vs. Altair? Then this episode is for you. We have Chris Moffitt, a Talk Python course author and founder of Practical Business Python, back on the show to discuss getting started with Python's data visualization frameworks. Links from the show Chris on Twitter: @chris1610 Python Data Visualization course: talkpython.fm 10 tips to move from Excel to Python episode: talkpython.fm Escaping Excel Hell with Python and Pandas episode: talkpython.fm PB Python: pbpython.com matplotlib: matplotlib.org Seaborn example: seaborn.pydata.org Altair: altair-viz.github.io Plotly sunburst: plotly.com Plotly treemap: plotly.com streamlit: streamlit.io Dash: dash.gallery Streamlit Talk Python episode: talkpython.fm splink package: github.com redframes package: github.com Edward Tufte book: edwardtufte.com Watch this episode on YouTube: youtube.com Episode transcripts: talkpython.fm --- Stay in touch with us --- Subscribe to us on YouTube: youtube.com Follow Talk Python on Twitter: @talkpython Follow Michael on Twitter: @mkennedy Sponsors Microsoft Talk Python Training AssemblyAI

Have you thought the standard output from Matplotlib is a bit generic looking? Would you like a quick way to add style and consistency to your data visualizations? This week on the show, Christopher Trudeau is here, bringing another batch of PyCoder's Weekly articles and projects.

Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Brian #1: Uncommon Uses of Python in Commonly Used Libraries by Eugene Yan Specifically, Using relative imports Example from sklearn's base.py from .utils.validation import check_X_y from .utils.validation import check_array “Relative imports ensure we search the current package (and import from it) before searching the rest of the PYTHONPATH. “ For relative imports, we have to use the from .something import thing form. We cannot use import .something since later on in the code .something isn't valid. There's a good discussion of relative imports in pep 328 Michael #2: Skyplane Cloud Transfers Skyplane is a tool for blazingly fast bulk data transfers in the cloud. Skyplane manages parallelism, data partitioning, and network paths to optimize data transfers, and can also spin up VM instances to increase transfer throughput. You can use Skyplane to transfer data: Between buckets within a cloud provider Between object stores across multiple cloud providers (experimental) Between local storage and cloud object stores Skyplane takes several steps to ensure the correctness of transfers: Checksums, verify files exist and match sizes. Data transfers in Skyplane are encrypted end-to-end. Security: Encrypted while in transit and over TLS + config options Brian #3: 7 things I've learned building a modern TUI framework by Will McGugan Specifically, DictViews are amazing. They have set operations. Example of using items() to get views, then ^ for symmetric difference (done at the C level): # Get widgets which are new or changed print(render_map.items() ^ new_render_map.items()) Lots of other great topics in the article lru_cache is fast Unicode art in addition to text in doc strings The fractions module and a cool embedded video demo of some of the new css stuff in Textual Python's object allocator ascii art Michael #4: ‘Unstoppable' Python Python popularity still soaring: ‘Unstoppable' Python once again ranked No. 1 in the August updates of both the Tiobe and Pypl indexes of programming language popularity. Python first took the top spot in the index last October, becoming the only language besides Java and C to hold the No. 1 position. “Python seems to be unstoppable,” said the Tiobe commentary accompanying the August index. In the alternative Pypl Popularity of Programming Language index, which assesses language popularity based on Google searches of programming language tutorials, Python is way out front. Extras Brian: Matplotlib stylesheets can make your chart look awesome with one line of code. But it never occurred to me that I could write my own style sheet. Here's an article discussing creation of custom matplotlib stylesheets The Magic of Matplotlib Stylesheets XKCD Plots Michael: Back on 295 we talked about Flet. We now have a Talk Python episode on it (live and polished versions). Joke: Rakes and AWS

Brent sits down with Tim Canham, Senior Software Engineer at NASA's Jet Propulsion Laboratory. We explore topics including the hardware and software powering NASA's Ingenuity Mars Helicopter; JPL's switch from Solaris to Linux; the open source projects, tools, and philosophy at JPL, ...and more. Special Guest: Tim Canham.

In this article, I will show you how to do noise reduction in 3 simple steps. We will be using a machine learning trained noise reduction model. It is one of the best noise reduction models I've found out there. I will share more about the model and how to apply it in the following paragraphs. In this project, we will be using three python packages. The packages are as follows: OpenCV, Matplotlib, and NumPy. OpenCV is a very well-known kit for computer vision. As a prerequisite for OpenCV library, we will need to install Numpy. We transform pixels into arrays when reading an image; NumPy is going to do that in the behind scenes. When dealing with multi-dimensional arrays, NumPy is perfect. This episode is also available as a blog post: http://sonsuzdesign.blog/2021/02/27/enhance-your-images-using-opencv-noise-reduction-algorithm/

Panelists Eric Berry | Justin Dorfman | Alyssa Wright | Richard Littauer Guest Travis Oliphant | Russell Pekrul Show Notes Hello and welcome to Sustain! Today, we have two guests from OpenTeams in Austin, Travis Oliphant and Russell Pekrul. Travis is the CEO and Russell is the Program Manager and the Founder and Director of FairOSS. We learn all about what OpenTeams and FairOSS are and how they work. Also, Travis tells us about the non-profit he started called NumFOCUS. Other topics discussed are dependencies and how their values are assigned, NumPy and SciPy, and building relationships with companies, which Russell mentions there is a bit of a “chicken and egg” problem here. There is some incredible advice and fascinating stories shared today so go ahead and download this episode now! [00:01:10] We find out what OpenTeams is and how it works. Travis also tells us when he wrote NumPy and SciPy and when he started OpenTeams. [00:07:18] Travis tells us about a non-profit he started with a bunch of people called NumFOCUS so there could be a home for the fiscal sponsor for open source projects. [00:09:24] Russell tells us what FairOSS is and how it works. [00:11:32] Alyssa asks Russell how does he first see the dependencies and then how does he assign that value? He mentions BackYourStack as a starting point. [00:13:00] Eric brings up one of the problems he’s found with trying to fund up open source is that it’s very difficult to solve the problem on more a grand scale. He wonders how Travis and Russell make the impact they want with the magnitude of problems they see. A key piece Travis brings up that they recognize is there’s a data gap and projects have to be participating. Alyssa wonders if projects are aware of their dependencies. [00:17:22] Richard asks about the dependency graph that they are making. He wonders how do you go down the stack and look all the way at the base and how do you judge the usefulness of what dependencies really matter for what code matters for the business proposition? Richard also wonders if anyone has done equity stuff for open source maintainers. [00:23:06] Alyssa is interested in learning more about how Travis and Russell are building the relationships with these companies and what we can do to help. [00:26:35] Alyssa asks Travis and Russell to talk about why this, why now, with this being a time of economic contraction, why is this important? Also, why have they been seeing traction during what can be difficult times for a lot of companies? [00:27:40] Eric asks if Travis can give an example of a project that he feels does that well, that doesn’t have to go through and do it twice, essentially. [00:29:48] Alyssa brings up investments around open source start-ups and how they start with a commitment towards open source and once the investment happens there’s a pivot. She wonders if Travis could talk about how this type of sustainability is shifting that model of these investments. Travis tells a story about speaking to the Founder of SaltStack and how their views matched. [00:34:03] We find out where you can learn more about FairOSS and follow them on this journey, invest, and join in. Spotlight [00:34:52] Justin’s spotlight is Curiefense, which extends Envoy proxy to protect all forms of web traffic. [00:35:15] Alyssa’s spotlight is Pixel8.earth. [00:36:06] Eric’s spotlight is OctoPrint. [00:36:53] Richard’s spotlight is Michael Oliphant’s work. [00:37:36] Russell’s spotlight is Conda. [00:38:20] Travis’s spotlight is Matplotlib. Quotes [00:03:25] “We were connecting and creating a social network long before the social networks started. That was the early days of social networks and it was addicting.” [00:04:14] “New libraries are starting to be written on numarray and we had SciPy written on numeric and there was this fork in this flegging scientific community in Python.” [00:21:18] “So that was a very exciting day. Actually, I remember I told my wife you know the problem I’ve been searching on for twenty years, I finally figured it out. I’ve been trying to figure out twenty years how to make this work, and I finally figured it out. I had to go start several companies and start a venture fund and get involved in finance and cap tables to really pull it off, but that got me excited. Now I also said, but we’re at the base of Mount Everest, like all we’ve got to do is climb to the top of this mountain and we’re there.” [00:22:44] “So you basically have a company and its value is spread to all the values of the projects. You have a bunch of those, have a thousand of those, that each add incrementally the value of a project. Invert the matrix and every project now has a linear dependency on companies that effectively you created an index fund out of every project.” [00:24:52] “The idea is if you can get open source contributors to recognize that they want to work only for companies that are participating people want to hire open source contributors. They’re some of the best people to bring into your company.” [00:25:21] “We found that companies would absolutely sponsor PyData and the reason they would is because they’re trying to hire people. They wanted to hire the best developers and they would. So, they really didn’t care so much about the projects they started, but they wanted the people.” [00:27:10] “Go make an open source project, then get somebody or connect with somebody who’s going to help you build a company that they’ll vest in and build something else. So, you basically have to do it twice.” [00:28:34] “I’ve had the chance to work at companies large and small, go in and see that’s used to do x, and realized it’s added billions of dollars of value to a lot of work for the world. And yet, the same time NumPy struggled, not enough funding to maintain itself.” [00:30:15] “I spoke to the founder of SaltStack that just got acquired by VMware. I spoke to him about his view and it was amazing how much it matched mine, in a sense that he recognized that open source is you build some of the value and you use it. The way you need to make money is to build something that uses it but isn’t the open source.” [00:32:41] “It’s not you’re monetizing open source, you’re empowering, you’re sustaining open source, by selling and connecting the economic value to the functional value that’s there.” [00:33:04] “There will still be challenges. I’m not naïve. Every new thing comes with a whole set of new challenges.” Links OpenTeams (https://openteams.com/about) FairOSS (https://faiross.org/) FairOSS, PBC Twitter (https://twitter.com/faiross_pbc) FairOSS Community (https://community.faiross.org/login) Travis Oliphant Twitter (https://twitter.com/teoliphant?lang=en) Anaconda Dividend Program (https://www.anaconda.com/blog/sustaining-the-open-source-ds-ml-ecosystem-with-the-anaconda-dividend-program) Quansight (https://www.quansight.com/) NumFOCUS (https://numfocus.org/) BackYourStack (https://backyourstack.com/) Dask (https://dask.org/) SaltStack (https://www.saltstack.com/) SciPy (https://www.scipy.org/) NumPy (https://numpy.org/) Curiefense (https://www.curiefense.io/) Pixel8.earth Ambassador Program (https://pixel8earth.medium.com/kicking-off-the-pixel8-earth-ambassador-program-80a87a70fb3a) OctoPrint (https://octoprint.org/) Michael Oliphant’s work (https://langev.com/index.php/author/moliphant/Michael+Oliphant) Conda (https://github.com/conda/conda) Matplotlib.com (https://matplotlib.org/) Credits Produced by Richard Littauer (https://www.burntfen.com/) Edited by Paul M. Bahr at Peachtree Sound (https://www.peachtreesound.com/) Show notes by DeAnn Bahr at Peachtree Sound (https://www.peachtreesound.com/) Special Guests: Russell Pekrul and Travis Oliphant.

In this episode #27, the hosts Naveen Samala & Sudhakar Nagandla have interacted with another guest Yaswanth. Yaswanth Sai Palaghat is currently working as a software engineer at a product based company in Hyderabad. A machine learning Evangelist with a major content delivery focus on machine learning. Owns a python library developed during his engineering.  Active GitHub user with more than 150 real-time projects available in GitHub. He considers himself as a Techie, loves exploring technologies and teaching the same in his community. He is an Udemy instructor with active courses. Yashwant is a YouTuber with a well-strengthened YouTube channel with the motto of delivering tech content and will always try to motivate engineering students with his videos. A content developer with active tech content delivery in all forms(text, graphics, videos).  A blogger and runs an active blog titled "TECHIE EMPIRE" which aims to help and motivate the engineering graduates to choose the right path on technologies. He is very active in social media with more than 20k connections all over his social profiles and pages in LinkedIn, Instagram and YouTube. Coming to his education, Yaswanth graduated from Narayana Engineering College in the stream of Computer Science and Engineering in 2019 in which he was a college topper.  Apart from tech life, he is interested in film making where he is into story writing, editing, direction and a protagonist too. He Owns a YouTube channel named "THE FUN BITE" with more than 10 short films. Listen to Yaswanth's guidance on: How to Explain Machine Learning to your grandparents? How ML can help in solving real world problems? How to start leaning ML? Resources for learning ML Is it important to have mathematical background for ML? ML vs Deep Learning – Basic differences How ML is related to AI & Data Science? Yaswanth's LinkedIn profile: https://www.linkedin.com/in/yaswanthpalaghat/ Resources shared by Yaswanth: 1. Linear Algebra & Statistics: Linear Algebra for ML :https://youtu.be/1VSZtNYMntM Statistics for ML : https://youtu.be/hjZJIVWHnPE 2. Python Programming:    Python for Absolute Beginners: https://www.udemy.com/course/pythonforabsolutebeginners/?referralCode=E2DBB8598710151C2783 3. Data Analytics with Python Libraries(Numpy, Pandas, Matplotlib):     Data Analytics A-Z with Python: https://www.udemy.com/course/dataanalyticsa-zwithpython/?referralCode=AB085EE2CA864CB69FD7 4. Data Cleaning:    https://youtu.be/2HzsTwHL7H0 5.Project-I     Data Analytics on Iris Flowers Dataset : https://github.com/yaswanthpalaghat/Pandas-implementation-and-workflow-on-iris-flowers-dataset   6.Computer Vision with OpenCV:    Understanding OpenCV : https://opencv.org/  7.Project-II    Building a Face Detection and Recognition Model : https://www.udemy.com/course/building-a-face-detection-and-recognition-model-from-scratch/?referralCode=28F5323F045A89665F90 8.Machine Learning with Scikit-Learn: https://youtu.be/pqNCD_5r0IU 9.Hands-on with ML Algorithms: https://youtu.be/RnFGwxJwx-0 10.Solving ML Challenges:   https://www.kaggle.com/learn/microchallenges 11.Project-III: Linear Regression on Diabetes DataSet :    https://github.com/yaswanthpalaghat/Linear-Regression-on-Diabetes-data-set-to-find-out-the-predicted-data-set 12. Natural Language Processing and Text analytics:  https://youtu.be/OQmDhwhj78Y 13.Project-IV      Sentiment Analysis using NLP : https://github.com/yaswanthpalaghat/NLP-Sentiment-analysis-using-Machine-Learning-and-flask Enjoy the episode! Do not forget to share your suggestions or feedback at theguidingvoice4u@gmail.com or by messaging at +91 9494 587 187  Subscribe to our YouTube Channel: https://www.youtube.com/c/TheGuidingVoice Also, follow The Guiding Voice on Social Media: LinkedIn: https://www.linkedin.com/company/theguidingvoice Facebook: http://facebook.com/theguidingvoice4u Twitter: http://twitter.com/guidingvoice Instagram: https://www.instagram.com/theguidingvoice4u/ Pinterest: https://in.pinterest.com/theguidingvoice4u/pins/  #growth #data #ML #Machinelearning #supervisedlearning #unsupervisedlearning #deeplearning  #Neuralnetwork #ANN #datascience #datascientist #machinelearning #AI #AIbasics #technologies #kdnuggets #career #jobs #careerguidance #mentorship #careerpath #progression #management #leadership #crisis #job #midcareer #youngprofessionals #careergraph #TGV #theguidingvoice #kaggle #analyticsvidya #udemy #insofe #coursera #opensource #transformation    

Link to bioRxiv paper: http://biorxiv.org/cgi/content/short/2020.08.13.249110v1?rss=1 Authors: Junier, I., Varoquaux, N. Abstract: Genome wide contact frequencies obtained using Hi-C-like experiments have raised novel challenges in terms of visualization and rationalization of chromosome structuring phenomena. In bacteria, display of Hi-C data should be congruent with the circularity of chromosomes. However, standard representations under the form of square matrices or horizontal bands are not adapted to periodic conditions as those imposed by (most) bacterial chromosomes. Here, we fill this gap and propose a Python library, built upon the widely used Matplotlib library, to display Hi-C data in circular strips, together with the possibility to overlay genomic data. The proposed tools are light and fast, aiming to facilitate the exploration and understanding of bacterial chromosome structuring data. The library further includes the possibility to handle linear chromosomes, providing a fresh way to display and explore eukaryotic data. Availability and implementation: The package runs under Python 3 and is freely available at https://github.com/TrEE-TIMC/circHiC. The documentation can be found at https://tree-timc.github.io/circhic/; images obtained in different organisms are provided in the gallery section and are accompanied with codes. Copy rights belong to original authors. Visit the link for more info

Haarlem, 1956. No, this isn't an episode about New York, we're talking Haarlem, Netherlands. Guido Van Rossum is born then, and goes on to college in Amsterdam where he gets a degree in math and computer science. He went on to work at the Centrum Wiskunde & Informatica, or CWI. Here, he worked on BSD Unix and the ABC Programming language, which had been written by Lambert Meertens, Leo Geurts, and Steven Pemberton from CWI.  He'd worked on ABC for a few years through the 1980s and started to realize some issues. It had initially been a monolithic implementation, which made it hard to implement certain new features, like being able to access file systems and functions within operating systems. But Meertens was an editor of the ALGOL 68 Report and so ABC did have a lot of the ALGOL 68 influences that are prevalent in a number of more modern languages and could compile for a number of operating systems. It was a great way to spend your 20s if you're Guido. But after some time building interpreters and operating systems, many programmers think they have some ideas for what they might do if they just… started over. Especially when they hit their 30s. And so as we turned the corner towards the increasingly big hair of the 1990s, Guido started a new hobby project over the holiday break for Christmas 1989.  He had been thinking of a new scripting language, loosely based on ABC. One that Unix and C programmers would be interested in, but maybe not as cumbersome as C had become. So he got to work on an interpreter. One that those open source type hackers might be interested in. ALGOL had been great for math, but we needed so much more flexibility in the 90s, unlike bangs. Bangs just needed Aquanet. He named his new creation Python because he loved Monty Python's Flying Circus. They had a great TV show from 1969 to 1974, and a string of movies in the 70s and early 80s. They've been popular amongst people in IT since I got into IT. Python is a funny language. It's incredibly dynamic. Like bash or a shell, we can fire it up, define a variable and echo that out on the fly. But it can also be procedural, object-oriented, or functional. And it has a standard library but is extensible so you can add libraries to do tons of new things that wouldn't make sense to be built in (and so bloat and slow down) other apps. For example, need to get started with big array processing for machine learning projects? Install TensorFlow or Numpy. Or according to your machine learning needs you have PyTorch, SciPi, Pandas, and the list goes on.  In 1994, 20 developers met at the US National Standards Bureau in Maryland, at the first workshop and the first Python evangelists were minted. It was obvious pretty quickly that the modular nature and ease of scripting, but with an ability to do incredibly complicated tasks, was something special. What was drawing this community in. Well, let's start with the philosophy, the Zen of Python as Tim Peters wrote it in 1999: Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one—and preferably only one—obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than right now.[a] If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea—let's do more of those! Those are important enough to be semi-official and can be found by entering “import this” into a python shell. Another reason python became important is that it's multi-paradigm. When I said it could be kinda' functional. Sure. Use one big old function for everything if you're moving from COBOL and just don't wanna' rethink the world. Or be overly object-oriented when you move from Java and build 800 functions to echo hello world in 800 ways. Wanna map reduce your lisp code. Bring it. Or add an extension and program in paradigms I've never heard of. The number of libraries and other ways to extend python out there is pretty much infinite.  And that extensibility was the opposite of ABC and why Python is special. This isn't to take anything away from the syntax. It's meant to be and is an easily readable language. It's very Dutch, with not a lot of frills like that. It uses white space much as the Dutch use silence. I wish it could stare at me like I was an idiot the way the Dutch often do. But alas, it doesn't have eyeballs. Wait, I think there's a library for that.  So what I meant by white space instead of punctuation is that it uses an indent instead of a curly bracket or keyword to delimit blocks of code. Increase the tabbing and you move to a new block. Many programmers do this in other languages just for readability. Python does it for code.  Basic statements included, which match or are similar to most languages, include if, for, while, try, raise, except, class, def, with, break, continue, pass, assert, yield, import and print until python 3 when that became a function. It's amazing what you can build with just a dozen and a half statements in programming. You can have more, but interpreters get slower and compilers get bigger and all that…  Python also has all the expressions you'd expect in a modern language, especial lambdas. And methods. And duck typing, or suitability for a method is determined by the properties of an object rather than the type. This can be great. Or a total pain. Which is why they'll eventually be moving to gradual typing.  The types of objects are bool, byte array, bytes, complex, dict, ellipsis (which I overuse), float, frozen set, int, list, NoneType (which I try to never use), NotImplementedType, range, set, str, and tuple so you can pop mixed tapes into a given object. Not to be confused with a thruple, but not to not be confused I guess…  Another draw of python was the cross-compiler concept. An early decision was to make python cable to talk to c. This won over the Unix and growing Linux crowds. And today we have cross-compilers for C and C++, Go, .Net, Java, R, machine code, and of course, Java.   Python 2 came in 2000. We got a garbage collection system and a few other features and 7 point releases over the next 10 years. Python 3 came in 2008 and represented a big change. It was partially backward-compatible but was the first Python release that wasn't fully backward-compatible. We have had 7 point releases in the past 10 years as well. 3 brought changes to function print, simpler syntax, moved to storing strings in unicode by default, added a range function, changed how global variables react inside for-loops, implemented a simpler set of rules for order comparisons, and much more.  At this point developers were experimenting with deploying microservices. Microservices is an a software development architecture where we build small services, perhaps just a script or a few scripts daisy chained together, that do small tasks. These are then more highly maintainable, more easily testable, often more scalable, can be edited and deployed independently, can be structured around capabilities, and each of the services can be owned by the team that created it with a contract to ensure we don't screw over other teams as we edit them.  Amazon introduced AWS Lambda in 2014 and it became clear quickly that the new micro services paradigm was accelerating the move of many SaaS-based tools to a micro services architecture. Now, teams could build in node or python or java or ruby or c# or heaven forbid Go. They could quickly stand up a small service and get teams able to consume the back end service in a way that is scalable and doesn't require standing up a server or even a virtual server, which is how we did things in EC2. The containerization concept is nothing new. We had chroot in 1979 with Unix v7 and Solaris brought us containerization in 2004. But those were more about security. Docker had shown up in 2013 and the idea of spinning up a container to run a script and give it its own library and lib container, that was special. And Amazon made it more so.  Again, libraries and modularization. And the modular nature is key for me. Let's say you need to do image processing. Pillow makes it easier to work with images of almost any image type you can think of. For example, it can display an image, convert it into different types, automatically generate thumbnails, run sooth, blur, contour, and even increase the detail. Libraries like that take a lot of the friction out of learning to display and manage images.  But Python can also create its own imagery. For example, Matplotlib generates two dimensional graphs and plots points on them. These can look as good as you want them to look and actually allows us to integrate with a ton of other systems.  Van Rossum's career wasn't all python though. He would go on to work at NIST then CNRI and Zope before ending up at Google in 2005, where he created Mondrian, a code review system. He would go to Dropbox in 2013 and retire from professional life in 2019. He stepped down as the “Benevolent dictator for life” of the Python project in 2018 and sat on the Python Steering Council for a term but is no longer involved. It's been one of the most intriguing “Transfers of power” I've seen but Python is in great hands to thrive in the future. This is the point when Python 2 was officially discontinued, and Python 3.5.x was thriving.  By thriving, as of mid-202, there are over 200,000 packages in the Python Package Index. Things from web frameworks and web scraping to automation, to graphical user interfaces, documentation, databases, analytics, networking, systems administrations, science, mobile, image management and processing. If you can think of it, there's probably a package to help you do it. And it's one of the easier languages.  Here's the thing. Python grew because of how flexible and easy it is to use. It didn't have the same amount of baggage as other languages. And that flexibility and modular nature made it great for workloads in a changing and more micro-service oriented world. Or, did it help make the world more micro-service oriented. It was a Christmas hobby project that has now ballooned into one of the most popular languages to write software in the word. You know what I did over my last holiday break? Sleep. I clearly should have watched more Monty Python so the short skits could embolden me to write a language perfect for making the programmers equivalent, smaller, more modular scripts and functions. So as we turn the corner into all the holidays in front of us, consider this while stuck at home, what hobby project can we propel forward and hopefully end up with the same type of impact Guido had. A true revolutionary in his own right.  So thank you to everyone involved in python and everyone that's contributed to those 200k+ projects. And thank you, listeners, for continuing to tun in to the history of computing podcast. We are so lucky to have you.

PyChat: Free Python DS book (online) Intro to NumPy, Matplotlib & sci-kit learn CfP for FlaskCon is open! Way to have more diverse board members is to have diverse voters - become a voting member of the PSF Test pip's alpha resolver New pymsbuild for windows people Workshop online PythonIreland Mid Meet - Hall of Fame: Ian Ozsvald - co-founder of PyData London Follow Ian on Twitter @ianozsvald PyPI highlights: Tesseract OCR text localization & detection Shed - black plus autoflake plus isort plus pyupgrade plus some custom fixers Listen to MidMeetPy podcast

Sponsored by Datadog: pythonbytes.fm/datadog Michael #1: PSF / JetBrains Survey via Jose Nario Let’s talk results: 84% of people who use Python do so as their primary language [unchanged] Other languages: JavaScript (down), Bash (down), HTML (down), C++ (down) Web vs Data Science languages: More C++ / Java / R / C# on Data Science side More SQL / JavaScript / HTML Why do you mainly use Python? 58% work and personal What do you use Python for? Average answers was 3.9 Data analysis [59% / 59% — now vs. last year] Web Development [51% / 55%] ML [40% / 39%] DevOps [39% / 43%] What do you use Python for the most? Web [28% / 29%] Data analysis [18% / 17%] Machine Learning [13% / 11%] Python 3 vs Python 2: 90% Python 3, 10% Python 2 Widest disparity of versions (pro 3) is in data science. Web Frameworks: Flask [48%] Django [44%] Data Science NumPy 63% Pandas 55% Matplotlib 46% Testing pytest 49% unittest 30% none 34% Cloud AWS 55% Google 33% DigitalOcean 22% Heroku 20% Azure 19% How do you run code in the cloud (in the production environment) Containers 47% VMs 46% PAAS 25% Editors PyCharm 33% VS Code 24% Vim 9% tool use version control 90% write tests 80% code linting 80% use type hints 65% code coverage 52% Brian #2: Hypermodern Python Claudio Jolowicz, @cjolowicz An opinionated and fun tour of Python development practices. Chapter 1: Setup Setup a project with pyenv and Poetry, src layout, virtual environments, dependency management, click for CLI, using requests for a REST API. Chapter 2: Testing Unit testing with pytest, using coverage.py, nox for automation, pytest-mock. Plus refactoring, handling exceptions, fakes, end-to-end testing opinions. Chapter 3: Linting Flake8, Black, import-order, bugbear, bandit, Safety. Plus more on managing dependencies, and using pre-commit for git hooks. Chapter 4: Typing mypy and pytype, adding annotations, data validation with Desert & Marshmallow, Typeguard, flake8-annotations, adding checks to test suite Chapter 5: Documentation docstrings, linting docstrings, docstrings in nox sessions and test suites, darglint, xdoctest, Sphinx, reStructuredText, and autodoc Chapter 6: CI/CD CI with GithHub Actions, reporting coverage with Codecov, uploading to PyPI, Release Drafter for release documentation, single-sourcing the package version, using TestPyPI, docs on RTD The series is worth it even for just the artwork. Lots of fun tools to try, lots to learn. Michael #3: Open AI Jukebox via Dan Bader Listen to the songs under “Curated samples.” A neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. Code is available on github. Dataset: To train this model, we crawled the web to curate a new dataset of 1.2 million songs (600,000 of which are in English), paired with the corresponding lyrics and metadata from LyricWiki. The top-level transformer is trained on the task of predicting compressed audio tokens. We can provide additional information, such as the artist and genre for each song. Two advantages: first, it reduces the entropy of the audio prediction, so the model is able to achieve better quality in any particular style; second, at generation time, we are able to steer the model to generate in a style of our choosing. Brian #4: The Curious Case of Python's Context Manager Redowan Delowar, @rednafi A quick tour of context managers that goes deeper than most introducitons. Writing custom context managers with __init__, __enter__, __exit__. Using the decorator contextlib.contextmanager Then it gets even more fun Context managers as decorators Nesting contexts within one with statement. Combining context managers into new ones Examples Context managers for SQLAlchemy sessions Context managers for exception handling Persistent parameters across http requests Michael #5: nbstripout via Clément Robert In the latest episode, you praised NBDev for having a git hook that strips out notebook outputs. strip output from Jupyter and IPython notebooks Opens a notebook, strips its output, and writes the outputless version to the original file. Useful mainly as a git filter or pre-commit hook for users who don’t want to track output in VCS. This does mostly the same thing as the Clear All Output command in the notebook UI. Has a nice youtube tutorial right in the pypi listing Just do nbstripout --``install in a git repo! Brian #6: Write ups for The 2020 Python Language Summit Guido talked about this in episode 179 But these write-ups are excellent and really interesting. Should All Strings Become f-strings?, Eric V. Smith Replacing CPython’s Parser with a PEG-based parser, Pablo Galindo, Lysandros Nikolaou, Guido van Rossum A Formal Specification for the (C)Python Virtual Machine, Mark Shannon HPy: a Future-Proof Way of Extending Python?, Antonio Cuni CPython Documentation: The Next 5 Years, Carol Willing, Ned Batchelder Lightning talks (pre-selected) What do you need from pip, PyPI, and packaging?, Sumana Harihareswara A Retrospective on My "Multi-Core Python" Project, Eric Snow The Path Forward for Typing, Guido van Rossum Property-Based Testing for Python Builtins and the Standard Library, Zac Hatfield-Dodds Core Workflow Updates, Mariatta Wijaya CPython on Mobile Platforms, Russell Keith-Magee Wanted to bring this up because Python is a living language and it’s important to pay attention and get involved, or at least pay attention to where Python might be going. Also, another way to get involved is to become a member of the PSF board of directors What’s a PSF board of directors member do? video There are some open seats, Nominations are open until May 31 Extras: Michael: Updated search engine for better result ranking Windel Bouwman wrote a nice little script for speedscope https://github.com/windelbouwman/pyspeedscope (follow up from Austin profiler) Jokes: “Due to social distancing, I wonder how many projects are migrating to UDP and away from TLS to avoid all the handshakes?” - From Sviatoslav Sydorenko “A chef and a vagrant walk into a bar. Within a few seconds, it was identical to the last bar they went to.” - From Benjamin Jones, crediting @lufcraft Understanding both of these jokes is left as an exercise for the reader.

Sponsored by Datadog: pythonbytes.fm/datadog Brian #1: D-Tale suggested by @davidouglasmit via twitter “D-Tale is the combination of a Flask back-end and a React front-end to bring you an easy way to view & analyze Pandas data structures. It integrates seamlessly with ipython notebooks & python/ipython terminals. Currently this tool supports such Pandas objects as DataFrame, Series, MultiIndex, DatetimeIndex & RangeIndex.” way cool UI for visualizing data Live Demo shows Describe shows column statistics, graph, and top 100 values filter, correlations, charts, heat map Michael #2: Carnets by Nicolas Holzschuch A standalone Jupyter notebooks implementation for iOS. The power of Jupyter notebooks. In your pocket. Anywhere. Everything runs on your device. No need to setup a server, no need for an internet connection. Standard packages like Numpy, Matplotlib, Sympy and Pandas are already installed. You're ready to edit notebooks. Carnets uses iOS 11 filesharing ability. You can store your notebooks in iCloud, access them using other apps, share them. Extended keyboard on iPads, you get an extended toolbar with basic actions on your keyboard. Install more packages: Add more Python packages with %pip (if they are pure Python). OpenSource: Carnets is entirely OpenSource, and released under the FreeBSD license. Brian #3: BeeWare Podium suggested by Katie McLaughlin, @glasnt on twitter NOT a pip install, download a binary from https://github.com/beeware/podium/releases Linux and macOS Still early, so you gotta do the open and trust from the apps directory thing for running stuff not from the app store. But Oh man is it worth it. HTML5 based presentation frameworks are cool. run a presentation right in your browser. My favorite has been remark.js presenter mode, notes are especially useful while practicing a talk running timer super helpful while giving a talk write talk in markdown, so it’s super easy to version control issues: presenter mode, full screen, with extended monitor hard to do. notes and timer on laptop, full presentation on extended screen super cool but requires full screening with mouse Podium uses similar syntax as remark.js and I think uses remark under the hood. but it’s a native app, not a browser Handles the presenter mode and extended screen smoothly, like keynote and others. Removes the need for boilerplate html in your markdown file (remark.js md files have cruft). Can’t wait to try this out for my next presentation Michael #4: pytest-mock-resources via Daniel Cardin pytest fixture factories to make it easier to test against code that depends on external resources like Postgres, Redshift, and MongoDB. Code which depends on external resources such a databases (postgres, redshift, etc) can be difficult to write automated tests for. Conventional wisdom might be to mock or stub out the actual database calls and assert that the code works correctly before/after the calls. Whether the actual query did the correct thing truly requires that you execute the query. Having tests depend upon a real postgres instance running somewhere is a pain, very fragile, and prone to issues across machines and test failures. Therefore pytest-mock-resources (primarily) works by managing the lifecycle of docker containers and providing access to them inside your tests. Brian #5: How James Bennet is testing in 2020 Follow up from Testing Django applications in 2018 Favors unittest over pytest. tox for testing over multiple Django and Python versions, including tox-travis plugin pyenv for local Python installation management and pyenv-virtualenv plugin for venvs. Custom runtests.py for setting up environment and running tests. Changed to src/ directory layout. Coverage and reporting failure if coverage dips, with a healthy perspective: “… this isn’t because I have 100% coverage as a goal. Achieving that is so easy in most projects that it’s meaningless as a way to measure quality. Instead, I use the coverage report as a canary. It’s a thing that shouldn’t change, and if it ever does change I want to know, because it will almost always mean something else has gone wrong, and the coverage report will give me some pointers for where to look as I start investigating.” Testing is more than tests, it’s also black, isort, flake8, mypy, and even spell checking sphinx documentation. Using tox.ini for utility scripts, like cleanup, pipupgrade, … Michael #6: Python and PyQt: Building a GUI Desktop Calculator by by Leodanis Pozo Ramos at realpython Some interesting take-aways: Basics of PyQt Widgets: QWidget is the base class for all user interface objects, or widgets. These are rectangular-shaped graphical components that you can place on your application’s windows to build the GUI. Layout Managers: Layout managers are classes that allow you to size and position your widgets at the places you want them to be on the application’s form. Main Windows: Most of the time, your GUI applications will be Main Window-Style. This means that they’ll have a menu bar, some toolbars, a status bar, and a central widget that will be the GUI’s main element. Applications: The most basic class you’ll use when developing PyQt GUI applications is QApplication. This class is at the core of any PyQt application. It manages the application’s control flow as well as its main settings. Signals and Slots: PyQt widgets act as event-catchers. Widgets always emit a signal, which is a kind of message that announces a change in its state. Due to Qt licensing, you can only use the free version for non-commercial projects or internal non-redistributed or purchase a commercial license for $5,500/yr/dev. Extras Brian PyCascades 2020 livestream videos of day 1 & day 2 are available. Huge shout-out and thank you to all of the volunteers for this event. In particular Nina Zakharenko for calming me down before my talk. Michael Recording for Python for .NET devs webcast available. Take some of our free courses with our mobile app. Joke Why do programmers confuse Halloween with Christmas? Because OCT 31 == DEC 25. Speed dating is useless. 5 minutes is not enough to properly explain the benefits of the Unix philosophy.

Quail Data #0007 - Stats Wars Rodolfo #1: MOSP MONARC Objects Sharing Platform (MOSP) es una plataforma para crear, editar y compartir objetos JSON validados de cualquier tipo. MONARC - Method for an Optimised aNAlysis of Risks by CASES (Método para un análisis optimizado de riesgos por CASOS.) Puede usar cualquier esquema JSON disponible para crear nuevos objetos JSON a través de un formulario web generado dinámicamente y basado en el esquema seleccionado. Sergio #2: Scikit Geometry "scikit-geometry también viene con funciones para calcular el diagrama de Voronoi, el casco convexo, cuadros delimitadores, la suma minkowski de dos polígonos, un árbol AABB para consultas vecinas más cercanas y muchas otras utilidades útiles para cálculos geométricos, con planes para agregar muchos más!" Rodolfo #3: pandapy Demos un momento para tomar en cuenta el siguiente meme: https://www.reddit.com/r/mathmemes/comments/ewct2v/euler_moment/ Ahora, ¿recuerdan, por una parte a Pandas? Y por otra parte, ¿a NumPy? Pues bueno, pueden pensar en este paquete como un hijo de ambos. PandaPy tiene la velocidad de NumPy y la usabilidad de Pandas (10x a 50x más rápido). Así como importas pandas como pd y numpy como np, el común es importar a pandapy como pp (ya sabes → pd & np = pp). Sergio #4: Como hacer tu propio blog sin ser un experto en computadoras con fast.ai y fast_template Una guía muy fácil de seguir para crear tu propio blog hosteado en GitHub pages sin tener que usar la linea de comando. Es muy practico y facil de seguir y ahora utiliza GitHub Actions para transformar tus notebooks de jupyter a blog posts Rodolfo #5: Construyendo un Python Data Science Container usando Docker Es un blog post que ilustra cómo crear un contenedor de Docker que incluya paquetería como NumPy, SciPy, Pandas, SciKit-Learn, Matplotlib y NLTK. Todo se realiza a través de la construcción de un Dockerfile basado en Alpine, una versión muuuy ligera de Linux. El post te da todos los comandos para levantar el contenedor. Sergio #6: Blog de Juvenal Campos - Como Visualizar Pirámides de Población en R Un paso a paso de como construir una piramide de poblacion con ggplot2 Juvenal usa blogdown de R para este blog - todxs deberiamos bloguear mas! Extras: Sergio: Lorem Ipsum pero mexicano ? jajaja https://ignaciochavez.com/projects/lorempaisum/ RStudioConf está aquí en San Francisco esta semana y tienen los materiales de sus talleres en GitHub pa quién no pudo asistir: https://github.com/rstudio-conf-2020 Rodo: Para la gente Pythonista que nos escucha, ¡ya hay fecha para el PyCon Latam 2020! 27-29 de agosto, Pto. Vallarta, Jalisco. ¡No se lo pueden perder! (https://twitter.com/PyLatam/status/1221886633210982402) Meme de la semana --- This episode is sponsored by · Anchor: The easiest way to make a podcast. https://anchor.fm/app --- Send in a voice message: https://anchor.fm/quaildata/message Support this podcast: https://anchor.fm/quaildata/support

Sponsored by Datadog: pythonbytes.fm/datadog Michael #1: Data driven journalism via cjworkbench via Michael Paholski The data journalism platform with built in training Think spreadsheet + ETL automation Designed around modular tools for data processing -- table in, table out -- with no code required Features include: Modules to scrape, clean, analyze and visualize data An integrated data journalism training program Connect to Google Drive, Twitter, and API endpoints. Every action is recorded, so all workflows are repeatable and transparent All data is live and versioned, and you can monitor for changes. Write custom modules in Python and add them to the module library Brian #2: remi: A Platform-independent Python GUI library for your applications. Python REMote Interface library. “Remi is a GUI library for Python applications which transpiles an application's interface into HTML to be rendered in a web browser. This removes platform-specific dependencies and lets you easily develop cross-platform applications in Python!” No dependencies. pip install git+https://github.com/dddomodossola/remi.git doesn’t install anything else. Yes. Another GUI in a web page, but for quick and dirty internal tools, this will be very usable. Basic app: import remi.gui as gui from remi import start, App class MyApp(App): def __init__(self, *args): super(MyApp, self).__init__(*args) def main(self): container = gui.VBox(width=120, height=100) self.lbl = gui.Label('Hello world!') self.bt = gui.Button('Press me!') self.bt.onclick.do(self.on_button_pressed) container.append(self.lbl) container.append(self.bt) return container def on_button_pressed(self, widget): self.lbl.set_text('Button pressed!') self.bt.set_text('Hi!') start(MyApp) Michael #3: Typer Build great CLIs. Easy to code. Based on Python type hints. Typer is FastAPI's little sibling. And it's intended to be the FastAPI of CLIs. Just declare once the types of parameters (arguments and options) as function parameters. You do that with standard modern Python types. You don't have to learn a new syntax, the methods or classes of a specific library, etc. Based on Click Example (min version) import typer def main(name: str): typer.echo(f"Hello {name}") if __name__ == "__main__": typer.run(main) Brian #4: Effectively using Matplotlib Chris Moffitt “… I think I was a little premature in dismissing matplotlib. To be honest, I did not quite understand it and how to use it effectively in my workflow.” That very much sums up my relationship with matplotlib. But I’m ready to take another serious look at it. one reason for complexity is 2 interfaces MATLAB like state-based interface object based interface (use this) recommendations: Learn the basic matplotlib terminology, specifically what is a Figure and an Axes . Always use the object-oriented interface. Get in the habit of using it from the start of your analysis. Start your visualizations with basic pandas plotting. Use seaborn for the more complex statistical visualizations. Use matplotlib to customize the pandas or seaborn visualization. Runs through an example Describes figures and plots Includes a handy reference for customizing a plot. Related: StackOverflow answer that shows how to generate and embed a matplotlib image into a flask app without saving it to a file. Style it with pylustrator.readthedocs.io :) Michael #5: Django Simple Task django-simple-task runs background tasks in Django 3 without requiring other services and workers. It runs them in the same event loop as your ASGI application. Here’s a simple overview of how it works: On application start, a queue is created and a number of workers starts to listen to the queue When defer is called, a task(function or coroutine function) is added to the queue When a worker gets a task, it runs it or delegates it to a threadpool On application shutdown, it waits for tasks to finish before exiting ASGI server It is required to run Django with ASGI server. Example from django_simple_task import defer def task1(): time.sleep(1) print("task1 done") async def task2(): await asyncio.sleep(1) print("task2 done") def view(requests): defer(task1) defer(task2) return HttpResponse(b"My View") Brian #6: PyPI Stats at pypistats.org Simple interface. Pop in a package name and get the download stats. Example use: Why is my open source project now getting PRs and issues? I’ve got a few packages on PyPI, not updated much. cards and submark are mostly for demo purposes for teaching testing. pytest-check is a pytest plugin that allows multiple failures per test. I only hear about issues and PRs on one of these. So let’s look at traffic. cards: downloads day: 2 week: 24 month: 339 submark: day: 5 week: 9 month: 61 pytest-check: day: 976 week: 4,524 month: 19,636 That totally explains why I need to start actually supporting pytest-check. Cool. Note: it’s still small. Top 20 packages are all downloaded over 1.3 million times per day. Extras: Comment from January Python PDX West meetup “Please remember to have one beginner friendly talk per meetup.” Good point. Even if you can’t present here in Portland / Hillsboro, or don’t want to, I’d love to hear feedback of good beginner friendly topics that are good for meetups. PyCascades 2020 discount code listeners-at-pycascades for 10% off FireFox 72 is out with anti-fingerprinting and PIP - Ars Technica Joke: Language essays comic

In the third episode of Deep Neural Notebooks, I talk with Michael Droettboom, from Mozilla. Michael is a Staff Data Engineer at Mozilla, where he works on managing the telemetry from Mozilla products, that can be used for improving the user experience and the product itself, while respecting data privacy and making sure that instead of snarfing it all up, only the absolutely minimal data is gathered from the user. He specialises in imaging and data: sheet music, scientific visualization, astronomy, biomedical data and software telemetry. He started his Open Source journey around 2007, contributing to Matplotlib, while he was a Senior Computer Scientist at the Space Telescope Science Institute. He later went on to become a lead developer of Matplotlib, carrying on the vision of Matplotlib’s original author, John Hunter. In this episode, we talk about his beginnings - his education in Computers’ and Music, his experience in Astronomy, working at the Space Telescope Science Institute and how he got into contributing to Open Source for Matplotlib. We also talk about his role at Mozilla, the importance of data privacy and the amazing project that he is currently working on - Pyodide, a tool to empower Data Science in the Browser. Michael also shares some advice for beginners trying to get into Data Science and communities just starting out with Open Source software. Links: Michael Droettboom: https://twitter.com/MDroettboom , http://droettboom.com/ GLEAN: https://github.com/mozilla/glean Pyodide: https://github.com/iodide-project/pyodide Deep Neural Notebooks: Deep Neural Notebooks is a podcast where I like to discuss a multitude of topics, ranging from Deep Learning and Computer Vision to Neuroscience and Open Source Software, through conversations with experts about their thoughts on the state of their specialisations, how things fit into the bigger picture, their journey so far and the road ahead. I believe that it is through conversations like these that we can boil down the essence of vast resources of knowledge and expertise into more consumable bits that can enrich our understanding of concepts and technologies that are shaping our world. If you like the content, please subscribe to the channel and leave a thumbs up, or a 5-star rating, depending on the streaming platform. Youtube: https://www.youtube.com/channel/UC66w1T4oMv66Jn1LR5CW2yg Apple Podcasts: https://podcasts.apple.com/us/podcast/deep-neural-notebooks/id1488705711?uo=4 Spotify: https://open.spotify.com/show/2eq1jD7V5K19aZUUJnIz5z Google Podcasts: https://www.google.com/podcasts?feed=aHR0cHM6Ly9hbmNob3IuZm0vcy8xMDZkYzIzOC9wb2RjYXN0L3Jzcw== Anchor: https://anchor.fm/deep-neural-notebooks Connect: Website: https://mukulkhanna.github.io LinkedIn: https://linkedin.com/in/mukulkhanna/ Twitter: https://twitter.com/mkulkhanna

Matplotlib is a Python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. Matplotlib can be used in Python scripts, the Python and IPython shells, the Jupyter notebook, web application servers, and four graphical user interface toolkits.

In this podcast episode, you will come to know about how much data visualization is important with a real-life incident of happened with me. We will also discuss types of data visualization and some of the important libraries to implement them. --- This episode is sponsored by · Anchor: The easiest way to make a podcast. https://anchor.fm/app --- Send in a voice message: https://anchor.fm/aihindishow/message Support this podcast: https://anchor.fm/aihindishow/support

Panel provides tools for easily composing widgets, plots, tables, and other viewable objects and controls into control panels, apps, and dashboards. Panel works with visualizations from Bokeh, Matplotlib, HoloViews, and other Python plotting libraries, making them instantly viewable either individually or when combined with interactive widgets that control them. Panel works equally well in Jupyter Notebooks, for creating quick data-exploration tools, or as standalone deployed apps and dashboards, and allows you to easily switch between those contexts as needed.

GUEST BIO: My guest on today’s show is Matt Harrison.  Matt is an instructor on Python and Data Science material.  He has been co-chair of the Utah Python user group and has presented at conferences including PyCon, OSCon and OpenWest.  Matt is also an author of a number of books about Python, including the Treading on Python Series.   EPISODE DESCRIPTION: Matt Harrison is Phil’s guest on today’s show. He is a Python and Data Science Consultant who offers customized training to corporations and startups as well as consulting services through his company MetaSnake. Over the years, he has worked with a range of languages and platforms, including Pandas, Pylons, Django, CherryPy, Postgres, AWS, SQLAlchemy, SciKit Learn, and Matplotlib. Matt was the co-chair of the Utah Python user group as well as an author and public speaker. In 2012, he published his first Python book Treading on Python Volume 1: Foundations of Python. Since then, he has published several other books and is currently busy writing more. KEY TAKEAWAYS: (1.02) – So Matt, can I ask you to expand on that brief intro and tell us a little bit more about yourself? Matt explains that he runs a small consulting and training company called MetaSnake. He works with businesses big and small and spends half his time speaking to very technical people teaching them Python and data science. Right now, he is working on three more books. (2.09) – Python is quite a theme, in your career, what made you choose that, in particular? The first language Matt learned was Perl. He used it for his first summer job, so became very comfortable with it.   Matt was lucky enough to work with a really smart guy. At the time, he was building on a model to pull out relevant terms for their corpus of text. The guy he was working with wanted to use Tcl to get the task done and Matt thought Perl would do a better job. So, they agreed to meet in the middle and use Python instead. At that time, it was a relatively new language. Within 3 days they had the proof of concept working. Python just gelled with him, unlike, Perl, C and Java, which he had mostly been using up to that point. (3.42) – Phil asks Matt to tell the audience more about his books, which Phil understands are mainly about Python. Matt explains that his book “Illustrated guide to Python 3” is for beginners. He has also written an intermediate book, one that covers the Pandas library and a few others. All of which are available on Amazon. (4.14) – Can you please share a unique career tip with the I.T. career audience? Matt says his advice is to – get comfortable with being uncomfortable. When he was just starting out, his local Python meetup took place a couple of miles away from where he lived. But, he didn’t attend partly because it would take him outside of his comfort zone. A decision he regrets to this day. At the time, it was a very tight-knit group. Not attending meant that he ended up missing out on a lot of important interactions. Plus, he eventually ended up meeting everyone anyway. At the time he did not understand the power of interacting with others. If you want to further your career, you need to get away from your computer sometimes and mix with other people. (6.00) – Can you tell us about your worst career moment? And what you learned from that experience. For Matt that was when he started his small vertical niche software company. A lot of firms let them liked the software. But very few of them decided to buy. In a surprising number of cases, this was because using the software would mean they would have to fire someone. A lot of the firms were small and employed family members, so they really did not want a piece of software to replace them. When Matt heard this he just said OK and moved on instead of trying another sales tactic. He knew he had a good product, but his lack of sales skills meant he could not close the deal. (8.20) – What was your best career moment? For Matt, that was when he wrote his first book in 2010. He wanted to reach more people with his training and realized he could do that as an author. His self-published book was a great success and opened a lot of doors for him. (10.00) – Phil asks if the process of writing books gets easier. Matt says yes, to a certain extent it does. Things do move faster once you have created a process that works for you. He now finds it easier to start typing and get in the flow. (10.59) – Can you tell us what excites you about the future of the IT industry and careers? The fact that a lot of companies are now waking up to the power of their data is exciting. There is a lot of low hanging fruit. As a result, you can make a huge difference and do so very quickly. (12.14) – What first attracted you to a career in IT? Matt always enjoyed creating things. When he was younger, he attended a lot of art classes and thought he might end up being an artist. But, a family member encouraged him to take a programming course. Once he did, he realized that there is a lot of creativity involved in most areas of technology, including in the data science field. (13.23) – What is the best career advice you have ever received? Matt says that he actually ignored the best piece of career advice he got. In 2004, he was advised to get a job at Google, but he didn’t. Fortunately, he did follow another great piece of advice, when he was told you need to network, he started doing exactly that. It is hard for a lot of programmers to network, because, by nature, many of them are introverts. In the IT world, you usually get good jobs, not because you interviewed well, but, because someone in the company knows you. So, learning to network is something everyone has to do. (14.33) – Phil points out that effective networking does not always have to happen face to face. There are plenty of online tools you can also use. Matt says that personally, he prefers face to face networking. However, he has seen people using Twitter and LinkedIn for job hunting. In fact, he tried it once and was contacted by about 40 people, so it can work. (15.43) – If you were to begin your IT career again, right now, what would you do? Matt is intrigued by the idea of taking the full-time, online immersive Computer Science courses offered by the Lambda School. They do not charge for their courses, at least not at first. Instead, you pay for your tuition once you get a job. Although, he goes on to say that he enjoyed his college experience. So, if he were to start again, he would probably still opt to attend Stanford and get a degree. His advice to someone who is just starting out would be to go and get a job at Google or somewhere similar. Work for a big company for a few years. Build up your connections and involve yourself in the community. Having a firm like Google on your resume is still something of a golden ticket. (18.40) – What are you currently focusing on in your career? For this year, Matt’s focus is growing MetaSnake. He really enjoys spending three or four days with a group of people who are motivated and excited to learn, so wants more clients. (19.53) – What is the number one non-technical skill that has helped you the most in your IT career? Matt likes to think of himself as creative. Fortunately, when it comes to machine learning and data exploration creativity is a great skill to have. (20.31) – Phil asks Matt to share a final piece of career advice with the I.T. Career Energizer audience. Matt’s advice is to look at where you are now. Then think about where you want to be a year and five years down the line. Ask yourself what you want to achieve. It could be publishing a book, speaking at conferences, or something else. Matt has found that desire and motivation have helped him to move his career forward. BEST MOMENTS: (4.30) MATT – "Get comfortable with being uncomfortable." (5.16) MATT – "The more I network and connect with people the more value I can bring to them and the more value they can bring to me." (11.46) MATT – "There is a lot of low hanging fruit that companies can take advantage of.” (13.51) MATT – "The best career advice I have seen is to network" (20.42) MATT – "Look hard at where you are and know where you want to be." CONTACT MATT: Twitter: https://twitter.com/__mharrison__ LinkedIn: https://www.linkedin.com/in/panela/ Website: https://www.metasnake.com/

Die sechste Folge beschäftigt sich mit einer der wohl bekanntesten und meistgenutzten Python-Bibliotheken: "Pandas" Diesmal haben wir als Expertengast Simon dabei, der uns mehr über die Funktionen von Pandas erzählt. Shownotes Unsere E-Mail für Fragen, Anregungen & Kommentare: hallo@python-podcast.de News Shared memory for multiprocessing (struct, wenn man das von Hand machen will) Operatoren für Dictionaries Pandas Pandas Cheatsheets Teil 1, Teil 2 Tutorialnotebook von Jochen Maßeinheiten für dataframes mit pint (noch nicht released) - verwendet die neue extension array api Erster Einblick in die Daten im Pandas Workflow mit df.head() df.tail() und df.describe() df.apply() Eher für Fortgeschrittene: Modern Pandas Artikel über pivot, stack und unstack Pandas 2.0 Podcasts und Talks Jeff Reback - What is the Future of Pandas Wes McKinney's Career In Python For Data Analysis - Episode 203 Episode #200: Escaping Excel Hell with Python and Pandas R R, R studio Shiny Picks Django Chat Podcast (Jochen) Django-ORM-like, aber für flat files:  alkali (Jochen) Matplotlib to Plotly (Simon) pickle (Dominik) Öffentlicher Tag auf konektom

En este capítulo presento qué son los Módulos, cómo se usan, los disponibles directamente con el lenguaje de programación Python, y una mención a las más destacables disponibles en la comunidad. . Pagina de información oficial: https://docs.python.org/3/library/index.html . Modulos de Python: TIME, DATETIME, RANDOM, MATH, STATISTICS, OS, OS.PATH, PATHLIB, SYS, SQLITE3, HASHLIB, CSV, GZIP, ZLIB, BZ2, LZMA, ZIPFILE, TARFILE, TKINTER,... . Módulos para Python: NumPy, SciPy, SymPy, BioPython, SQLAlchemi, Colorama, wxPython, PyQT, PyGTK, Kivy, Matplotlib, Seaborn, Bokeh, PyGame, PyGlet, Twisted, Scrapy, NLTK, Request, Pillow, Keras, Pytorch, Scikit-Learn, Pandas, Theano, TensorFlow,... . Aquí tenéis mi página web: https://unosycerospatxi.wordpress.com/ . UN SALUDO!!!!! Espero que os guste!!!

Si quieres ver el vídeo con slides: https://youtu.be/KyiH9xesHGk Like investors, animals make their best bets based on current conditions and their own knowledge. They dynamically combine information sources to reduce risk and maximize profits. Thanks to the most recent improvements on artificial intelligence tools, massive amounts of data can be now analysed to adapt ourselves to the future behaviour of animals, the climate, shoppers or the financial market. These current developments allow us to train computers using Python, Pandas and other pets to mimic animal survival techniques and to make the best of our investing strategies. Compiling the right set of tools is crucial for successful data scientists. My Swiss knife to survive in the wild financial market has: Python, Pandas, Matplotlib, Sklearn and a bit of Keras. In this talk I will show you some tips and tricks to become a more complete data scientist with my favourite animals.

We try to answer what happens to an open source project after a developers death, we tell you about the last bootstrapped tech company in Silicon Valley, we have an update to the NetBSD Thread sanitizer, and show how to use use cabal on OpenBSD This episode was brought to you by Headlines Life after death, for code (https://www.wired.com/story/giving-open-source-projects-life-after-a-developers-death/) YOU'VE PROBABLY NEVER heard of the late Jim Weirich or his software. But you've almost certainly used apps built on his work. Weirich helped create several key tools for Ruby, the popular programming language used to write the code for sites like Hulu, Kickstarter, Twitter, and countless others. His code was open source, meaning that anyone could use it and modify it. "He was a seminal member of the western world's Ruby community," says Justin Searls, a Ruby developer and co-founder of the software company Test Double. When Weirich died in 2014, Searls noticed that no one was maintaining one of Weirich's software-testing tools. That meant there would be no one to approve changes if other developers submitted bug fixes, security patches, or other improvements. Any tests that relied on the tool would eventually fail, as the code became outdated and incompatible with newer tech. The incident highlights a growing concern in the open-source software community. What happens to code after programmers pass away? Much has been written about what happens to social-media accounts after users die. But it's been less of an issue among programmers. In part, that's because most companies and governments relied on commercial software maintained by teams of people. But today, more programs rely on obscure but crucial software like Weirich's. Some open-source projects are well known, such as the Linux operating system or Google's artificial-intelligence framework TensorFlow. But each of these projects depend on smaller libraries of open-source code. And those libraries depend on other libraries. The result is a complex, but largely hidden, web of software dependencies. That can create big problems, as in 2014 when a security vulnerability known as "Heartbleed" was found in OpenSSL, an open-source program used by nearly every website that processes credit- or debit-card payments. The software comes bundled with most versions of Linux, but was maintained by a small team of volunteers who didn't have the time or resources to do extensive security audits. Shortly after the Heartbleed fiasco, a security issue was discovered in another common open-source application called Bash that left countless web servers and other devices vulnerable to attack. There are surely more undiscovered vulnerabilities. Libraries.io, a group that analyzes connections between software projects, has identified more than 2,400 open-source libraries that are used in at least 1,000 other programs but have received little attention from the open-source community. Security problems are only one part of the issue. If software libraries aren't kept up to date, they may stop working with newer software. That means an application that depends on an outdated library may not work after a user updates other software. When a developer dies or abandons a project, everyone who depends on that software can be affected. Last year when programmer Azer Koçulu deleted a tiny library called Leftpad from the internet, it created ripple effects that reportedly caused headaches at Facebook, Netflix, and elsewhere. The Bus Factor The fewer people with ownership of a piece of software, the greater the risk that it could be orphaned. Developers even have a morbid name for this: the bus factor, meaning the number of people who would have to be hit by a bus before there's no one left to maintain the project. Libraries.io has identified about 3,000 open-source libraries that are used in many other programs but have only a handful of contributors. Orphaned projects are a risk of using open-source software, though commercial software makers can leave users in a similar bind when they stop supporting or updating older programs. In some cases, motivated programmers adopt orphaned open-source code. That's what Searls did with one of Weirich's projects. Weirich's most-popular projects had co-managers by the time of his death. But Searls noticed one, the testing tool Rspec-Given, hadn't been handed off, and wanted to take responsibility for updating it. But he ran into a few snags along the way. Rspec-Given's code was hosted on the popular code-hosting and collaboration site GitHub, home to 67 million codebases. Weirich's Rspec-Given page on GitHub was the main place for people to report bugs or to volunteer to help improve the code. But GitHub wouldn't give Searls control of the page, because Weirich had not named him before he died. So Searls had to create a new copy of the code, and host it elsewhere. He also had to convince the operators of Ruby Gems, a “package-management system” for distributing code, to use his version of Rspec-Given, instead of Weirich's, so that all users would have access to Searls' changes. GitHub declined to discuss its policies around transferring control of projects. That solved potential problems related to Rspec-Given, but it opened Searls' eyes to the many things that could go wrong. “It's easy to see open source as a purely technical phenomenon,” Searls says. “But once something takes off and is depended on by hundreds of other people, it becomes a social phenomenon as well.” The maintainers of most package-management systems have at least an ad-hoc process for transferring control over a library, but that process usually depends on someone noticing that a project has been orphaned and then volunteering to adopt it. "We don't have an official policy mostly because it hasn't come up all that often," says Evan Phoenix of the Ruby Gems project. "We do have an adviser council that is used to decide these types of things case by case." Some package managers now monitor their libraries and flag widely used projects that haven't been updated in a long time. Neil Bowers, who helps maintain a package manager for the programming language Perl, says he sometimes seeks out volunteers to take over orphan projects. Bowers says his group vets claims that a project has been abandoned, and the people proposing to take it over. A 'Dead-Man's Switch' Taking over Rspec-Given inspired Searls, who was only 30 at the time, to make a will and a succession plan for his own open-source projects. There are other things developers can do to help future-proof their work. They can, for example, transfer the copyrights to a foundation, such as the Apache Foundation. But many open-source projects essentially start as hobbies, so programmers may not think to transfer ownership until it is too late. Searls suggests that GitHub and package managers such as Gems could add something like a "dead man's switch" to their platform, which would allow programmers to automatically transfer ownership of a project or an account to someone else if the creator doesn't log in or make changes after a set period of time. But a transition plan means more than just giving people access to the code. Michael Droettboom, who took over a popular mathematics library called Matplotlib after its creator John Hunter died in 2012, points out that successors also need to understand the code. "Sometimes there are parts of the code that only one person understands," he says. "The knowledge exists only in one person's head." That means getting people involved in a project earlier, ideally as soon as it is used by people other than the original developer. That has another advantage, Searls points out, in distributing the work of maintaining a project to help prevent developer burnout. The Last Bootstrapped Tech Company In Silicon Valley (https://www.forbes.com/sites/forbestechcouncil/2017/12/12/the-last-bootstrapped-tech-company-in-silicon-valley/2/#4d53d50f1e4d) My business partner, Matt Olander, and I were intimately familiar with the ups and downs of the Silicon Valley tech industry when we acquired the remnants of our then-employer BSDi's enterprise computer business in 2002 and assumed the roles of CEO and CTO. Fast-forward to today, and we still work in the same buildings where BSDi started in 1996, though you'd hardly recognize them today. As the business grew from a startup to a global brand, our success came from always ensuring we ran a profitable business. While that may sound obvious, keep in mind that we are in the heart of Silicon Valley where venture capitalists hunt for the unicorn company that will skyrocket to a billion-dollar valuation. Unicorns like Facebook and Twitter unquestionably exist, but they are the exception. Live By The VC, Die By The VC After careful consideration, Matt and I decided to bootstrap our company rather than seek funding. The first dot-com bubble had recently burst, and we were seeing close friends lose their jobs right and left at VC-funded companies based on dubious business plans. While we did not have much cash on hand, we did have a customer base and treasured those customers as our greatest asset. We concluded that meeting their needs was the surest path to meeting ours, and the rest would simply be details to address individually. This strategy ended up working so well that we have many of the same customers to this day. After deciding to bootstrap, we made a decision on a matter that has left egg on the face of many of our competitors: We seated sales next to support under one roof at our manufacturing facility in Silicon Valley. Dell's decision to outsource some of its support overseas in the early 2000s was the greatest gift it could have given us. Some of our sales and support staff have worked with the same clients for over a decade, and we concluded that no amount of funding could buy that mutual loyalty. While accepting venture capital or an acquisition may make you rich, it does not guarantee that your customers, employees or even business will be taken care of. Our motto is, “Treat your customers like friends and employees like family,” and we have an incredibly low employee turnover to show for it. Thanks to these principles, iXsystems has remained employee-owned, debt-free and profitable from the day we took it over -- all without VC funding, which is why we call ourselves the "last bootstrapped tech company in Silicon Valley." As a result, we now provide enterprise servers to thousands of customers, including top Fortune 500 companies, research and educational institutions, all branches of the military, and numerous government entities. Over time, however, we realized that we were selling more and more third-party data storage systems with every order. We saw this as a new opportunity. We had partnered with several storage vendors to meet our customers' needs, but every time we did, we opened a can of worms with regard to supporting our customers to our standards. Given a choice of risking being dragged down by our partners or outmaneuvered by competitors with their own storage portfolios, we made a conscious decision to develop a line of storage products that would not only complement our enterprise servers but tightly integrate with them. To accelerate this effort, we adopted the FreeNAS open-source software-defined storage project in 2009 and haven't looked back. The move enabled us to focus on storage, fully leveraging our experience with enterprise hardware and our open source heritage in equal measures. We saw many storage startups appear every quarter, struggling to establish their niche in a sea of competitors. We wondered how they'd instantly master hardware to avoid the partnering mistakes that we made years ago, given that storage hardware and software are truly inseparable at the enterprise level. We entered the storage market with the required hardware expertise, capacity and, most importantly, revenue, allowing us to develop our storage line at our own pace. Grow Up, But On Your Own Terms By not having the external pressure from VCs or shareholders that your competitors have, you're free to set your own priorities and charge fair prices for your products. Our customers consistently tell us how refreshing our sales and marketing approaches are. We consider honesty, transparency and responsible marketing the only viable strategy when you're bootstrapped. Your reputation with your customers and vendors should mean everything to you, and we can honestly say that the loyalty we have developed is priceless. So how can your startup venture down a similar path? Here's our advice for playing the long game: Relate your experiences to each fad: Our industry is a firehose of fads and buzzwords, and it can be difficult to distinguish the genuine trends from the flops. Analyze every new buzzword in terms of your own products, services and experiences, and monitor customer trends even more carefully. Some buzzwords will even formalize things you have been doing for years. Value personal relationships: Companies come and go, but you will maintain many clients and colleagues for decades, regardless of the hat they currently wear. Encourage relationship building at every level of your company because you may encounter someone again. Trust your instincts and your colleagues: No contractual terms or credit rating system can beat the instincts you will develop over time for judging the ability of individuals and companies to deliver. You know your business, employees and customers best. Looking back, I don't think I'd change a thing. We need to be in Silicon Valley for the prime customers, vendors and talent, and it's a point of pride that our customers recognize how different we are from the norm. Free of a venture capital “runway” and driven by these principles, we look forward to the next 20 years in this highly-competitive industry. Creating an AS for fun and profit (http://blog.thelifeofkenneth.com/2017/11/creating-autonomous-system-for-fun-and.html) At its core, the Internet is an interconnected fabric of separate networks. Each network which makes up the Internet is operated independently and only interconnects with other networks in clearly defined places. For smaller networks like your home, the interaction between your network and the rest of the Internet is usually pretty simple: you buy an Internet service plan from an ISP (Internet Service Provider), they give you some kind of hand-off through something like a DSL or cable modem, and give you access to "the entire Internet". Your router (which is likely also a WiFi access point and Ethernet switch) then only needs to know about two things; your local computers and devices are on one side, and the ENTIRE Internet is on the other side of that network link given to you by your ISP. For most people, that's the extent of what's needed to be understood about how the Internet works. Pick the best ISP, buy a connection from them, and attach computers needing access to the Internet. And that's fine, as long as you're happy with only having one Internet connection from one vendor, who will lend you some arbitrary IP address(es) for the extend of your service agreement, but that starts not being good enough when you don't want to be beholden to a single ISP or a single connection for your connectivity to the Internet. That also isn't good enough if you are an Internet Service Provider so you are literally a part of the Internet. You can't assume that the entire Internet is that way when half of the Internet is actually in the other direction. This is when you really have to start thinking about the Internet and treating the Internet as a very large mesh of independent connected organizations instead of an abstract cloud icon on the edge of your local network map. Which is pretty much never for most of us. Almost no one needs to consider the Internet at this level. The long flight of steps from DSL for your apartment up to needing to be an integral part of the Internet means that pretty much regardless of what level of Internet service you need for your projects, you can probably pay someone else to provide it and don't need to sit down and learn how BGP works and what an Autonomous System is. But let's ignore that for one second, and talk about how to become your own ISP. To become your own Internet Service Provider with customers who pay you to access the Internet, or be your own web hosting provider with customers who pay you to be accessible from the Internet, or your own transit provider who has customers who pay you to move their customer's packets to other people's customers, you need a few things: Your own public IP address space allocated to you by an Internet numbering organization Your own Autonomous System Number (ASN) to identify your network as separate from everyone else's networks At least one router connected to a different autonomous system speaking the Border Gateway Protocol to tell the rest of the Internet that your address space is accessible from your autonomous system. So... I recently set up my own autonomous system... and I don't really have a fantastic justification for it... My motivation was twofold: One of my friends and I sat down and figured it out that splitting the cost of a rack in Hurricane Electric's FMT2 data center marginally lowered our monthly hosting expenses vs all the paid services we're using scattered across the Internet which can all be condensed into this one rack. And this first reason on its own is a perfectly valid justification for paying for co-location space at a data center like Hurricane Electric's, but isn't actually a valid reason for running it as an autonomous system, because Hurricane Electric will gladly let you use their address space for your servers hosted in their building. That's usually part of the deal when you pay for space in a data center: power, cooling, Internet connectivity, and your own IP addresses. Another one of my friends challenged me to do it as an Autonomous System. So admittedly, my justification for going through the additional trouble to set up this single rack of servers as an AS is a little more tenuous. I will readily admit that, more than anything else, this was a "hold my beer" sort of engineering moment, and not something that is at all needed to achieve what we actually needed (a rack to park all our servers in). But what the hell; I've figured out how to do it, so I figured it would make an entertaining blog post. So here's how I set up a multi-homed autonomous system on a shoe-string budget: Step 1. Found a Company Step 2. Get Yourself Public Address Space Step 3. Find Yourself Multiple Other Autonomous Systems to Peer With Step 4. Apply for an Autonomous System Number Step 5. Source a Router Capable of Handling the Entire Internet Routing Table Step 6. Turn it All On and Pray And we're off to the races. At this point, Hurricane Electric is feeding us all ~700k routes for the Internet, we're feeding them our two routes for our local IPv4 and IPv6 subnets, and all that's left to do is order all our cross-connects to other ASes in the building willing to peer with us (mostly for fun) and load in all our servers to build our own personal corner of the Internet. The only major goof so far has been accidentally feeding the full IPv6 table to our first other peer that we turned on, but thankfully he has a much more powerful supervisor than the Sup720-BXL, so he just sent me an email to knock that off, a little fiddling with my BGP egress policies, and we were all set. In the end, setting up my own autonomous system wasn't exactly simple, it was definitely not justified, but some times in life you just need to take the more difficult path. And there's a certain amount of pride in being able to claim that I'm part of the actual Internet. That's pretty neat. And of course, thanks to all of my friends who variously contributed parts, pieces, resources, and know-how to this on-going project. I had to pull in a lot of favors to pull this off, and I appreciate it. News Roundup One year checkpoint and Thread Sanitizer update (https://blog.netbsd.org/tnf/entry/one_year_checkpoint_and_thread) The past year has been started with bugfixes and the development of regression tests for ptrace(2) and related kernel features, as well as the continuation of bringing LLDB support and LLVM sanitizers (ASan + UBsan and partial TSan + Msan) to NetBSD. My plan for the next year is to finish implementing TSan and MSan support, followed by a long run of bug fixes for LLDB, ptrace(2), and other related kernel subsystems TSan In the past month, I've developed Thread Sanitizer far enough to have a subset of its tests pass on NetBSD, started with addressing breakage related to the memory layout of processes. The reason for this breakage was narrowed down to the current implementation of ASLR, which was too aggressive and which didn't allow enough space to be mapped for Shadow memory. The fix for this was to either force the disabling of ASLR per-process, or globally on the system. The same will certainly happen for MSan executables. After some other corrections, I got TSan to work for the first time ever on October 14th. This was a big achievement, so I've made a snapshot available. Getting the snapshot of execution under GDB was pure hazard. ``` $ gdb ./a.out GNU gdb (GDB) 7.12 Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64--netbsd". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./a.out...done. (gdb) r Starting program: /public/llvm-build/a.out [New LWP 2] WARNING: ThreadSanitizer: data race (pid=1621) Write of size 4 at 0x000001475d70 by thread T1: #0 Thread1 /public/llvm-build/tsan.c:4:10 (a.out+0x46bf71) Previous write of size 4 at 0x000001475d70 by main thread: #0 main /public/llvm-build/tsan.c:10:10 (a.out+0x46bfe6) Location is global 'Global' of size 4 at 0x000001475d70 (a.out+0x000001475d70) Thread T1 (tid=2, running) created by main thread at: #0 pthreadcreate /public/llvm/projects/compiler-rt/lib/tsan/rtl/tsaninterceptors.cc:930:3 (a.out+0x412120) #1 main /public/llvm-build/tsan.c:9:3 (a.out+0x46bfd1) SUMMARY: ThreadSanitizer: data race /public/llvm-build/tsan.c:4:10 in Thread1 Thread 2 received signal SIGSEGV, Segmentation fault. ``` I was able to get the above execution results around 10% of the time (being under a tracer had no positive effect on the frequency of successful executions). I've managed to hit the following final results for this month, with another set of bugfixes and improvements: check-tsan: Expected Passes : 248 Expected Failures : 1 Unsupported Tests : 83 Unexpected Failures: 44 At the end of the month, TSan can now reliably executabe the same (already-working) program every time. The majority of failures are in tests verifying sanitization of correct mutex locking usage. There are still problems with NetBSD-specific libc and libpthread bootstrap code that conflicts with TSan. Certain functions (pthreadcreate(3), pthreadkeycreate(3), _cxaatexit()) cannot be started early by TSan initialization, and must be deferred late enough for the sanitizer to work correctly. MSan I've prepared a scratch support for MSan on NetBSD to help in researching how far along it is. I've also cloned and adapted the existing FreeBSD bits; however, the code still needs more work and isn't functional yet. The number of passed tests (5) is negligible and most likely does not work at all. The conclusion after this research is that TSan shall be finished first, as it touches similar code. In the future, there will be likely another round of iterating the system structs and types and adding the missing ones for NetBSD. So far, this part has been done before executing the real MSan code. I've added one missing symbol that was missing and was detected when attempting to link a test program with MSan. Sanitizers The GCC team has merged the LLVM sanitizer code, which has resulted in almost-complete support for ASan and UBsan on NetBSD. It can be found in the latest GCC8 snapshot, located in pkgsrc-wip/gcc8snapshot. Though, do note that there is an issue with getting backtraces from libasan.so, which can be worked-around by backtracing ASan events in a debugger. UBsan also passes all GCC regression tests and appears to work fine. The code enabling sanitizers on the GCC/NetBSD frontend will be submitted upstream once the backtracing issue is fixed and I'm satisfied that there are no other problems. I've managed to upstream a large portion of generic+TSan+MSan code to compiler-rt and reduce local patches to only the ones that are in progress. This deals with any rebasing issues, and allows me to just focus on the delta that is being worked on. I've tried out the LLDB builds which have TSan/NetBSD enabled, and they built and started fine. However, there were some false positives related to the mutex locking/unlocking code. Plans for the next milestone The general goals are to finish TSan and MSan and switch back to LLDB debugging. I plan to verify the impact of the TSan bootstrap initialization on the observed crashes and research the remaining failures. This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: The scourge of systemd (https://blog.ungleich.ch/en-us/cms/blog/2017/12/10/the-importance-of-devuan/) While this article is actually couched in terms of promoting devuan, a de-systemd-ed version of debian, it would seem the same logic could be applied to all of the BSDs Let's say every car manufacturer recently discovered a new technology named "doord", which lets you open up car doors much faster than before. It only takes 0.05 seconds, instead of 1.2 seconds on average. So every time you open a door, you are much, much faster! Many of the manufacturers decide to implement doord, because the company providing doord makes it clear that it is beneficial for everyone. And additional to opening doors faster, it also standardises things. How to turn on your car? It is the same now everywhere, it is not necessarily to look for the keyhole anymore. Unfortunately though, sometimes doord does not stop the engine. Or if it is cold outside, it stops the ignition process, because it takes too long. Doord also changes the way your navigation system works, because that is totally related to opening doors, but leads to some users being unable to navigate, which is accepted as collateral damage. In the end, you at least have faster door opening and a standard way to turn on the car. Oh, and if you are in a traffic jam and have to restart the engine often, it will stop restarting it after several times, because that's not what you are supposed to do. You can open the engine hood and tune that setting though, but it will be reset once you buy a new car. Some of you might now ask themselves "Is systemd THAT bad?". And my answer to it is: No. It is even worse. Systemd developers split the community over a tiny detail that decreases stability significantly and increases complexity for not much real value. And this is not theoretical: We tried to build Data Center Light on Debian and Ubuntu, but servers that don't boot, that don't reboot or systemd-resolved that constantly interferes with our core network configuration made it too expensive to run Debian or Ubuntu. Yes, you read right: too expensive. While I am writing here in flowery words, the reason to use Devuan is hard calculated costs. We are a small team at ungleich and we simply don't have the time to fix problems caused by systemd on a daily basis. This is even without calculating the security risks that come with systemd. Using cabal on OpenBSD (https://deftly.net/posts/2017-10-12-using-cabal-on-openbsd.html) Since W^X became mandatory in OpenBSD (https://undeadly.org/cgi?action=article&sid=20160527203200), W^X'd binaries are only allowed to be executed from designated locations (mount points). If you used the auto partition layout during install, your /usr/local/ will be mounted with wxallowed. For example, here is the entry for my current machine: /dev/sd2g on /usr/local type ffs (local, nodev, wxallowed, softdep) This is a great feature, but if you build applications outside of the wxallowed partition, you are going to run into some issues, especially in the case of cabal (python as well). Here is an example of what you would see when attempting to do cabal install pandoc: qbit@slip[1]:~? cabal update Config file path source is default config file. Config file /home/qbit/.cabal/config not found. Writing default configuration to /home/qbit/.cabal/config Downloading the latest package list from hackage.haskell.org qbit@slip[0]:~? cabal install pandoc Resolving dependencies... ..... cabal: user error (Error: some packages failed to install: JuicyPixels-3.2.8.3 failed during the configure step. The exception was: /home/qbit/.cabal/setup-exe-cache/setup-Simple-Cabal-1.22.5.0-x86_64-openbsd-ghc-7.10.3: runProcess: runInteractiveProcess: exec: permission denied (Permission denied) The error isn't actually what it says. The untrained eye would assume permissions issue. A quick check of dmesg reveals what is really happening: /home/qbit/.cabal/setup-exe-cache/setup-Simple-Cabal-1.22.5.0-x86_64-openbsd-ghc-7.10.3(22924): W^X binary outside wxallowed mountpoint OpenBSD is killing the above binary because it is violating W^X and hasn't been safely kept in its /usr/local corral! We could solve this problem quickly by marking our /home as wxallowed, however, this would be heavy handed and reckless (we don't want to allow other potentially unsafe binaries to execute.. just the cabal stuff). Instead, we will build all our cabal stuff in /usr/local by using a symlink! doas mkdir -p /usr/local/{cabal,cabal/build} # make our cabal and build dirs doas chown -R user:wheel /usr/local/cabal # set perms rm -rf ~/.cabal # kill the old non-working cabal ln -s /usr/local/cabal ~/.cabal # link it! We are almost there! Some cabal packages build outside of ~/.cabal: cabal install hakyll ..... Building foundation-0.0.14... Preprocessing library foundation-0.0.14... hsc2hs: dist/build/Foundation/System/Bindings/Posix_hsc_make: runProcess: runInteractiveProcess: exec: permission denied (Permission denied) Downloading time-locale-compat-0.1.1.3... ..... Fortunately, all of the packages I have come across that do this all respect the TMPDIR environment variable! alias cabal='env TMPDIR=/usr/local/cabal/build/ cabal' With this alias, you should be able to cabal without issue (so far pandoc, shellcheck and hakyll have all built fine)! TL;DR # This assumes /usr/local/ is mounted as wxallowed. # doas mkdir -p /usr/local/{cabal,cabal/build} doas chown -R user:wheel /usr/local/cabal rm -rf ~/.cabal ln -s /usr/local/cabal ~/.cabal alias cabal='env TMPDIR=/usr/local/cabal/build/ cabal' cabal install pandoc FreeBSD and APRS, or "hm what happens when none of this is well documented.." (https://adrianchadd.blogspot.co.uk/2017/10/freebsd-and-aprs-or-hm-what-happens.html) Here's another point along my quest for amateur radio on FreeBSD - bring up basic APRS support. Yes, someone else has done the work, but in the normal open source way it was .. inconsistently documented. First is figuring out the hardware platform. I chose the following: A Baofeng UV5R2, since they're cheap, plentiful, and do both VHF and UHF; A cable to do sound level conversion and isolation (and yes, I really should post a circuit diagram and picture..); A USB sound device, primarily so I can whack it into FreeBSD/Linux devices to get a separate sound card for doing radio work; FreeBSD laptop (it'll become a raspberry pi + GPS + sensor + LCD thingy later, but this'll do to start with.) The Baofeng is easy - set it to the right frequency (VHF APRS sits on 144.390MHz), turn on VOX so I don't have to make up a PTT cable, done/done. The PTT bit isn't that hard - one of the microphone jack pins is actually PTT (if you ground it, it engages PTT) so when you make the cable just ensure you expose a ground pin and PTT pin so you can upgrade it later. The cable itself isn't that hard either - I had a baofeng handmic lying around (they're like $5) so I pulled it apart for the cable. I'll try to remember to take pictures of that. Here's a picture I found on the internet that shows the pinout: image (https://3.bp.blogspot.com/-58HUyt-9SUw/Wdz6uMauWlI/AAAAAAAAVz8/e7OrnRzN3908UYGUIRI1EBYJ5UcnO0qRgCLcBGAs/s1600/aprs-cable.png) Now, I went a bit further. I bought a bunch of 600 ohm isolation transformers for audio work, so I wired it up as follows: From the audio output of the USB sound card, I wired up a little attenuator - input is 2k to ground, then 10k to the input side of the transformer; then the output side of the transformer has a 0.01uF greencap capacitor to the microphone input of the baofeng; From the baofeng I just wired it up to the transformer, then the output side of that went into a 0.01uF greencap capacitor in series to the microphone input of the sound card. In both instances those capacitors are there as DC blockers. Ok, so that bit is easy. Then on to the software side. The normal way people do this stuff is "direwolf" on Linux. So, "pkg install direwolf" installed it. That was easy. Configuring it up was a bit less easy. I found this guide to be helpful (https://andrewmemory.wordpress.com/tag/direwolf/) FreeBSD has the example direwolf config in /usr/local/share/doc/direwolf/examples/direwolf.conf . Now, direwolf will run as a normal user (there's no rc.d script for it yet!) and by default runs out of the current directory. So: $ cd ~ $ cp /usr/local/share/doc/direwolf/examples/direwolf.conf . $ (edit it) $ direwolf Editing it isn't that hard - you need to change your callsign and the audio device. OK, here is the main undocumented bit for FreeBSD - the sound device can just be /dev/dsp . It isn't an ALSA name! Don't waste time trying to use ALSA names. Instead, just find the device you want and reference it. For me the USB sound card shows up as /dev/dsp3 (which is very non specific as USB sound devices come and go, but that's a later problem!) but it's enough to bring it up. So yes, following the above guide, using the right sound device name resulted in a working APRS modem. Next up - something to talk to it. This is called 'xastir'. It's .. well, when you run it, you'll find exactly how old an X application it is. It's very nostalgically old. But, it is enough to get APRS positioning up and test both the TCP/IP side of APRS and the actual radio radio side. Here's the guide I followed: (https://andrewmemory.wordpress.com/2015/03/22/setting-up-direwolfxastir-on-a-raspberry-pi/) So, that was it! So far so good. It actually works well enough to decode and watch APRS traffic around me. I managed to get out position information to the APRS network over both TCP/IP and relayed via VHF radio. Beastie Bits Zebras All the Way Down - Bryan Cantrill (https://www.youtube.com/watch?v=fE2KDzZaxvE) Your impact on FreeBSD (https://www.freebsdfoundation.org/blog/your-impact-on-freebsd/) The Secret to a good Gui (https://bsdmag.org/secret-good-gui/) containerd hits v1.0.0 (https://github.com/containerd/containerd/releases/tag/v1.0.0) FreeBSD 11.1 Custom Kernels Made Easy - Configuring And Installing A Custom Kernel (https://www.youtube.com/watch?v=lzdg_2bUh9Y&t=) Debugging (https://pbs.twimg.com/media/DQgCNq6UEAEqa1W.jpg:large) *** Feedback/Questions Bostjan - Backup Tapes (http://dpaste.com/22ZVJ12#wrap) Philipp - A long time ago, there was a script (http://dpaste.com/13E8RGR#wrap) Adam - ZFS Pool Monitoring (http://dpaste.com/3BQXXPM#wrap) Damian - KnoxBug (http://dpaste.com/0ZZVM4R#wrap) ***

Naoya Ito さんをゲストに迎えて、ディープワーク、データサイエンス、Python, GC, マネジメント、Google Apps などについて話しました。 Show Notes Deep Work: 大事なことに集中する けものフレンズプロジェクト WEB+DB PRESS Vol.97 私たちはいかにして環状線で”悪さをする列車”を捕まえたか Rebuild: 171: Psychologically Safe Podcast (naoya) scikit-learn: machine learning in Python pandas Matplotlib 優良AIスタートアップの見分け方 Apple Rebuilds Siri Backend Services Using Apache Mesos 集合知プログラミング データサイエンスにおけるRubyの現在の位置づけと可能性 Rubyにおける機械学習と統計分析のための環境整備の取り組み GC切って性能向上 Dismissing Python Garbage Collection at Instagram Ruby 2.1: Out-of-Band GC 例えば GC を止める・Ruby ウェブアプリケーションの高速化 - 2nd life The Julia Language Google：マネージャはやはり重要な存在である Reflecting on one very, very strange year at Uber Mad Men Shift Google Pulls 'Sparrow' Mail Apps From App Store Boxy. Inbox by Gmail, finally on your Mac. Mac: Routing Links to Chrome Profiles Choosy: A smarter default browser for Mac OS X What is Azure Active Directory? Okta

Live stream to http://twitch.tv/adafruit looking at Jupyter/IPython as a tool for programming Python on the Raspberry Pi. Looks at how to install and run Jupyter on the Pi, and how to use it to do simple tasks like graph sensor data. Links mentioned in the video: - Jupyter homepage: http://jupyter.org/ - Jupyter installation: http://jupyter.readthedocs.io/en/latest/install.html - Introducing IPython: http://ipython.readthedocs.io/en/stable/interactive/tutorial.html - MCP9808 temperature sensor guide: https://learn.adafruit.com/mcp9808-temperature-sensor-python-library/overview - IPython Plotting: http://ipython.readthedocs.io/en/stable/interactive/plotting.html - Plotting with Matplotlib: http://nbviewer.jupyter.org/github/ipython/ipython/blob/1.x/examples/notebooks/Part%203%20-%20Plotting%20with%20Matplotlib.ipynb Acknowledgements: - Music: bartlebeats - Intro shuttle footage: NASA - Intro fonts: Typodermic - Intro inspiration: Mr. Wizards's World - Matrix background: cool-retro-term & cmatrix ----------------------------------------- Visit the Adafruit shop online - http://www.adafruit.com Subscribe to Adafruit on YouTube: http://adafru.it/subscribe Join our weekly Show & Tell on G+ Hangouts On Air: http://adafru.it/showtell Watch our latest project videos: http://adafru.it/latest New tutorials on the Adafruit Learning System: http://learn.adafruit.com/ Music by bartlebeats: http://soundcloud.com/bartlebeats -----------------------------------------

Seguramente el tema del que te voy a hablar hoy en el podcast te suene a ciencia ficción y creas que es algo que solo lo podemos ver en las películas del mismo género. Sin duda alguna, no estamos en lo más alto en la gráfica de desarrollo en cuanto a soluciones y aplicaciones en esta materia, pero esto no quiere decir que no podamos investigar y aprender de esta ciencia. Ya te conté ¿por qué debemos aprender visión artificial? y hoy te voy a hablar como podemos introducirnos en la visión artificial, OpenCV y Phyton.Antes de continuar quiero hablarte del Campus de Programarfacil. Si quieres crear tus propios proyectos con Arduino o algún dispositivo Open Hardware, debes dominar dos disciplinas, la programación y la electrónica. En el Campus estoy volcando todo mi conocimiento en estas materias con cursos de diferentes niveles, básico, intermedio y avanzado. Tendrás a tu disposición un formulario de soporte premium y sorteos de material electrónico e informático. Entra y busca tu curso.Este tema no es nuevo en el podcast. Ya he hablado en diferentes capítulos:18. Realidad aumentada44. Tratamiento de imágenes con JavaScript64. Proyectos curiosos con Arduino67. Big Data y visión artificialHoy voy a profundizar en la materia y te voy a dar los pasos necesarios para empezar a programar con la biblioteca más famosa de visión artificial, OpenCV.¿Qué es OpenCV?OpenCV es una biblioteca libre desarrollada originalmente por Intel. Vio la luz en el año 1999. Escrita originalmente en C/C++, su mejor virtud es que es multiplataforma, se puede ejecutar en diferentes sistemas operativos (Linux, Windows, Mac OS X, Android e iOS). También la podemos utilizar en diferentes lenguajes de programación como Java, Objective C, Python y mi favorito C#. Precisamente para este último existe una versión que se llama EmguCV.En junio de 2015 se produjo un hito importante, por fin la versión 3.0 estaba disponible. Si hechas números, en 16 años (de 1999 a 2015) solo ha habido 3 versiones. Esto es debido a que desde un principio esta biblioteca ha sido robusta y muy eficiente.En esta última versión cabe destacar que por fin es compatible con la última versión de Python, la 3.0. Esto permite aprovechar todas las ventajas de la última versión de este lenguaje.Quizás sea la biblioteca de visión artificial más importante y más usada del mundo. Es utilizada por universidades, empresas y gente del movimiento Maker para dar rienda suelta a su imaginación al tratarse de un software libre.Pasos para instalar OpenCV y PythonTe preguntarás ¿por qué Python? Aunque todavía no he tratado este lenguaje de programación ni en el blog, ni en el podcast, si que te puedo contar que Python es muy sencillo de usar, favoreciendo el código legible gracias a su sintaxis sencilla.Debemos ser conscientes que el lenguaje nativo de OpenCV es C/C++, con la complejidad que ello conlleva si queremos utilizar esta biblioteca en nuestros proyectos.Lo que más me gusta de Python es que es un lenguaje fácilmente portable a otras plataformas entre las que se incluye Raspberry Pi. Si además disponemos de una cámara conectada, imagínate lo que podemos llegar a conseguir.Aunque en mi día a día yo utilizo Windows y en el Campus he decidido empezar a con este sistema operativo, se puede hacer de igual manera con Linux y OS X.La decisión de empezar por Windows es muy sencilla. Es el sistema operativo más utilizado del mundo y no porque lo diga yo, solo tienes que ver los datos estadísticos que nos proporciona Net Market Share. Según esta empresa, más del 90% de usuarios utilizan Windows.estadistica-uso-sistema-operativoAún así podemos pensar que es una estrategia de ventas y que esta empresa puede pertenecer al magnate de Redmond. Por eso voy a compartir los datos estadísticos obtenidos de Google Analytics sobre el uso de sistemas operativos en esta web osea, vosotros los usuarios.estadistica-analytics-sistema-operativoComo puedes ver hay una diferencia aplastante con el resto de perseguidores. Por eso he optado empezar por Windows, para poder llegar al mayor número de gente y que nadie se sienta excluido.Lo primero que debemos saber antes de empezar con los pasos a seguir para instalar OpenCV y Python, es que esto ya no es una tecnología plug and play. Estamos acostumbrados a hablar de Processing, Arduino, Scratch y las tecnologías fáciles de usar. Con OpenCV la cosa se complica, sobre todo a la hora de preparar el sistema. Pero yo te voy a dar los pasos necesarios para que empieces de una forma muy sencilla. La instalación consta de 3 pasos.Paso 1: Instalación de Python 3.0 con paquetes adicionalesYa no solo tenemos que instalar el lenguaje de programación, para utilizar OpenCV necesitamos instalar, además, ciertos paquetes de Python que nos hará la vida más fácil cuando desarrollemos aplicaciones en visión artificial.NumPy: es una biblioteca de código abierto que da soporte a vectores y arrays para Python.SciPy: es una biblioteca de código abierto que contiene herramientas y algoritmos matemáticos para Python.Matplotlib: es una biblioteca de código abierto para la generación de gráficos a partir de vectores y arrays.Pip: gestor de paquetes para Python.Se puede instalar cada paquete por separado, pero existen plataformas como Anaconda 3 donde viene todo integrado en un único instalador. Te recomiendo que lo hagas con este tipo de plataformas.Paso 2: Instalar OpenCV para Python 3Quizás este paso pudiera ser el más complicado pero gracias al gestor de paquetes Pip se hace muy sencillo. Solo debemos de descargar la versión para nuestro sistema operativo en formato whl y luego instalarlo. Es muy simple gracias al gestor de paquetes.Paso 3: Instalar el entorno de desarrollo (Opcional)Este paso es opcional, podemos utilizar el bloc de notas de Windows para programar en Python. Mi consejo es utilizar Sublime Text 3 y el plugin Anaconda, que convierte este IDE en un entorno de desarrollo optimizado para Python con todas sus funcionalidades.Y estos serían los 3 pasos recomendados para configurar el sistema. Puedes ir al Campus y ver los como lo hago yo paso a paso con vídeos, imágenes y el código necesario para que todo funcione correctamente.El recurso del oyenteHoy traigo un recurso del oyente especial, el email recibido por Antonio Otero. Ha significado mucho par mi porque el objetivo de este proyecto es precisamente ese, ayudar a la gente y en este caso se ha conseguido.Gracias señores por su buena labor.Siento la necesidad de comentarles una situación. (ya os di las gracias en un comentario, pero quiero extenderme mas)Aparte de mi trabajo como desarrollador web, soy formador de inserción para el empleo. Este año me a tocado dar un curso de microsistemas a un grupo algo especial. (Jóvenes entre 18 y 22 años que digamos andan un poco perdidos por no decir nada mas, unos panoramas....).Acostumbrado a mis clases habituales (para "adultos"), no daba con la manera de interesarles en la materia. El temario es muy variado, SO, hardware, electrónica muy básica, scripts mantenimiento.... todo básico pero muy amplio.No encontraba la manera y estaba sufriendo porque no conseguía enderezarlos, estando al borde de la expulsión de algunos alumnos.El caso es que como todas las mañanas en mi hora de trafico hacia el curso y harto de las noticias de política, se me ocurrió poner vuestros podcasts, y habéis sido una inspiración para mi. Habéis cambiado mi forma de ver algunas cosas, me habéis contagiado vuestra ilusión (quiza ya habia perdido alguna) y como buen virus yo se la he trasmitido a mis alumnos.Con scratch he conseguido que se interesen por la programación, y ahora me hacen script de linux bastante majos. incluso hemos estado con ensamblador (muy básico). Pero espero que me programen arduino con c :-)Con arduino están emocionados (he comprado 4 de mi bolsillo pues el centro no los pone). y eso que aún no los han tocado, pero aprenden la teoría con gran interés deseando ponerla en practica :-)En fin, han cambiado de comportamiento completamente, están involucradisimos y no faltan a una clase, y quiero haceros participes de este éxito.La única mala noticia es que estoy llegando al posdcast de esta semana y no se si aguantare a esperar una semana para escucharos de nuevo :-)Gracias, Antonio OteroYa me despido por esta semana, recuerda que nos puedes encontrar en Twitter y Facebook.Cualquier duda o sugerencia en los comentarios de este artículo o a través del formulario de contacto.