POPULARITY
Summary Building a database engine requires a substantial amount of engineering effort and time investment. Over the decades of research and development into building these software systems there are a number of common components that are shared across implementations. When Paul Dix decided to re-write the InfluxDB engine he found the Apache Arrow ecosystem ready and waiting with useful building blocks to accelerate the process. In this episode he explains how he used the combination of Apache Arrow, Flight, Datafusion, and Parquet to lay the foundation of the newest version of his time-series database. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Dagster offers a new approach to building and running data platforms and data pipelines. It is an open-source, cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability, a declarative programming model, and best-in-class testability. Your team can get up and running in minutes thanks to Dagster Cloud, an enterprise-class hosted solution that offers serverless and hybrid deployments, enhanced security, and on-demand ephemeral test deployments. Go to dataengineeringpodcast.com/dagster (https://www.dataengineeringpodcast.com/dagster) today to get started. Your first 30 days are free! Data lakes are notoriously complex. For data engineers who battle to build and scale high quality data workflows on the data lake, Starburst powers petabyte-scale SQL analytics fast, at a fraction of the cost of traditional methods, so that you can meet all your data needs ranging from AI to data applications to complete analytics. Trusted by teams of all sizes, including Comcast and Doordash, Starburst is a data lake analytics platform that delivers the adaptability and flexibility a lakehouse ecosystem promises. And Starburst does all of this on an open architecture with first-class support for Apache Iceberg, Delta Lake and Hudi, so you always maintain ownership of your data. Want to see Starburst in action? Go to dataengineeringpodcast.com/starburst (https://www.dataengineeringpodcast.com/starburst) and get $500 in credits to try Starburst Galaxy today, the easiest and fastest way to get started using Trino. Join us at the top event for the global data community, Data Council Austin. From March 26-28th 2024, we'll play host to hundreds of attendees, 100 top speakers and dozens of startups that are advancing data science, engineering and AI. Data Council attendees are amazing founders, data scientists, lead engineers, CTOs, heads of data, investors and community organizers who are all working together to build the future of data and sharing their insights and learnings through deeply technical talks. As a listener to the Data Engineering Podcast you can get a special discount off regular priced and late bird tickets by using the promo code dataengpod20. Don't miss out on our only event this year! Visit dataengineeringpodcast.com/data-council (https://www.dataengineeringpodcast.com/data-council) and use code dataengpod20 to register today! Your host is Tobias Macey and today I'm interviewing Paul Dix about his investment in the Apache Arrow ecosystem and how it led him to create the latest PFAD in database design Interview Introduction How did you get involved in the area of data management? Can you start by describing the FDAP stack and how the components combine to provide a foundational architecture for database engines? This was the core of your recent re-write of the InfluxDB engine. What were the design goals and constraints that led you to this architecture? Each of the architectural components are well engineered for their particular scope. What is the engineering work that is involved in building a cohesive platform from those components? One of the major benefits of using open source components is the network effect of ecosystem integrations. That can also be a risk when the community vision for the project doesn't align with your own goals. How have you worked to mitigate that risk in your specific platform? Can you describe the operational/architectural aspects of building a full data engine on top of the FDAP stack? What are the elements of the overall product/user experience that you had to build to create a cohesive platform? What are some of the other tools/technologies that can benefit from some or all of the pieces of the FDAP stack? What are the pieces of the Arrow ecosystem that are still immature or need further investment from the community? What are the most interesting, innovative, or unexpected ways that you have seen parts or all of the FDAP stack used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on/with the FDAP stack? When is the FDAP stack the wrong choice? What do you have planned for the future of the InfluxDB IOx engine and the FDAP stack? Contact Info LinkedIn (https://www.linkedin.com/in/pauldix/) pauldix (https://github.com/pauldix) on GitHub Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ (https://www.pythonpodcast.com) covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast (https://www.themachinelearningpodcast.com) helps you go from idea to production with machine learning. Visit the site (https://www.dataengineeringpodcast.com) to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com (mailto:hosts@dataengineeringpodcast.com)) with your story. Links FDAP Stack Blog Post (https://www.influxdata.com/blog/flight-datafusion-arrow-parquet-fdap-architecture-influxdb/) Apache Arrow (https://arrow.apache.org/) DataFusion (https://arrow.apache.org/datafusion/) Arrow Flight (https://arrow.apache.org/docs/format/Flight.html) Apache Parquet (https://parquet.apache.org/) InfluxDB (https://www.influxdata.com/products/influxdb/) Influx Data (https://www.influxdata.com/) Podcast Episode (https://www.dataengineeringpodcast.com/influxdb-timeseries-data-platform-episode-199) Rust Language (https://www.rust-lang.org/) DuckDB (https://duckdb.org/) ClickHouse (https://clickhouse.com/) Voltron Data (https://voltrondata.com/) Podcast Episode (https://www.dataengineeringpodcast.com/voltron-data-apache-arrow-episode-346/) Velox (https://github.com/facebookincubator/velox) Iceberg (https://iceberg.apache.org/) Podcast Episode (https://www.dataengineeringpodcast.com/iceberg-with-ryan-blue-episode-52/) Trino (https://trino.io/) ODBC == Open DataBase Connectivity (https://en.wikipedia.org/wiki/Open_Database_Connectivity) GeoParquet (https://github.com/opengeospatial/geoparquet) ORC == Optimized Row Columnar (https://orc.apache.org/) Avro (https://avro.apache.org/) Protocol Buffers (https://protobuf.dev/) gRPC (https://grpc.io/) The intro and outro music is from The Hug (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/Love_death_and_a_drunken_monkey/04_-_The_Hug) by The Freak Fandango Orchestra (http://freemusicarchive.org/music/The_Freak_Fandango_Orchestra/) / CC BY-SA (http://creativecommons.org/licenses/by-sa/3.0/)
This week Chris Currier and I talk about mobile forensics and protocol buffers.
On a previous episode of Go Time we discussed binary bloat, and how the Go protocol buffer implementation is a big offender. In this episode we dive into the history of protocol buffers and gRPC, then we discuss how the protocol and the implementation can vary and lead to things like binary bloat.
On a previous episode of Go Time we discussed binary bloat, and how the Go protocol buffer implementation is a big offender. In this episode we dive into the history of protocol buffers and gRPC, then we discuss how the protocol and the implementation can vary and lead to things like binary bloat.
The foundational technology for Muse 2 is local-first sync, which draws from over a decade of computer science research on CRDTs. Mark, Adam Wiggins, and Adam Wulf get technical to describe the Muse sync technology architecture in detail. Topics include the difference between transactional, blob, and ephemeral data; the “atoms” concept inspired by Datomic; Protocol Buffers; and the user's data as a bag of edits. Plus: why sync is a powerful substrate for end-user programming. @MuseAppHQ hello@museapp.com Show notes Adam Wulf @adamwulf Fantastical Loose Leaf Wulf's iOS ink libraries OpenGL Bézier curves Houston Muse 2.0 launches May 24 Metamuse episode on local-first software Core Data Pocket Clue, Wunderlist CouchDB, Firebase Adam's writeup on sync technologies from 2014 Evernote Pixelpusher Slow Software CRDTs, operational transform Automerge Actual Budget last write wins Actual open source hybrid logical clock, vector clock CloudKit lazy loading API versioning Protocol Buffers Wulf's article on atoms Datomic “put a UUID and a version number on everything” Swift property wrappers functional reactive programming Sourcery Sentry HDD indicator light Muse job post for a local-first engineer Local-first day at ECOOP 2022
Neste programa abordamos sobre o serviço gRPC, é um framework criado pelo Google com objetivo de facilitar o processo de comunicação entre sistemas, de uma forma extremamente rápida. Assuntos abordados no tema O que é gRPC? Para o que serve, de modo geral? Protocol Buffers (protobuf) API REST x gRPC e suas diferenças Principais vantagens do HTTP2 Vantagem e desvantagens em utilizar gRPC Qual seria o cenário ideal para optar por gRPC e não REST? Links úteis Cupom de desconto Kamo Coffee CAFEDEBUG10 compre seu café no site abaixo https://www.kamocoffee.com.br/ https://grpc.io/ https://blog.lsantos.dev/guia-grpc-1/ https://medium.com/mobicareofficial/iniciando-com-grpc-c48d81774266 https://gago.io/blog/grpc-no-asp-net-core-guia-introdutorio/ https://github.com/grpc/grpc-web Participantes Jéssica Nathany (Programadora e host) LinkedIn: https://www.linkedin.com/in/jessica-nathany-carvalho-freitas-38260868/ Weslley Fratini (Software Developer e co-host) LinkedIn: https://www.linkedin.com/in/weslley-fratini/ Lucas Santos (Sr. Software Engineer na Klarna | Google Developer Expert | Docker Captain) Site: https://info.lsantos.dev Twitter: https://twitter.lsantos.dev LinkedIn: https://linkedin.lsantos.dev Canal: https://youtube.lsantos.dev Apoia.se: https://apoia.se/cafedebug
@izumin5210 をゲストに呼んで Protocol Buffers と gRPC を利用したマイクロサービス間通信の考え方とTipsについて聞きました。 トピック gRPC が使えない環境における Protocol Buffers のメリット Protocol Buffers の書き方で気をつけること インターフェースの重要性 proto ファイルの管理方法 参考リンク マイクロサービス共通ライブラリで “Governance through code” を実現する React でデザインシステムを正しく実装する - コンポーネントカタログを超えて Wantedly Engineering Handbook protobufスキーマとgRPC通信 github.com/izumin5210/grapi 公式ドキュメントの読み方 Language Guide (proto3) API Design Guide マイクロサービスでもポチポチ確認するための Kubefork ソフトウェア設計の Why & What & How
In this episode of Elixir Mix, we talk with Andrea Leopardi about how they solved sharing Protobuf protocols across multiple projects for their RabbitMQ consumers. We also learn the benefits they found of using Elixir in a microservices architecture, the benefits of Broadway and much more! Panelists Josh Adams Sophie DeBenedetto Mark Ericksen Guest Andrea Leopardi "The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today! Links community Sharing Protobuf schemas across services Microservice Architecture Protocol Buffers GitHub/protocolbuffers/protobuf GitHub/bitwalker/exprotobuf GitHub/tony612/protobuf-elixir GitHub/Dependabot Dependabot Twitter Andrea Leopardi: @whatyouhide GitHub Andrea Leopardi https://andrealeopardi.com Picks Josh Adams: Helm Charts ConcourseCI Sophie DeBenedetto: Introducing Telemetry Mark Ericksen: JC Label Maker Andrea Leopardi: Exercising at home! Follow on Twitter: Elixir Mix - @elixir_mix Mark Ericksen - @brainlid Sophie DeBenedetto - @sm_debenedetto Josh Adams - @knewter
In this episode of Elixir Mix, we talk with Andrea Leopardi about how they solved sharing Protobuf protocols across multiple projects for their RabbitMQ consumers. We also learn the benefits they found of using Elixir in a microservices architecture, the benefits of Broadway and much more! Panelists Josh Adams Sophie DeBenedetto Mark Ericksen Guest Andrea Leopardi "The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today! Links community Sharing Protobuf schemas across services Microservice Architecture Protocol Buffers GitHub/protocolbuffers/protobuf GitHub/bitwalker/exprotobuf GitHub/tony612/protobuf-elixir GitHub/Dependabot Dependabot Twitter Andrea Leopardi: @whatyouhide GitHub Andrea Leopardi https://andrealeopardi.com Picks Josh Adams: Helm Charts ConcourseCI Sophie DeBenedetto: Introducing Telemetry Mark Ericksen: JC Label Maker Andrea Leopardi: Exercising at home! Follow on Twitter: Elixir Mix - @elixir_mix Mark Ericksen - @brainlid Sophie DeBenedetto - @sm_debenedetto Josh Adams - @knewter
gRPC (gRPC Remote Procedure Calls[1]) is an open source remote procedure call (RPC) system initially developed at Google in 2015[2]. It uses HTTP/2 for transport, Protocol Buffers as the message format. In this video I want to explore gRPC, go through examples, pros and cons of gRPC. Client/ Server communication SOAP HTTP (REST) WebSockets Client Libraries gRPC gRPC Demo todos gRPC Pros and Cons Pros Fast two/uni and request Unform One library to rule them all Progress feedback( long synchronous requests) drop pluggable wait...) cancel request All benefits of H2 and Protobuff Cons schema based (not everyone wants schema) Thick client - limited languages - Proxies still don’t understand it Still young Error handling No native browser support Timeouts, circuit breaker just like any RPC (pub/sub rules in this case) Can you create your own protocol? Spotify example with Hermes --- Send in a voice message: https://anchor.fm/hnasr/message
In this episode of ElixirMix, we visit with Steven Nunez about how Flatiron School adopted Elixir and is using RabbitMQ. He shares how he decides to “rails new” or “mix phx.new” for a project. How adopting Elixir in a team goes better when the team “falls in love” with what it gives them. Steven shares how their RabbitMQ queues are setup, how the messages are designed, how to spread the patterns throughout the teams and projects, and much more! Panelists Mark Ericksen Josh Adams Sophie DeBenedetto Eric Oestrich Guest Steven Nunez Sponsors CacheFly ____________________________________________________________ "The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today! ____________________________________________________________ Links Pluralsight Github jondot/sneakers RabbitMQ Kafka Apache Eventide Protocol Buffers RabbitMQ Tutorial One Elixir Github bitwalker/exprotobuf RabbitMQ Tutorial Six Elixir Steven Nunez Twitter Flatiron School Twitter Picks Josh Adams: Website Generator Statically Typed Site Generator VVVV Sophie DeBenedetto: A Tour of Go Eric Oestrich: GitHub TerryCavanagh/VVVVVV The Expanse Mark Ericksen: GitHub dashbitco/nimble_pool
In this episode of ElixirMix, we visit with Steven Nunez about how Flatiron School adopted Elixir and is using RabbitMQ. He shares how he decides to “rails new” or “mix phx.new” for a project. How adopting Elixir in a team goes better when the team “falls in love” with what it gives them. Steven shares how their RabbitMQ queues are setup, how the messages are designed, how to spread the patterns throughout the teams and projects, and much more! Panelists Mark Ericksen Josh Adams Sophie DeBenedetto Eric Oestrich Guest Steven Nunez Sponsors CacheFly ____________________________________________________________ "The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today! ____________________________________________________________ Links Pluralsight Github jondot/sneakers RabbitMQ Kafka Apache Eventide Protocol Buffers RabbitMQ Tutorial One Elixir Github bitwalker/exprotobuf RabbitMQ Tutorial Six Elixir Steven Nunez Twitter Flatiron School Twitter Picks Josh Adams: Website Generator Statically Typed Site Generator VVVV Sophie DeBenedetto: A Tour of Go Eric Oestrich: GitHub TerryCavanagh/VVVVVV The Expanse Mark Ericksen: GitHub dashbitco/nimble_pool
What are protocol buffers? Why do we care? How do we use them, and what the heck is gRPC?!?!? Follow Us Frank: Twitter, Blog, GitHub James: Twitter, Blog, GitHub Merge Conflict: Twitter, Facebook, Website, Chat on Discord Music : Amethyst Seer - Citrine by Adventureface ⭐⭐ Review Us (https://itunes.apple.com/us/podcast/merge-conflict/id1133064277?mt=2&ls=1) ⭐⭐ Machine transcription available on http://mergeconflict.fm
What are protocol buffers? Why do we care? How do we use them, and what the heck is gRPC?!?!? Follow Us Frank: Twitter, Blog, GitHub James: Twitter, Blog, GitHub Merge Conflict: Twitter, Facebook, Website, Chat on Discord Music : Amethyst Seer - Citrine by Adventureface ⭐⭐ Review Us (https://itunes.apple.com/us/podcast/merge-conflict/id1133064277?mt=2&ls=1) ⭐⭐ Machine transcription available on http://mergeconflict.fm
In this podcast we sit down with Matt Klein, software plumber at Lyft and creator of Envoy, and discuss topics including the continued evolution of the popular proxy, the strength of the open source Envoy community, and the value of creating and implementing standards throughout the technology stack. We also explore the larger topic of cloud natives platforms, and discuss the tradeoffs between using a simple and opinionated platform against something that is bespoke and more configurable, but also more complex. Related to this, Matt shares his thoughts on when and how to make the decision within an organisation to embrace technology like container orchestration and service meshes. Finally, we explore the creation of the new Envoy Mobile project. The goal of this project is to expand the capabilities provided by Envoy all the way out to mobile devices powered by Android and iOS. For example, most current user-focused traffic shifting that is conducted at the edge is implemented with coarse-grained approaches via by BGP and DNS, and using something like Envoy within mobile app networking stacks should allow finer-grained control. Why listen to this podcast: - The Envoy Proxy community has grown from strength-to-strength over the last year, from the inaugural EnvoyCon that ran alongside KubeCon NA 2018, to the increasing number of code contributions from engineers working across the industry - Attempting to create a community-driven “universal proxy data plane” with clearly defined APIs, like Envoy’s XDS API, has allowed vendors to collaborate on a shared abstraction while still allowing room for “differentiated success” to be built on top of this standard Google’s gRPC framework is adopting the Envoy XDS APIs, as this will allow both Envoy and gRPC instances to be operated via a single control plane, for example, Google Cloud Platform’s Traffic Director service. - There is a tendency within the software development industry to fetishise architectures that are designed and implemented by the unicorn tech companies, but not every organisation operates at this scale. - However, there has also been industry pushback against the complexity that modern platform components like container orchestration and service meshes can introduce to a technology stack. - Using a platform within these components provides the best return on investment when an organisation’s software architecture and development teams have reached a certain size. - Function-as-a-Service (Faas)-type platforms will most likely be how engineers will interact with software in the future. Business-focused developers often do not want to interact with the platform plumbing Envoy Mobile is building on prior art, and aims to expand the capabilities provided by Envoy all the way out to mobile devices using Android and iOS. Most current end user traffic shifting is implemented with coarse-grained approaches via BGP and DNS, and using something like Envoy instead will allow finer-grained control. - Using Envoy Mobile in combination with Protocol Buffers 3, which supports annotations on APIs, can facilitate working with APIs offline, configuring caching, and handling poor networking conditions. One of the motivations for this work is that small increases in application response times can lead to better business outcomes. More on this: Quick scan our curated show notes on InfoQ https://bit.ly/33nlGMu You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Check the landing page on InfoQ: https://bit.ly/33nlGMu
This week we welcome Hana Dusíková to the show and we chat about her compiler time regular expressions library, Protocol Buffers, std::embed and getting good compile and runtime performance when doing metaprogramming. Unfortunately, due to an extended edit time, the volunteer and diversity ticket programmes for C++ on Sea, mentioned during the discussion, have already closed. The student programme is still open as this show is published.
ajiyoshiさん、ぺいさんとGoCon Spring 2018、Go言語、コード生成、Protocol Buffers、JSONなどについて話しました。 Go Conference 2018 Spring src/cmd/compile/internal/gc/walk.go のMapのASTを処理するあたり GoらしいAPIを求める旅路 (Go Conference 2018 Spring) github.com/go-chi/chi gorilla/mux julienschmidt/httprouter コードジェネレートとの付き合い方 @Go Conference 2018 Spring swaggo/swag: Automatically generate RESTful API documentation with Swagger 2.0 for Go. 115枚目のスライド swaggo/swag/issues/88: Docs generation loop Protocol Buffers protocプラグインの書き方 今さらProtocol Buffersと、手に馴染む道具の話 全ての管理画面開発に悩めるエンジニアに捧ぐ 〜Viron誕生〜 管理画面は設定ファイルぐらいシンプルに作れるべき!『Viron』を使ってみました - pixiv inside OpenRTB Integration
Coding Solo - A podcast about freelancing in the UK - codingsolo
Today Alex and David discuss the start of the new Coding Solo podcast, how they got into freelancing, what worried them about it. They also discuss the end result they are experiencing today alongside some interesting technology / resources you can tap into. # Follow Us Alex - https://twitter.com/alexbilbie David - https://twitter.com/davzie # Give Us Feedback or Ask Questions feedback@codingsolo.works # Mentions Career Fork Book: https://leanpub.com/freelancedeveloperbook Alex's Allergy App: https://canteatthat.com/ Protocol Buffers: https://developers.google.com/protocol-buffers/
Digging back into our archive of interviews from Google Cloud Next, Mark and Francesc talk to Brad Rydzewski, creator of Drone, about the open source continuous integration and delivery platform. We are also excited to have the amazing Jessie Frazelle joining us as well! About Brad Rydzewksi Brad Rydzewski is the creator of the open source Drone project, which provides container based continuous delivery. About Jessie Frazelle Jessie Frazelle is also part of the Google Cloud Platform Developer Advocacy team, and is generally known as “That container girl”, and is an avid “Door to door leenuux salesperson.” Cool things of the week Announcing general availability of Google Cloud Dataflow for Python blog Google Cloud Platform for Data Scientists: Using R with Google Cloud SQL for MySQL blog Cloud SQL for PostgreSQL: Managed PostgreSQL for your mobile and geospatial applications in Google Cloud blog Interview Drone homepage github Drone on Container Engine github Kubernetes Namespaces docs Docker compose docs Drone Plugins site http://try.drone.io/ Question of the week This questions of the week comes from Rokesh Jankie: What is protocol buffers, and why should we all start using it? Protocol Buffers site gRPC previously on the podcast episode 15 episode 43 FlatBuffers site Where can you find us next? Mark will be heading to Vancouver Unity Games Meetup and Polyglot Vancouver Meetup, and then on to East Coast Games Conference and Vector in April. Francesc will be presenting at Gophercon China in April, and will then head off to New York!
Sam Phippen helps us celebrate episode 100, as we discuss Diesel bugs, REST, RPC, and more. Diesel LEFT JOINS bug Google Spanner Information Schema Standard Spanner Beta Paper HTML5 formaction Must be Willing to Relocate to San Francisco GRPC Protocol Buffers The Listen gem breaks my laptop Thank you to our sponsor this week, FreshBooks!
This week on the show, we've got FreeBSD quarterly Status reports to discuss, OpenBSD changes to the installer, EC2 and IPv6 and more. Stay This episode was brought to you by Headlines OpenBSD changes of note 6 (http://www.tedunangst.com/flak/post/openbsd-changes-of-note-6) OpenBSD can now be cross built with clang. Work on this continues Build ld.so with -fno-builtin because otherwise clang would optimize the local versions of functions like dlmemset into a call to memset, which doesn't exist. Add connection timeout for ftp (http). Mostly for the installer so it can error out and try something else. Complete https support for the installer. I wonder how they handle certificate verification. I need to look into this as I'd like to switch the FreeBSD installer to this as well New ocspcheck utility to validate a certificate against its ocsp responder. net lock here, net lock there, net lock not quite everywhere but more than before. More per cpu counters in networking code as well. Disable and lock Silicon Debug feature on modern Intel CPUs. Prevent wireless frame injection attack described at 33C3 in the talk titled “Predicting and Abusing WPA2/802.11 Group Keys” by Mathy Vanhoef. Add support for multiple transmit ifqueues per network interface. Supported drivers include bge, bnx, em, myx, ix, hvn, xnf. pledge now tracks when a file as opened and uses this to permit or deny ioctl. Reimplement httpd's support for byte ranges. Fixes a memory DOS. FreeBSD 2016Q4 Status Report (https://www.freebsd.org/news/status/report-2016-10-2016-12.html) An overview of some of the work that happened in October - December 2016 The ports tree saw many updates and surpassed 27,000 ports The core team was busy as usual, and the foundation attended and/or sponsored a record 24 events in 2016. CEPH on FreeBSD seems to be coming along nicely. For those that do not know, CEPH is a distributed filesystem that can sit on top of another filesystem. That is, you can use it to create a clustered filesystem out of a bunch of ZFS servers. Would love to have some viewers give it a try and report back. OpenBSM, the FreeBSD audit framework, got some updates Ed Schouten committed a front end to export sysctl data in a format usable by Prometheus, the open source monitoring system. This is useful for other monitoring software too. Lots of updates for various ARM boards There is an update on Reproducible Builds in FreeBSD, “ It is now possible to build the FreeBSD base system (kernel and userland) completely reproducibly, although it currently requires a few non-default settings”, and the ports tree is at 80% reproducible Lots of toolchain updates (gcc, lld, gdb) Various updates from major ports teams *** Amazon rolls out IPv6 support on EC2 (http://www.daemonology.net/blog/2017-01-26-IPv6-on-FreeBSD-EC2.html) A few hours ago Amazon announced that they had rolled out IPv6 support in EC2 to 15 regions — everywhere except the Beijing region, apparently. This seems as good a time as any to write about using IPv6 in EC2 on FreeBSD instances. First, the good news: Future FreeBSD releases will support IPv6 "out of the box" on EC2. I committed changes to HEAD last week, and merged them to the stable/11 branch moments ago, to have FreeBSD automatically use whatever IPv6 addresses EC2 makes available to it. Next, the annoying news: To get IPv6 support in EC2 from existing FreeBSD releases (10.3, 11.0) you'll need to run a few simple commands. I consider this unfortunate but inevitable: While Amazon has been unusually helpful recently, there's nothing they could have done to get support for their IPv6 networking configuration into FreeBSD a year before they launched it. You need the dual-dhclient port: pkg install dual-dhclient And the following lines in your /etc/rc.conf: ifconfigDEFAULT="SYNCDHCP acceptrtadv" ipv6activateallinterfaces="YES" dhclientprogram="/usr/local/sbin/dual-dhclient" + It is good to see FreeBSD being ready to use this feature on day 0, not something we would have had in the past Finally, one important caveat: While EC2 is clearly the most important place to have IPv6 support, and one which many of us have been waiting a long time to get, this is not the only service where IPv6 support is important. Of particular concern to me, Application Load Balancer support for IPv6 is still missing in many regions, and Elastic Load Balancers in VPC don't support IPv6 at all — which matters to those of us who run non-HTTP services. Make sure that IPv6 support has been rolled out for all the services you need before you start migrating. Colin's blog also has the details on how to actually activate IPv6 from the Amazon side, if only it was as easy as configuring it on the FreeBSD side *** FreeBSD's George Neville-Neil tries valiantly for over an hour to convince a Linux fan of the error of their ways (https://www.youtube.com/watch?v=cofKxtIO3Is) In today's episode of the Lunduke Hour I talk to George Neville-Neil -- author and FreeBSD advocate. He tries to convince me, a Linux user, that FreeBSD is better. + They cover quite a few topics, including: + licensing, and the motivations behind it + vendor relations + community + development model + drivers and hardware support + George also talks about his work with the FreeBSD Foundation, and the book he co-authored, “The Design and Implementation of the FreeBSD Operating System, 2nd Edition” News Roundup An interactive script that makes it easy to install 50+ desktop environments following a base install of FreeBSD 11 (https://github.com/rosedovell/unixdesktops) And I thought I was doing good when I wrote a patch for the installer that enables your choice of 3 desktop environments... This is a collection of scripts meant to install desktop environments on unix-like operating systems following a base install. I call one of these 'complete' when it meets the following requirements: + A graphical logon manager is presented without user intervention after powering on the machine + Logging into that graphical logon manager takes the user into the specified desktop environment + The user can open a terminal emulator I need to revive my patch, and add Lumina to it *** Firefox 51 on sparc64 - we did not hit the wall yet (https://blog.netbsd.org/tnf/entry/firefox_51_on_sparc64_we) A NetBSD developers tells the story of getting Firefox 51 running on their sparc64 machine It turns out the bug impacted amd64 as well, so it was quickly fixed They are a bit less hopeful about the future, since Firefox will soon require rust to compile, and rust is not working on sparc64 yet Although there has been some activity on the rust on sparc64 front, so maybe there is hope The post also look at a few alternative browsers, but it not hopeful *** Introducing Bloaty McBloatface: a size profiler for binaries (http://blog.reverberate.org/2016/11/07/introducing-bloaty-mcbloatface.html) I'm very excited to announce that today I'm open-sourcing a tool I've been working on for several months at Google. It's called Bloaty McBloatface, and it lets you explore what's taking up space in your .o, .a, .so, and executable binary files. Bloaty is available under the Apache 2 license. All of the code is available on GitHub: github.com/google/bloaty. It is quick and easy to build, though it does require a somewhat recent compiler since it uses C++11 extensively. Bloaty primarily supports ELF files (Linux, BSD, etc) but there is some support for Mach-O files on OS X too. I'm interested in expanding Bloaty's capabilities to more platforms if there is interest! I need to try this one some of the boot code files, to see if there are places we can trim some fat We've been using Bloaty a lot on the Protocol Buffers team at Google to evaluate the binary size impacts of our changes. If a change causes a size increase, where did it come from? What sections/symbols grew, and why? Bloaty has a diff mode for understanding changes in binary size The diff mode looks especially interesting. It might be worth setting up some kind of CI testing that alerts if a change results in a significant size increase in a binary or library *** A BSD licensed mdns responder (https://github.com/kristapsdz/mdnsd) One of the things we just have to deal with in the modern world is service and system discovery. Many of us have fiddled with avahi or mdnsd and related “mdns” services. For various reasons those often haven't been the best-fit on BSD systems. Today we have a github project to point you at, which while a bit older, has recently been updated with pledge() support for OpenBSD. First of all, why do we need an alternative? They list their reasons: This is an attempt to bring native mdns/dns-sd to OpenBSD. Mainly cause all the other options suck and proper network browsing is a nice feature these days. Why not Apple's mdnsd ? 1 - It sucks big time. 2 - No BSD License (Apache-2). 3 - Overcomplex API. 4 - Not OpenBSD-like. Why not Avahi ? 1 - No BSD License (LGPL). 2 - Overcomplex API. 3 - Not OpenBSD-like 4 - DBUS and lots of dependencies. Those already sound like pretty compelling reasons. What makes this “new” information again is the pledge support, and perhaps it's time for more BSD's to start considering importing something like mdnsd into their base system to make system discovery more “automatic” *** Beastie Bits Benno Rice at Linux.Conf.Au: The Trouble with FreeBSD (https://www.youtube.com/watch?v=Ib7tFvw34DM) State of the Port of VMS to x86 (http://vmssoftware.com/pdfs/State_of_Port_20170105.pdf) Microsoft Azure now offers Patent Troll Protection (https://thestack.com/cloud/2017/02/08/microsoft-azure-now-offers-patent-troll-ip-protection/) FreeBSD Storage Summit 2017 (https://www.freebsdfoundation.org/news-and-events/event-calendar/freebsd-storage-summit-2017/) If you are going to be in Tokyo, make sure you come to (http://bhyvecon.org/) Feedback/Questions Farhan - Laptops (http://pastebin.com/bVqsvM3r) Hjalti - rclone (http://pastebin.com/7KWYX2Mg) Ivan - Jails (http://pastebin.com/U5XyzMDR) Jungle - Traffic Control (http://pastebin.com/sK7uEDpn) ***
Hajime Morita さんをゲストに迎えて、達人プログラマーなどについて話しました。 Show Notes Rebuild: Supporter Naoya Ito: "業界の悪習: 新人に10冊も20冊も自分が読んだ本を薦める" 新装版 達人プログラマー 職人から名匠への道 | Amazon 新装版 達人プログラマー 職人から名匠への道 | オーム社 eBook Store The Pragmatic Bookshelf Convolutional neural network Rational Unified Process UML 統一モデリング言語 Plantuml レガシーコード改善ガイド Add Code from a Template | Android Studio Protocol Buffers Amazon Athena Sumo Logic Splunk jq Becky! Internet Mail Wanderlust リファクタリング 既存のコードを安全に改善する CODE COMPLETE 第2版 上 Error handling and Go Thinking in React - React Design Patterns Martin Fowler UNIXという考え方―その設計思想と哲学 Takuto Wada: "若者への課題図書としてまずは『達人プログラマー』と『UNIXという考え方』を挙げた" 新卒ソフトウェアエンジニアのための技術書100冊 - クックパッド開発者ブログ The Best Software Writing I: Selected and Introduced by Joel Spolsky steps to phantasien
More Than Just Code podcast - iOS and Swift development, news and advice
We answer an askMTJC about our use of Swift 3. Aaron discusses some feedback on his Mac: End of Life prognostication on episode 111, as well as, feedback on Air Pods. Jaime tells us about his new position at Simple which leads into more fintech and banking. We follow up on ProtoBufs in Kitura and the Omni Group's new pricing strategy. We discuss the new App Store Search Ads, opposition to React Native and the Pixel phone by Google. Picks: Sky Force 2014 for Apple TV, an iOS Simulator logging protip and CatPaint (stealth pick) Sponsored by: Hired CatPaint & Sticker by @CoryDMC Episode 113 Show Notes: THE MAC DOES HAVE A FUTURE, EVEN IF IT’S OS DOESN’T Dave Rogers on Episode 111 – Storm & Stress Christophe Fondacci on Episode 111 – Storm & Stress Christopher Stott Simple Fintech Tangerine Protocol Buffers in your Kitura Apps - Swift@IBM GraphQL REST Experimenting with App Store Search Ads Branch.io Toronto Blue Jays 2016 MLB Sticker Pack The Omni Group is moving to free downloads with In App Purchases Curtis Herbert - Challenging our Assumptions to Succeed in the App Store Why I’m not a React Native programmer New leak tells us absolutely everything about Google's Pixel phones Nimbus Steel Series Controller Episode 113 Picks: Sky Force 2014 for Apple TV Protip: Make iOS Simulator logging great again CatPaint with Sticker Pack (stealth pick)
In the fifteenth episode of this podcast, your hosts Francesc and Mark interview Varun Talwar. Varun is a product manager in charge of gRPC, an open source project created at Google that helps you build distributed systems like we do internally at Google. About Varun Varun is a product manager in Google Cloud team and has recently taken on gRPC. Prior to this he was responsible for Google Cloud Launcher, a launchpad to easily spin up popular software images on Google Compute Engine. He is a long time Googler who has previously worked on YouTube, Maps and Adsense. Follow Varun on Twitter @varungyan. Cool thing of the week Spotify is now on Google Cloud Platform: Spotify chooses Google Cloud Platform to power data infrastructure blog Announcing Spotify Infrastructure's Googley Future blog Google's BigQuery is da bomb - I can start with 2.2Billion ‘things' and compute/summarize down to 20K in < 1 min. tweet Interview Resources: grpc.io gRPC on GitHub Mailing list for gRPC: grpc-io@googlegroups.com Take a REST with HTTP/2, Protobufs, and Swagger blog Protocol Buffers docs etcd: distributed key-value store with grpc/http2 blog Flatbuffers docs Game on! Flatbuffers video thrift docs Question of the week Special guests Sara Robinson and David East. Firebase authentication with email and password docs Firebase authData for iOS docs Firebase UI for Android and iOS
Carl and Richard talk to Demis Bellot about ServiceStack, a set of tools for building web services and MVC web sites with incredible performance. Demis talks about his thinking behind ServiceStack, its support for a diverse set of protocols and how it compares to WCF and WebAPI. The conversation also dives into Google's Protocol Buffers, an extremely lean protocol even faster than JSON for web services as well as Dart, Google's optionally typed, higher-level language that transpiles to Javascript. Awesome conversation with a hugely smart guy!Support this podcast at — https://redcircle.com/net-rocks/donations
Jeden Dienstag ab 19:00 findet im www.raumzeitlabor.de die Offene RaumZeitLaborierung statt. Dieses Video ist die Aufnahme des Vortrags „Protocol Buffers" von sECuRE am 2012-03-27.
Carl and Richard talk to Demis Bellot about ServiceStack, a set of tools for building web services and MVC web sites with incredible performance. Demis talks about his thinking behind ServiceStack, its support for a diverse set of protocols and how it compares to WCF and WebAPI. The conversation also dives into Google's Protocol Buffers, an extremely lean protocol even faster than JSON for web services as well as Dart, Google's optionally typed, higher-level language that transpiles to Javascript. Awesome conversation with a hugely smart guy!Support this podcast at — https://redcircle.com/net-rocks/donations