Podcasts about Amazon Mechanical Turk

Artificial Intelligence fuels both enthusiasm and panic. Technologists are inclined to give their creations leeway, pretend they're animated beings, and consider them efficient. As users, we may complain when these technologies don't obey, or worry about their influence on our choices and our livelihoods. And yet, we also yearn for their convenience, see ourselves reflected in them, and treat them as something entirely new. But when we overestimate the automation of these tools, award-winning author Antonio A. Casilli argues, we fail to recognize how our fellow humans are essential to their efficiency. The danger is not that robots will take our jobs, but that humans will have to do theirs. In this bracing and powerful book, Antonia A. Casilli uses up-to-the-minute research to show how today's technologies, including AI, continue to exploit human labor—even ours. He connects the diverse activities of today's tech laborers: platform workers, like Uber drivers and Airbnb hosts; “micro workers,” including those performing atomized tasks like data entry on Amazon Mechanical Turk; and the rest of us, as we evaluate text or images to show we're not robots, react to Facebook posts, or approve or improve the output of generative AI. As Casilli shows us, algorithms, search engines, and voice assistants wouldn't function without unpaid or underpaid human contributions. Further, he warns that if we fail to recognize this human work, we risk a dark future for all human labor. Waiting for Robots: The Hired Hands of Automation (U Chicago Press, 2025) urges us to move beyond the simplistic notion that machines are intelligent and autonomous. As the proverbial Godot, robots are the bearers of a messianic promise that is always postponed. Instead of bringing prosperity for all, they discipline the workforce, so we don't dream of a world without drudgery and exploitation. Casilli's eye-opening book makes clear that most “automation” requires human labor—and likely always will—shedding new light on today's consequences and tomorrow's threats of failing to recognize and compensate the “click workers” of today. Michael O. Johnston, Ph.D. is a Assistant Professor of Sociology at William Penn University. He is the author of The Social Construction of a Cultural Spectacle: Floatzilla (Lexington Books, 2023) and Community Media Representations of Place and Identity at Tug Fest: Reconstructing the Mississippi River (Lexington Books, 2022). His general area of study is at the intersection of space, behavior, and identity. He is currently conducting research about the negotiation that humans make between oneself, identification of place, and the attachment/s they have to those places. To learn more about Michael O. Johnston you can go to his personal website, Google Scholar, Bluesky (@professorjohnst.bsky.social),Twitter (@ProfessorJohnst), or by email (johnstonmo@wmpenn.edu) Learn more about your ad choices. Visit megaphone.fm/adchoices Support our show by becoming a premium member! https://newbooksnetwork.supportingcast.fm/new-books-network

Artificial Intelligence fuels both enthusiasm and panic. Technologists are inclined to give their creations leeway, pretend they're animated beings, and consider them efficient. As users, we may complain when these technologies don't obey, or worry about their influence on our choices and our livelihoods. And yet, we also yearn for their convenience, see ourselves reflected in them, and treat them as something entirely new. But when we overestimate the automation of these tools, award-winning author Antonio A. Casilli argues, we fail to recognize how our fellow humans are essential to their efficiency. The danger is not that robots will take our jobs, but that humans will have to do theirs. In this bracing and powerful book, Antonia A. Casilli uses up-to-the-minute research to show how today's technologies, including AI, continue to exploit human labor—even ours. He connects the diverse activities of today's tech laborers: platform workers, like Uber drivers and Airbnb hosts; “micro workers,” including those performing atomized tasks like data entry on Amazon Mechanical Turk; and the rest of us, as we evaluate text or images to show we're not robots, react to Facebook posts, or approve or improve the output of generative AI. As Casilli shows us, algorithms, search engines, and voice assistants wouldn't function without unpaid or underpaid human contributions. Further, he warns that if we fail to recognize this human work, we risk a dark future for all human labor. Waiting for Robots: The Hired Hands of Automation (U Chicago Press, 2025) urges us to move beyond the simplistic notion that machines are intelligent and autonomous. As the proverbial Godot, robots are the bearers of a messianic promise that is always postponed. Instead of bringing prosperity for all, they discipline the workforce, so we don't dream of a world without drudgery and exploitation. Casilli's eye-opening book makes clear that most “automation” requires human labor—and likely always will—shedding new light on today's consequences and tomorrow's threats of failing to recognize and compensate the “click workers” of today. Michael O. Johnston, Ph.D. is a Assistant Professor of Sociology at William Penn University. He is the author of The Social Construction of a Cultural Spectacle: Floatzilla (Lexington Books, 2023) and Community Media Representations of Place and Identity at Tug Fest: Reconstructing the Mississippi River (Lexington Books, 2022). His general area of study is at the intersection of space, behavior, and identity. He is currently conducting research about the negotiation that humans make between oneself, identification of place, and the attachment/s they have to those places. To learn more about Michael O. Johnston you can go to his personal website, Google Scholar, Bluesky (@professorjohnst.bsky.social),Twitter (@ProfessorJohnst), or by email (johnstonmo@wmpenn.edu) Learn more about your ad choices. Visit megaphone.fm/adchoices Support our show by becoming a premium member! https://newbooksnetwork.supportingcast.fm/sociology

Artificial Intelligence fuels both enthusiasm and panic. Technologists are inclined to give their creations leeway, pretend they're animated beings, and consider them efficient. As users, we may complain when these technologies don't obey, or worry about their influence on our choices and our livelihoods. And yet, we also yearn for their convenience, see ourselves reflected in them, and treat them as something entirely new. But when we overestimate the automation of these tools, award-winning author Antonio A. Casilli argues, we fail to recognize how our fellow humans are essential to their efficiency. The danger is not that robots will take our jobs, but that humans will have to do theirs. In this bracing and powerful book, Antonia A. Casilli uses up-to-the-minute research to show how today's technologies, including AI, continue to exploit human labor—even ours. He connects the diverse activities of today's tech laborers: platform workers, like Uber drivers and Airbnb hosts; “micro workers,” including those performing atomized tasks like data entry on Amazon Mechanical Turk; and the rest of us, as we evaluate text or images to show we're not robots, react to Facebook posts, or approve or improve the output of generative AI. As Casilli shows us, algorithms, search engines, and voice assistants wouldn't function without unpaid or underpaid human contributions. Further, he warns that if we fail to recognize this human work, we risk a dark future for all human labor. Waiting for Robots: The Hired Hands of Automation (U Chicago Press, 2025) urges us to move beyond the simplistic notion that machines are intelligent and autonomous. As the proverbial Godot, robots are the bearers of a messianic promise that is always postponed. Instead of bringing prosperity for all, they discipline the workforce, so we don't dream of a world without drudgery and exploitation. Casilli's eye-opening book makes clear that most “automation” requires human labor—and likely always will—shedding new light on today's consequences and tomorrow's threats of failing to recognize and compensate the “click workers” of today. Michael O. Johnston, Ph.D. is a Assistant Professor of Sociology at William Penn University. He is the author of The Social Construction of a Cultural Spectacle: Floatzilla (Lexington Books, 2023) and Community Media Representations of Place and Identity at Tug Fest: Reconstructing the Mississippi River (Lexington Books, 2022). His general area of study is at the intersection of space, behavior, and identity. He is currently conducting research about the negotiation that humans make between oneself, identification of place, and the attachment/s they have to those places. To learn more about Michael O. Johnston you can go to his personal website, Google Scholar, Bluesky (@professorjohnst.bsky.social),Twitter (@ProfessorJohnst), or by email (johnstonmo@wmpenn.edu) Learn more about your ad choices. Visit megaphone.fm/adchoices Support our show by becoming a premium member! https://newbooksnetwork.supportingcast.fm/science-technology-and-society

Artificial Intelligence fuels both enthusiasm and panic. Technologists are inclined to give their creations leeway, pretend they're animated beings, and consider them efficient. As users, we may complain when these technologies don't obey, or worry about their influence on our choices and our livelihoods. And yet, we also yearn for their convenience, see ourselves reflected in them, and treat them as something entirely new. But when we overestimate the automation of these tools, award-winning author Antonio A. Casilli argues, we fail to recognize how our fellow humans are essential to their efficiency. The danger is not that robots will take our jobs, but that humans will have to do theirs. In this bracing and powerful book, Antonia A. Casilli uses up-to-the-minute research to show how today's technologies, including AI, continue to exploit human labor—even ours. He connects the diverse activities of today's tech laborers: platform workers, like Uber drivers and Airbnb hosts; “micro workers,” including those performing atomized tasks like data entry on Amazon Mechanical Turk; and the rest of us, as we evaluate text or images to show we're not robots, react to Facebook posts, or approve or improve the output of generative AI. As Casilli shows us, algorithms, search engines, and voice assistants wouldn't function without unpaid or underpaid human contributions. Further, he warns that if we fail to recognize this human work, we risk a dark future for all human labor. Waiting for Robots: The Hired Hands of Automation (U Chicago Press, 2025) urges us to move beyond the simplistic notion that machines are intelligent and autonomous. As the proverbial Godot, robots are the bearers of a messianic promise that is always postponed. Instead of bringing prosperity for all, they discipline the workforce, so we don't dream of a world without drudgery and exploitation. Casilli's eye-opening book makes clear that most “automation” requires human labor—and likely always will—shedding new light on today's consequences and tomorrow's threats of failing to recognize and compensate the “click workers” of today. Michael O. Johnston, Ph.D. is a Assistant Professor of Sociology at William Penn University. He is the author of The Social Construction of a Cultural Spectacle: Floatzilla (Lexington Books, 2023) and Community Media Representations of Place and Identity at Tug Fest: Reconstructing the Mississippi River (Lexington Books, 2022). His general area of study is at the intersection of space, behavior, and identity. He is currently conducting research about the negotiation that humans make between oneself, identification of place, and the attachment/s they have to those places. To learn more about Michael O. Johnston you can go to his personal website, Google Scholar, Bluesky (@professorjohnst.bsky.social),Twitter (@ProfessorJohnst), or by email (johnstonmo@wmpenn.edu) Learn more about your ad choices. Visit megaphone.fm/adchoices Support our show by becoming a premium member! https://newbooksnetwork.supportingcast.fm/technology

Artificial Intelligence fuels both enthusiasm and panic. Technologists are inclined to give their creations leeway, pretend they're animated beings, and consider them efficient. As users, we may complain when these technologies don't obey, or worry about their influence on our choices and our livelihoods. And yet, we also yearn for their convenience, see ourselves reflected in them, and treat them as something entirely new. But when we overestimate the automation of these tools, award-winning author Antonio A. Casilli argues, we fail to recognize how our fellow humans are essential to their efficiency. The danger is not that robots will take our jobs, but that humans will have to do theirs. In this bracing and powerful book, Antonia A. Casilli uses up-to-the-minute research to show how today's technologies, including AI, continue to exploit human labor—even ours. He connects the diverse activities of today's tech laborers: platform workers, like Uber drivers and Airbnb hosts; “micro workers,” including those performing atomized tasks like data entry on Amazon Mechanical Turk; and the rest of us, as we evaluate text or images to show we're not robots, react to Facebook posts, or approve or improve the output of generative AI. As Casilli shows us, algorithms, search engines, and voice assistants wouldn't function without unpaid or underpaid human contributions. Further, he warns that if we fail to recognize this human work, we risk a dark future for all human labor. Waiting for Robots: The Hired Hands of Automation (U Chicago Press, 2025) urges us to move beyond the simplistic notion that machines are intelligent and autonomous. As the proverbial Godot, robots are the bearers of a messianic promise that is always postponed. Instead of bringing prosperity for all, they discipline the workforce, so we don't dream of a world without drudgery and exploitation. Casilli's eye-opening book makes clear that most “automation” requires human labor—and likely always will—shedding new light on today's consequences and tomorrow's threats of failing to recognize and compensate the “click workers” of today. Michael O. Johnston, Ph.D. is a Assistant Professor of Sociology at William Penn University. He is the author of The Social Construction of a Cultural Spectacle: Floatzilla (Lexington Books, 2023) and Community Media Representations of Place and Identity at Tug Fest: Reconstructing the Mississippi River (Lexington Books, 2022). His general area of study is at the intersection of space, behavior, and identity. He is currently conducting research about the negotiation that humans make between oneself, identification of place, and the attachment/s they have to those places. To learn more about Michael O. Johnston you can go to his personal website, Google Scholar, Bluesky (@professorjohnst.bsky.social),Twitter (@ProfessorJohnst), or by email (johnstonmo@wmpenn.edu) Learn more about your ad choices. Visit megaphone.fm/adchoices Support our show by becoming a premium member! https://newbooksnetwork.supportingcast.fm/book-of-the-day

Microwork is paid work which usually involves short and repetitive tasks carried out on a smartphone or computer. It could be identifying objects shown in an image, watching videos, labelling data, translating short sentences or recording one's voice for example. It can also be charging electric scooters or taking photos of products for an app. It's as simple as registering on a platform which acts as the middle man between workers and companies. Amazon Mechanical Turk is an example of one such platform. That sounds simple enough; what is life like as a microworker then? In under 3 minutes, we answer your questions! To listen to the last episodes, you can click here: How does pollution affect my mental health? What is the soft evening concept from Tiktok? What is Hugh Jackman's 85% rule? A podcast written and realised by Joseph Chance. In partnership with upday UK. Learn more about your ad choices. Visit megaphone.fm/adchoices

Microwork is paid work which usually involves short and repetitive tasks carried out on a smartphone or computer. It could be identifying objects shown in an image, watching videos, labelling data, translating short sentences or recording one's voice for example. It can also be charging electric scooters or taking photos of products for an app. It's as simple as registering on a platform which acts as the middle man between workers and companies. Amazon Mechanical Turk is an example of one such platform. That sounds simple enough; what is life like as a microworker then? In under 3 minutes, we answer your questions! To listen to the last episodes, you can click here: How does pollution affect my mental health? What is the soft evening concept from Tiktok? What is Hugh Jackman's 85% rule? A podcast written and realised by Joseph Chance. In partnership with upday UK. Learn more about your ad choices. Visit megaphone.fm/adchoices

Episode: 2765 The Mechanical Turk.  Today, the chess playing automaton.

Amazon Mechanical Turk è un servizio di crowdworking nato nel 2005, attraverso il quale si commissionano a persone i lavori che le macchine non sono in grado di fare. Lavori come correggere errori di battitura, riconoscere immagini, inserire sottotitoli, trascrivere registrazioni, rispondere a sondaggi. A svolgerli, ci pensano schiere di invisibili senza contratti né tutele, incollati agli schermi dei computer, retribuiti pochissimo, a volte solo con buoni d'acquisto. Li chiamano turkers alludendo al Turco meccanico, l'automa – imbattibile campione di scacchi – che incantò l'Europa e l'America all'inizio dell'Ottocento. E che nascondeva dentro di sé il segreto di un “pilota” umano. I turkers sono il rimosso di carne e ossa dietro l'economia delle app: il simbolo del nuovo sfruttamento al tempo degli algoritmi. Learn more about your ad choices. Visit megaphone.fm/adchoices

We're trying a new format, inspired by Acquired.fm! No guests, no news, just highly prepared, in-depth conversation on one topic that will level up your understanding. We aren't experts, we are learning in public. Please let us know what we got wrong and what you think of this new format!When you ask someone to break down the basic ingredients of a Large Language Model, you'll often hear a few things: You need lots of data. You need lots of compute. You need models with billions of parameters. Trust the Bitter Lesson, more more more, scale is all you need. Right?Nobody ever mentions the subtle influence of great benchmarking.LLM Benchmarks mark our progress in building artificial intelligences, progressing from * knowing what words go with others (1985 WordNet)* recognizing names and entities (2004 Enron Emails) * and image of numbers, letters, and clothes (1998-2017 MNIST)* language translation (2002 BLEU → 2020 XTREME)* more and more images (2009 ImageNet, CIFAR)* reasoning in sentences (2016 LAMBADA) and paragraphs (2019 AI2RC, DROP)* stringing together whole sentences (2018 GLUE and SuperGLUE)* question answering (2019 CoQA)* having common sense (2018 Swag and HellaSwag, 2019 WinoGrande)* knowledge of all human tasks and professional exams (2021 MMLU)* knowing everything (2022 BIG-Bench)People who make benchmarks are the unsung heroes of LLM research, because they dream up ever harder tests that last ever shorter periods of time.In our first AI Fundamentals episode, we take a trek through history to try to explain what we have learned about LLM Benchmarking, and what issues we have discovered with them. There are way, way too many links and references to include in this email. You can follow along the work we did for our show prep in this podcast's accompanying repo, with all papers and selected tests pulled out.Enjoy and please let us know what other fundamentals topics you'd like us to cover!Timestamps* [00:00:21] Benchmarking Questions* [00:03:08] Why AI Benchmarks matter* [00:06:02] Introducing Benchmark Metrics* [00:08:14] Benchmarking Methodology* [00:09:45] 1985-1989: WordNet and Entailment* [00:12:44] 1998-2004 Enron Emails and MNIST* [00:14:35] 2009-14: ImageNet, CIFAR and the AlexNet Moment for Deep Learning* [00:17:42] 2018-19: GLUE and SuperGLUE - Single Sentence, Similarity and Paraphrase, Inference* [00:23:21] 2018-19: Swag and HellaSwag - Common Sense Inference* [00:26:07] Aside: How to Design Benchmarks* [00:26:51] 2021: MMLU - Human level Professional Knowledge* [00:29:39] 2021: HumanEval - Code Generation* [00:31:51] 2020: XTREME - Multilingual Benchmarks* [00:35:14] 2022: BIG-Bench - The Biggest of the Benches* [00:37:40] EDIT: Why BIG-Bench is missing from GPT4 Results* [00:38:25] Issue: GPT4 vs the mystery of the AMC10/12* [00:40:28] Issue: Data Contamination* [00:42:13] Other Issues: Benchmark Data Quality and the Iris data set* [00:45:44] Tradeoffs of Latency, Inference Cost, Throughput* [00:49:45] ConclusionTranscript[00:00:00] Hey everyone. Welcome to the Latent Space Podcast. This is Alessio, partner and CTO and residence at Decibel Partners, and I'm joined by my co-host, swyx writer and editor of Latent Space.[00:00:21] Benchmarking Questions[00:00:21] Up until today, we never verified that we're actually humans to you guys. So we'd have one good thing to do today would be run ourselves through some AI benchmarks and see if we are humans.[00:00:31] Indeed. So, since I got you here, Sean, I'll start with one of the classic benchmark questions, which is what movie does this emoji describe? The emoji set is little Kid Bluefish yellow, bluefish orange Puffer fish. One movie does that. I think if you added an octopus, it would be slightly easier. But I prepped this question so I know it's finding Nemo.[00:00:57] You are so far a human. Second one of these emoji questions instead, depicts a superhero man, a superwoman, three little kids, one of them, which is a toddler. So you got this one too? Yeah. It's one of my favorite movies ever. It's the Incredibles. Uh, second one was kind of a letdown, but the first is a.[00:01:17] Awesome. Okay, I'm gonna ramp it up a little bit. So let's ask something that involves a little bit of world knowledge. So when you drop a ball from rest, it accelerates downward at 9.8 meters per second if you throw it downward instead, assuming no air resistance, so you're throwing it down instead of dropping it, it's acceleration immediately after leaving your hand is a 9.8 meters per second.[00:01:38] B, more than 9.8 meters per second. C less than 9.8 meters per second. D cannot say unless the speed of the throw is. I would say B, you know, I started as a physics major and then I changed, but I think I, I got enough from my first year. That is B Yeah. Even proven that you're human cuz you got it wrong.[00:01:56] Whereas the AI got it right is 9.8 meters per second. The gravitational constant, uh, because you are no longer accelerating after you leave the hand. The question says if you throw it downward after leaving your hand, what is the. It is, it goes back to the gravitational constant, which is 9.8 meters per, I thought you said you were a physics major.[00:02:17] That's why I changed. So I'm a human. I'm a human. You're human. You're human. But you, you got them all right. So I can't ramp it up. I can't ramp it up. So, Assuming, uh, the AI got all of that right, you would think that AI will get this one wrong. Mm-hmm. Because it's just predicting the next token, right?[00:02:31] Right. In the complex Z plane, the set of points satisfying the equation. Z squared equals modulars. Z squared is A, a pair points B circle, C, a half line D, online D square. The processing is, this is going on in your head. You got minus three. A line. This is hard. Yes, that is. That is a line. Okay. What's funny is that I think if, if an AI was doing this, it would take the same exact amount of time to answer this as it would every single other word.[00:03:05] Cuz it's computationally the same to them. Right.[00:03:08] Why AI Benchmarks matter[00:03:08] Um, so anyway, if you haven't caught on today, we're doing our first, uh, AI fundamentals episode, which just the two of us, no guess because we wanted to go deep on one topic and the topic. AI benchmarks. So why are we focusing on AI benchmarks? So, GPT4 just came out last week and every time a new model comes out, All we hear about is it's so much better than the previous model on benchmark X, on benchmark Y.[00:03:33] It performs better on this, better on that. But most people don't actually know what actually goes on under these benchmarks. So we thought it would be helpful for people to put these things in context. And also benchmarks evolved. Like the more the models improve, the harder the benchmarks get. Like I couldn't even get one of the questions right.[00:03:52] So obviously they're working and you'll see that. From the 1990s where some of the first ones came out to day, the, the difficulty of them is truly skyrocketed. So we wanna give a, a brief history of that and leave you with a mental model on, okay, what does it really mean to do well at X benchmark versus Y benchmark?[00:04:13] Um, so excited to add that in. I would also say when you ask people what are the ingredients going into a large language model, they'll talk to you about the data. They'll talk to you about the neural nets, they'll talk to you about the amount of compute, you know, how many GPUs are getting burned based on this.[00:04:30] They never talk to you about the benchmarks. And it's actually a shame because they're so influential. Like that is the entirety of how we judge whether a language model is better than the other. Cuz a language model can do anything out of. Potentially infinite capabilities. How do you judge one model versus another?[00:04:48] How do you know you're getting better? And so I think it's an area of intense specialization. Also, I think when. Individuals like us, you know, we sort of play with the language models. We are basically doing benchmarks. We're saying, look, it's, it's doing this awesome thing that I found. Guess what? There have been academics studying this for 20 years who have, uh, developed a science to this, and we can actually benefit from studying what they have done.[00:05:10] Yep. And obviously the benchmarks also drive research, you know, in a way whenever you're working on, in a new model. Yeah. The benchmark kind of constraints what you're optimizing for in a way. Because if you've read a paper and it performs worse than all the other models, like you're not gonna publish it.[00:05:27] Yeah. So in a way, there's bias in the benchmark itself. Yeah. Yeah. We'll talk a little bit about that. Right. Are we optimizing for the right things when we over-optimize for a single benchmark over over some others? And also curiously, when GPT4 was released, they emitted some very. Commonplace industry benchmarks.[00:05:44] So the way that you present yourself, it is a form of marketing. It is a form of trying to say you're better than something else. And, and trying to explain where you think you, you do better. But it's very hard to verify as well because there are certain problems with reproducing benchmarks, uh, especially when you come to large language models.[00:06:02] Introducing Benchmark Metrics[00:06:02] So where do we go from here? Should we go over the, the major concept? Yeah. When it comes to benchmark metrics, we get three main measures. Accuracy, precision, recall accuracy is just looking at how many successful prediction the model does. Precision is the ratio of true positives, meaning how many of them are good compared to the overall amount of predictions made Versus recall is what proportion of the positives were identified.[00:06:31] So if you think. Spotify playlist to maybe make it a little more approachable, precision is looking. How many songs in a Spotify playlist did you like versus recall is looking at of all the Spotify songs that you like in the word, how many of them were put in the in the playlist? So it's more looking at how many of the true positives can you actually bring into the model versus like more focusing on just being right.[00:06:57] And the two things are precision and recall are usually in tension.. If you're looking for a higher position, you wanna have a higher percentage of correct results. You're usually bringing recall down because you lead to kind of like lower response sets, you know, so there's always trade offs. And this is a big part of the benchmarking too.[00:07:20] You know, what do you wanna optimize for? And most benchmarks use this, um, F1 score, which is the harmonic mean of precision and recall. Which is, you know, we'll put it in the show notes, but just like two times, like the, you know, precision Times Recall divided by the sum. So that's one. And then you get the Stanford Helm metrics.[00:07:38] Um, yeah, so ultimately I think we have advanced a lot in the, in the past few decades on how we measure language models. And the most interesting one came out January of this year from Percy Lang's research lab at Stanford, and he's got. A few metrics, accuracy, calibration, robustness, fairness, efficiency, general information bias and toxicity, and caring that your language models are not toxic and not biased.[00:08:03] So is is, mm-hmm. Kind of a new thing because we have solved the other stuff, therefore we get to care about the toxic of, uh, the language models yelling at us.[00:08:14] Benchmarking Methodology[00:08:14] But yeah, I mean, maybe we can also talk about the other forms of how their be. Yeah, there's three main modes. You can need a benchmark model in a zero shot fashion, few shot or fine tune models, zero shots.[00:08:27] You do not provide any example and you're just testing how good the model is at generalizing few shots, you have a couple examples that you provide and then. You see from there how good the model is. These are the number of examples usually represented with a K, so you might see few shots, K equal five, it means five examples were passed, and then fine tune is you actually take a bunch of data and fine tune the model for that specific task, and then you test it.[00:08:55] These all go from the least amount of work required to the most amount of work required. If you're doing zero shots benchmarking, you do not need to have any data, so you can just take 'em out and do. If you're fine tuning it, you actually need a lot of data and a lot of compute time. You're expecting to see much better results from there.[00:09:14] Yeah. And sometimes the number of shots can go up to like a hundred, which is pretty surprising for me to see that people are willing to test these language models that far. But why not? You just run the computer a little bit longer. Yeah. Uh, what's next? Should we go into history and then benchmarks? Yeah.[00:09:29] History of Benchmarking since 1985[00:09:29] Okay, so I was up all night yesterday. I was like, this is a fascinating topic. And I was like, all right, I'll just do whatever's in the G PT three paper. And then I read those papers and they all cited previous papers, and I went back and back and back all the way to 1985. The very first benchmark that I can find.[00:09:45] 1985-1989: WordNet and Entailment[00:09:45] Which is WordNet, which is uh, an English benchmark created in at Princeton University by George Miller and Christian Fellbaum. Uh, so fun fact, Chris George Miller also authored the paper, the Magical Number seven plus Minus two, which is the observation that people have a short term memory of about seven for things.[00:10:04] If you have plus or minus two of seven, that's about all you can sort of remember in the short term, and I just wanted. Say like, this was before computers, right? 1985. This was before any of these personal computers were around. I just wanna give people a sense of how much work manual work was being done by these people.[00:10:22] The database, uh, WordNet. Sorry. The WordNet database contains 155,000 words organized in 175,000 sys. These sys are basically just pairings of nouns and verbs and adjectives and adverbs that go together. So in other words, for example, if you have nouns that are hyper names, if every X is a, is a kind of Y.[00:10:44] So a canine is a hyper name of a dog. It's a holo. If X is a part of Y, so a building is a hollow name of a window. The most interesting one for in terms of formal, uh, linguistic logic is entailment, which captures the relationship between two words, where the verb Y is entailed by X. So if by doing X, you must be doing Y.[00:11:02] So in other words, two, sleep is entailed by two snore because you cannot snore without also sleeping and manually mapping 155,000 words like that, the relationships between all of them in a, in a nested tree, which is. Incredible to me. Mm-hmm. And people just did that on faith. They were like, this will be useful somehow.[00:11:21] Right. Uh, and they were interested in cycle linguistics, like understanding how humans thought, but then it turned out that this was a very good dataset for understanding semantic similarity, right? Mm-hmm. Like if you measure the distance between two words by traversing up and down the graph, you can find how similar to two words are, and therefore, Try to figure out like how close they are and trade a model to, to predict that sentiment analysis.[00:11:42] You can, you can see how far something is from something that is considered a good sentiment or a bad sentiment or machine translation from one language to the other. Uh, they're not 200 word languages, which is just amazing. Like people had to do this without computers. Penn Tree Bank, I was in 1989, I went to Penn, so I always give a shout out to my university.[00:12:01] This one expanded to 4.5 million words of text, which is every uh, wall Street Journal. For three years, hand collected, hand labeled by grad students your tuition dollars at work. So I'm gonna skip forward from the eighties to the nineties. Uh, NYS was the most famous data set that came out of this. So this is the, uh, data set of 60,000.[00:12:25] Training images of, uh, of numbers. And this was the first visual dataset where, uh, people were tr tracking like, you know, handwritten numbers and, and mapping them to digital numbers and seeing what the error rate for them was. Uh, these days I think this can be trained in like e every Hello world for machine learning is just train missed in like four lanes of code.[00:12:44] 1998-2004 Enron Emails and MNIST[00:12:44] Then we have the Enron email data set. Enron failed in 2001. Uh, the emails were released in 2004 and they've been upgraded every, uh, every few years since then. That is 600,000 emails by 150 senior employees of Enron, which is really interesting because these are email people emailing each other back and forth in a very natural.[00:13:01] Context not knowing they're being, they're about to be observed, so you can do things like email classification, email summarization, entity recognition and language modeling, which is super cool. Any thoughts about that be before we go into the two thousands? I think like in a way that kind of puts you back to the bias, you know, in some of these benchmarks, in some of these data sets.[00:13:21] You know, like if your main corpus of benchmarking for entity recognition is a public energy company. Mm-hmm. You know, like if you're building something completely different and you're building a model for that, maybe it'll be worse. You know, you start to see how we started. With kind of like, WordNet is just like human linguistics, you know?[00:13:43] Yes. It's not domain related. And then, um, same with, you know, but now we're starting to get into more and more domain-specific benchmarks and you'll see this increase over time. Yeah. NY itself was very biased towards, um, training on handwritten letter. Uh, and handwritten numbers. So, um, in 2017 they actually extended it to Eist, which is an extended to extension to handwritten letters that seems very natural.[00:14:08] And then 2017, they also had fashion ness, which is a very popular data set, which is images of clothing items pulled from Zando. So you can see the capabilities of computer vision growing from single digit, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, to all the letters of the alphabet. To now we can recognize images, uh, of fashion, clothing items.[00:14:28] So it's pretty. So the big one for deep learning, cuz all of that was just, just the appetizers, just getting started.[00:14:35] 2009-2014 : ImageNet, CIFAR and the AlexNet Moment for Deep Learning[00:14:35] The big one for deep learning was ImageNet, which is where Fafa Lee came into the picture and that's why she's super well known. She started working in 2006 and released it in 2009. Fun fact, she actually met with, uh, Christian Feldbaum, who was, uh, one of the co-authors of, uh, war.[00:14:51] To create ImageNet. So there's a direct lineage from Words to Images. Yeah. And uh, they use Amazon Mechanical Turk to help with classification images. No longer grad students. But again, like I think, uh, this goes, kind of goes back to your observation about bias, like when I am a mechanical Turk worker. And I'm being paid by the image to classify an image.[00:15:10] Do you think I'll be very careful at my job? Right? Yeah. Whereas when I'm a, you know, Enron employee, emailing my, my fellow coworker, trying to just communicate something of, of natural language that is a different type of, uh, environment. Mm-hmm. So it's a pretty interesting benchmark. So it was released in 2009 ish and, you know, people were sort of competing to recognize and classify that properly.[00:15:33] The magic moment for ImageNet came in 2012, uh, which is called the AlexNet moment cuz I think that grad student that, um, created this recognition model was, uh, named Alex, I forget his last name, achieved a error rate of 15%, which is, More than 10% lower than the runner up. So it was used just so much better than the second place that everyone else was like, what are you doing?[00:15:54] Uh, and it turned out that he was, he was the first to use, uh, deep learning, uh, c n n 10 percentage points. So like 15 and the other one was 25. Yeah, exactly. So it was just so much, so much better than the others. It was just unbelievable that no one else was, no other approach was even coming close.[00:16:09] Therefore, everyone from there on out for the next, until today we're just learning the lessons of deep learning because, um, it is so much superior to the other approaches. And this was like a big. Images and visual moment because then you had like a sci-fi 10, which is a, another, like a data set that is mostly images.[00:16:27] Mm-hmm. Focused. Mm-hmm. So it took a little bit before we got back to to text. And nowadays it feels like text, you know, text models are kind of eating the word, you know, we're making the text one multi-model. Yeah. So like we're bringing the images to GBT four instead of the opposite. But yeah, in 2009 we had a, another 60,000 images that set.[00:16:46] 32 by 32. Color images with airplanes, automobiles, like, uh, animals, like all kind of stuff. Like I, I think before we had the numbers, then we had the handwritten letters. Then we had clothing, and then we finally made clothing items came after, oh, clothing items. 2009. Yeah, this is 2009. I skipped, I skipped time a little bit.[00:17:08] Yeah, yeah. But yeah, CFR 10 and CFR 100. CFR 10 was for 10 classes. And that that was chosen. And then obviously they optimized that and they were like, all right, we need a new problem now. So in 20 14, 5 years later, they introduced CFAR 100, which was a hundred classes of other items. And I think this is a very general pattern, which is used.[00:17:25] You create a data set for a specific be. You think it's too hard for machines? Mm-hmm. It lasts for five years before it's no longer too hard for machines, and you have to find a new data set and you have to extend it again. So it's Similarly, we are gonna find that in glue, which is another, which is one of more modern data sets.[00:17:42] 2018-19: GLUE and SuperGLUE - Single Sentence, Similarity and Paraphrase, Inference[00:17:42] This one came out in 2018. Glue stands for general Language Understanding Evaluation. This is one of the most influential, I think, early. Earlier, um, language model benchmarks, and it has nine tasks. Um, so it has single sentence tasks, similarity and paraphrase tasks and inference tasks. So a single sentence task, uh, would be something like, uh, the Stanford Sentiment Tree Bank, which is a.[00:18:05] Uh, sentences from movie reviews and human annotations of the sentiment, whether it's positive or negative, in a sort of like a four point scale. And your job is to predict the task of a single sentence. This similarity task would involve corpuses, like the Microsoft research paraphrase corpus. So it's a corpus of sentence pairs automatically extracted from online news sources with human annotations for whether or not the sentence is in the para semantically equivalent.[00:18:28] So you just predict true or false and again, Just to call back to the math that we did earlier in this episode, the classes here are imbalance. This data set, for example, is 68% positive. So we report both accuracy and F1 scores. F1 is a more balanced approach because it, it adjusts for, uh, imbalanced, um, data sets.[00:18:48] Mm-hmm. Yeah. And then finally, inference. Inference is the one where we really start to have some kind of logic. So for example, the M N L I. Um, actually I'm, I'm gonna focus on squad, the Stanford questioning question answering dataset. It's another data set of pairs, uh, questions, uh, uh, p question paragraphs, pairs.[00:19:04] So where one of the sentences of the paragraph drawn from Wikipedia contains the answer to the corresponding question, we convert the task into a sentence, para classification by forming a pair between each question in each sentence into corresponding context and filtering out pairs of low overlap. So basically annotating whether or not.[00:19:20] Is the answer to the question inside of this paragraph that I pulled. Can you identify that? And again, like Entailment is kind of included inside of each of these inference tasks because it starts to force the language model to understand whether or not one thing implies the other thing. Mm-hmm. Yeah.[00:19:37] And the, the models evolving. This came out in 2018, lasted one year exactly. One year later, people were like, that's too easy. That's too easy. So in 2019, they actually came out with super. I love how you'll see later with like swag and hella swag. It's like they come up with very good names for these things.[00:19:55] Basically what's super glue dead is stick glue and try and move outside of the single sentence evaluation. So most of the tasks that. Sean was talking about focus on one sentence. Yeah, one sentence, one question. It's pretty straightforward in that way. Superglue kind of at the, so one, it went from single sentence to having some multi sentence and kind of like a context driven thing.[00:20:21] So you might have questions where, The answer is not in the last paragraph that you've read. So it starts to test the, the context window on this model. Some of them are more, in order to know the answer, you need to know what's not in the question kind of thing. So like you may say, Hey, this drink is owned by the Coca-Cola company.[00:20:43] Is this a Pepsi product? You know, so you need to make the connection false. Exactly, yeah. Then you have also like, um, embedded clauses. So you have things that are not exactly said, have to be inferred, and like a lot of this stack is very conversational. So some of the example contain a lot of the, um, um, you know, or this question's very hard to read out.[00:21:07] Yeah, I know. It's like, it sounds like you are saying, um, but no, you're actually, you're actually. And yet I hope to see employer base, you know, helping out child, um, care centers at the place of employment, things like that, that will help out. It's kind of hard to even read it. And then the hypothesis is like they're setting a trend.[00:21:27] It's going from something very simple like a big p d extract to something that is more similar to how humans communicate. Transcripts, like audio transcripts. Exactly. Of how people talk. Yeah. And some of them are also, Plausibility. You know, like most of these models have started to get good at understanding like a clear cause, kind of like a.[00:21:48] You know, cause effect things. But some of the plausible ones are like, for example, this one is a copa. They're called choice of plausible alternatives. The premises, my body cast a shadow over the grass. What's the cost for this alternative? One, the sun was rising. Alternative to the grass was cut.[00:22:07] Obviously it's the sun was rising, but nowhere. In the question we're actually mentioning the sun, uh, we are mentioning the grass. So some models, some of the older models might see the grass and make the connection that the grass is part of the reason, but the models start to get better and better and go from simply looking at the single sentence context to a more of a, a word new, uh, word knowledge.[00:22:27] It's just really impressive, like the fact that. We can expect that out of a model. It still blows my mind. I think we should not take it for granted that when we're evaluating models, we're asking questions like this that is not obvious from just the given text itself. Mm-hmm. So it, it is just coming with a memorized view of the world, uh, or, or world knowledge. And it understands the premise on, on some form. It is not just random noise. Yeah, I know. It's really impressive. This one, I actually wanted multi rc I actually wanted to spring on you as a, as a test, but it's just too long to read. It's just like a very long logic question.[00:23:03] And then it'll ask you to do, uh, comprehension. But uh, yeah, we'll just, we'll just kinda skip that. We'll put it, we'll put it in the show notes, and then you have to prove us that you're a human. Send us the answer exactly. Exactly and subscribe to the podcast. So superglue was a lot harder, and I think also was superseded eventually, pretty soon.[00:23:21] 2018-2019: Swag and HellaSwag - Common Sense Inference[00:23:21] And, uh, yeah, then we started coming onto the more recent cohort of tests. I don't know how to introduce the rest. Uh, there, there are just so many tests here that I, I struggle a little bit picking from these. Uh, but perhaps we can talk about swag and heli swyx since you mentioned it. Yeah. So SWAG stands for situations with Adversarial Generations.[00:23:39] Uh, also came out in 2018, but this guy, zes Etal, likes to name his data sets and his benchmarks in a very memorable way. And if you look at the PDF of the paper, he also has a little icon, uh, image icon for swag. And he doesn't just go by, uh, regular language. So he definitely has a little bit of branding to this and it's.[00:24:00] Part. So I'll give you an example of the kind of problems that swyx poses. Uh, it it is focused on common sense inference. So what's common sense inference? So, for example, given a partial description, like she opened the hood of the car, humans can reason about the situation and anticipate what might come next.[00:24:16] Then she examined the engine. So you're supposed to pick based on what happened in the first part. What is most likely to happen in the second part based on the, uh, multiple choice question, right? Another example would be on stage, a woman takes a seat at the piano. She a, sits on a bench as her sister plays at the doll.[00:24:33] B. Smiles with someone as the music play. C is in the crowd watching the dancers. D nervously set her fingers on the keys, so A, B, C, or D. It's not all of them are plausible. When you look at the rules of English, we're we've, we're not even checking for whether or not produces or predicts grammatical English.[00:24:54] We're checking for whether the language model can correctly pick what is most likely given the context. The only information that you're given is on stage. A woman takes a seat at the piano, what is she most likely to do next? And D makes sense. It's arguable obviously. Sometimes it could be a. In common sense, it's D.[00:25:11] Mm-hmm. So we're training these models to have common. Yeah, which most humans don't have. So it's a, it's already a step up. Obviously that only lasted a year. Uh, and hello, SWAG was no longer, was no longer challenging in 2019, and they started extending it quite a lot more, a lot more questions. I, I forget what, how many questions?[00:25:33] Um, so Swag was a, swag was a data set. A hundred thousand multiple choice questions. Um, and, and part of the innovation of swag was really that you're generating these questions rather than manually coming up with them. Mm-hmm. And we're starting to get into not just big data, but big questions and big benchmarks of the, of the questions.[00:25:51] That's where the adversarial generations come in, but how that swag. Starts pulling in from real world questions and, and data sets like, uh, wikiHow and activity net. And it's just really, you know, an extension of that. I couldn't even add examples just cuz there's so many. But just to give you an idea of, uh, the progress over time.[00:26:07] Aside: How to Design Benchmarks[00:26:07] Most of these benchmarks are, when they're released, they set. Benchmark at a level where if you just randomly guessed all of the questions, you'll get a 25%. That's sort of the, the baseline. And then you can run each of the language models on them, and then you can run, uh, human evaluations on them. You can have median evaluations, and then you have, um, expert evaluations of humans.[00:26:28] So the randoms level was, uh, for halla. swyx was 20. GT one, uh, which is the, uh, 2019 version that got a 41 on the, on the Hello Sue X score. Bert from Google, got 47. Grover, also from Google, got 57 to 75. Roberta from Facebook, got 85 G P T, 3.5, got 85, and then GPT4 got 95 essentially solving hello swag. So this is useless too.[00:26:51] 2021 - MMLU - Human level Professional Knowledge[00:26:51] We need, we need super Hell now's use this. Super hell swyx. I think the most challenging one came from 2021. 2021 was a very, very good year in benchmarking. So it's, we had two major benchmarks that came out. Human eval and M M L U, uh, we'll talk about mm. M L U first, cuz that, that's probably the more, more relevant one.[00:27:08] So M M L U. Stands for measuring mul massive multitask language understanding, just by far the biggest and most comprehensive and most human-like, uh, benchmark that we've had for until 2021. We had a better one in 2022, but we'll talk about that. So it is a test that covers 57 tasks, including elementary, math, US history, computer science law, and more.[00:27:29] So to attain high accuracy on this task, models must possess extensive world knowledge and prop problem solving. Its. Includes practice questions for the GRE test and the U United States, um, m l e, the medical exam as. It also includes questions from the undergrad courses from Oxford, from all the way from elementary high school to college and professional.[00:27:49] So actually the opening question that I gave you for this podcast came from the math test from M M L U, which is when you drop a ball from rest, uh, what happens? And then also the question about the Complex Z plane, uh, but it equally is also asking professional medicine question. So asking a question about thyroid cancer and, uh, asking you to diagnose.[00:28:10] Which of these four options is most likely? And asking a question about microeconomics, again, giving you a, a situation about regulation and monopolies and asking you to choose from a list of four questions. Mm-hmm. Again, random baseline is 25 out of 100 G P T two scores, 32, which is actually pretty impressive.[00:28:26] GT three scores between 43 to 60, depending on the the size. Go. Scores 60, chinchilla scores 67.5, GT 3.5 scores, 70 GPT4 jumps, one in 16 points to 86.4. The author of M M L U, Dan Hendrix, uh, was commenting on GPT4 saying this is essentially solved. He's basically says like, GT 4.5, the, the next incremental improvement on GPT4 should be able to reach expert level human perform.[00:28:53] At which point it is passing simultaneously, passing all the law exams, all the medical exams, all the graduate student exams, every single test from AP history to computer science to. Math to physics, to economics. It's very impressive. Yeah. And now you're seeing, I mean, it's probably unrelated, but Ivy League universities starting to drop the a t as a requirement for getting in.[00:29:16] So yeah. That might be unrelated as well, because, uh, there's a little bit of a culture war there with regards to, uh, the, the inherent bias of the SATs. Yeah. Yeah. But I mean, that's kinda, I mean exactly. That's kinda like what we were talking about before, right? It's. If a model can solve all of these, then like how good is it really?[00:29:33] How good is it as a Exactly. Telling us if a person should get in. It captures it. Captures with just the beginning. Yeah. Right.[00:29:39] 2021: HumanEval - Code Generation[00:29:39] Well, so I think another significant. Benchmark in 2021 was human eval, which is, uh, the first like very notable benchmark for code code generation. Obviously there's a, there's a bunch of research preceding this, but this was the one that really caught my eye because it was simultaneously introduced with Open Eyes Codex, which is the code generation model, the version of G P T that was fine tuned for generating code.[00:30:02] Uh, and that is, Premise of, well, there is the origin or the the language model powering GitHub co-pilot and yeah, now we can write code with language models, just with that, with that benchmark. And it's good too. That's the other thing, I think like this is one where the jump from GT 3.5 to GPT4 was probably the biggest, like GT 3.4 is like 48% on. On this benchmark, GPT4 is 67%. So it's pretty big. Yeah. I think coders should rest a little bit. You know, it's not 90 something, it's, it's still at 67, but just wait two years. You know, if you're a lawyer, if you're a lawyer, you're done. If you're a software engineer, you got, you got a couple more years, so save your money.[00:30:41] Yeah. But the way they test it is also super creative, right? Like, I think maybe people don't understand that actually all of the tests that are given here are very intuitive. Like you. 90% of a function, and then you ask the language model to complete it. And if it completes it like any software engineer would, then you give it a win.[00:31:00] If not, you give it a loss, run that model 164 times, and that is human eval. Yeah. Yeah. And since a lot of our listeners are engineers too, I think the big thing here is, and there was a, a link that we had that I missed, but some of, for example, some of. Coding test questions like it can answer older ones very, very well.[00:31:21] Like it doesn't not answer recent ones at all. So like you see some of like the data leakage from the training, like since it's been trained on the issues, massive data, some of it leaks. So if you're a software engineer, You don't have to worry too much. And hopefully, especially if you're not like in the JavaScript board, like a lot of these frameworks are brand new every year.[00:31:41] You get a lot of new technologies. So there's Oh, there's, oh yeah. Job security. Yes, exactly. Of course. Yeah. You got a new, you have new framework every year so that you have job security. Yeah, exactly. I'll sample, uh, data sets.[00:31:51] 2020 - XTREME - Multilingual Benchmarks[00:31:51] So before we get to big bench, I'll mention a couple more things, which is basically multilingual benchmarks.[00:31:57] Uh, those are basically simple extensions of monolingual benchmarks. I feel like basical. If you can. Accurately predicts the conversion of one word or one part of the word to another part of the word. Uh, you get a score. And, and I think it's, it's fairly intuitive over there. Uh, but I think the, the main benchmarks to know are, um, extreme, which is the, uh, x the x lingual transfer evaluation, the multilingual encoders, and much prefer extreme.[00:32:26] I know, right? Uh, that's why, that's why they have all these, uh, honestly, I think they just wanted the acronym and then they just kinda worked backwards. And then the other one, I can't find it in my notes for, uh, what the other multilingual ones are, but I, I just think it's interesting to always keep in mind like what the other.[00:32:43] Language capabilities are like, one language is basically completely equivalent to another. And I think a lot of AI ethicists or armchair AI ethicists are very angry that, you know, most of the time we optimize for English because obviously that has, there's the most, uh, training corpuses. I really like extreme the work that's being done here, because they took a, a huge amount of effort to make sure they cover, uh, sparse languages like the, the less popular ones.[00:33:06] So they had a lot of, uh, the, the, obviously the, the popular. Uh, the world's top languages. But then they also selected to maximize language diversity in terms of the complete diversity in, uh, human languages like Tamil Telugu, maam, and Sohi and Yoruba from Africa. Mm-hmm. So I just thought like that kind of effort is really commendable cuz uh, that means that the rest of the world can keep up in, in this air race.[00:33:28] Right. And especially on a lot of the more human based things. So I think we talked about this before, where. A lot of Israel movies are more[00:33:36] focused on culture and history and like are said in the past versus a lot of like the Western, did we talk about this on the podcast? No, not on the podcast. We talked and some of the Western one are more focused on the future and kind of like what's to come.[00:33:48] So I feel like when you're, some of the benchmarks that we mentioned before, you know, they have movie reviews as like, uh, one of the. One of the testing things. Yeah. But there's obviously a big cultural difference that it's not always captured when you're just looking at English data. Yeah. So if you ask the a motto, it's like, you know, are people gonna like this movie that I'm writing about the future?[00:34:10] Maybe it's gonna say, yeah, that's a really good idea. Or if I wanna do a movie about the past, it's gonna be like maybe people want to hear about robots. But that wouldn't be the case in, in every country. Well, since you and I speak different languages, I speak Chinese, you speak Italian, I'm sure you've tested the Italian capabilities.[00:34:29] What do you think? I think like as. Italy, it's so much more, um, dialect driven. So it can be, it can be really hard. So what kind of Italian does g PT three speak? Actually Italian, but the reality is most people have like their own, their own like dialect. So it would be really hard for a model to fool. An Italian that it's like somebody from where they are, you know?[00:34:49] Yeah. Like you can actually tell if you're speaking to AI bot in Chinese because they would not use any of the things that human with humans would use because, uh, Chinese humans would use all sorts of replacements for regular Chinese words. Also, I tried one of those like language tutor things mm-hmm.[00:35:06] That people are making and they're just not good Chinese. Not colloquial Chinese, not anything that anyone would say. They would understand you, but they were from, right, right.[00:35:14] 2022: BIG-Bench - The Biggest of the Benches[00:35:14] So, 2022, big bench. This was the biggest of the biggest, of the biggest benchmarks. I think the, the main pattern is really just, Bigger benchmarks rising in opposition to bigger and bigger models.[00:35:27] In order to evaluate these things, we just need to combine more and more and way more tasks, right? Like swag had nine tasks, hello swag had nine more tasks, and then you're, you're just adding and adding and adding and, and just running a battery of tasks all over. Every single model and, uh, trying to evaluate how good they are at each of them.[00:35:43] Big bench was 204 tasks contributed by 442 authors across 132 institutions. The task topics are diverse, drawing from linguistics, childhood development, math, common sense reasoning, biology, physics, social bias, software development, and beyond. I also like the fact that these authors also selected tasks that are not solved by current language models, but also not solvable by memorizing the internet, which is mm-hmm.[00:36:07] Tracking back to a little bit of the issues that we're, we're gonna cover later. Right. Yeah. I think that's, that's super interesting. Like one of, some of the examples would include in the following chess position, find a checkmate, which is, some humans cannot do that. What is the name of the element within a topic number of six?[00:36:22] Uh, that one you can look up, right? By consulting a periodic table. We just expect language models to memorize that. I really like this one cuz it's, uh, it's inherent. It's, uh, something that you can solve.[00:36:32] Identify whether this sentence has an anachronism. So, option one. During the Allied bombardment of the beaches of Iwojima, Ralph spoke loudly into his radio.[00:36:41] And in option two, during the allied bombardment of the beaches of Iwojima, Ralph spoke loudly into his iPhone. And you have to use context of like when iPhone, when Ally bombarding. Mm-hmm. And then sort of do math to like compare one versus the other and realize that okay, this one is the one that's out of place.[00:36:57] And that's asking more and more and more of the language model to do in implicitly, which is actually modeling what we do when we listen to language, which is such a big. Gap. It's such a big advancement from 1985 when we were comparing synonyms. Mm-hmm. Yeah, I know. And it's not that long in the grand scheme of like humanity, you know, like it's 40 years.[00:37:17] It's crazy. It's crazy. So this is a big missing gap in terms of research. Big benches seems like the most comprehensive, uh, set of benchmarks that we have. But it is curiously missing from Gypsy four. Mm-hmm. I don't know. On paper, for code, I only see Gopher two 80. Yeah. On it. Yeah. Yeah. It could be a curious emission because it maybe looks.[00:37:39] Like it didn't do so well.[00:37:40] EDIT: Why BIG-Bench is missing from GPT4 Results[00:37:40] Hello, this is Swyx from the editing room sometime in the future. I just wanted to interject that. Uh, we now know why the GPT for benchmark results did not include the big bench. Benchmark, even though that was the state-of-the-art benchmark at the time. And that's because the. Uh, GPC four new the Canary G U I D of the big bench.[00:38:02] Benchmark. Uh, so Canary UID is a random string, two, six[00:38:08] eight six B eight, uh, blah, blah, blah. It's a UID. UID, and it should not be knowable by the language model. And in this case it was therefore they had to exclude big bench and that's. And the issue of data contamination, which we're about to go into right now.[00:38:25] Issue: GPT4 vs the mystery of the AMC10/12[00:38:25] And there's some interesting, if you dive into details of GPT4, there's some interesting results in GPT4, which starts to get into the results with benchmarking, right? Like so for example, there was a test that GPT4 published that is very, very bizarre to everyone who is even somewhat knowledgeable.[00:38:41] And this concerns the Ammc 10 and AMC 12. So the mc. Is a measure of the American math 10th grade student and the AMC12 is a, uh, is a measure of the American 12th grade student. So 12 is supposed to be harder than 10. Because the students are supposed to be older, it's, it's covering topics in algebra, geometry number, theory and combinatorics.[00:39:04] GPT4 scored a 30 on AMC10 and scored a 60 on AMC12. So the harder test, it got twice as good, and 30 was really, really bad. So the scoring format of AMC10. It is 25 questions. Each correct answer is worth six points. Each incorrect answer is worth 1.5 points and unanswered questions receive zero points.[00:39:25] So if you answer every single question wrong, you will get more than GPT4 got on AMC10. You just got everything wrong. Yeah, it's definitely better in art medics, you know, but it's clearly still a, a long way from, uh, from being even a high school student. Yeah. There's a little bit of volatility in these results and it, it shows that we, it's not quite like machine intelligence is not the same, or not linearly scaling and not intuitive as human intelligence.[00:39:54] And it's something that I think we should be. Aware of. And when it freaks out in certain ways, we should not be that surprised because Yeah, we're seeing that. Yeah. I feel like part of it is also human learning is so structured, you know, like you learn the new test, you learn the new test, you learn the new test.[00:40:10] But these models, we kind of throw everything at them all at once, you know, when we train them. So when, when the model is strained, are you excusing the model? No, no, no. I'm just saying like, you know, and you see it in everything. It's like some stuff. I wonder what the percentage of. AMC 10 versus AMC 12.[00:40:28] Issue: Data Contamination[00:40:28] Content online is, yes. This comes in a topic of contamination and memorization. Right. Which we can get into if we, if we, if we want. Yeah. Yeah, yeah. So, uh, we're getting into benchmarking issues, right? Like there's all this advancements in benchmarks, uh, language models. Very good. Awesome. Awesome, awesome. Uh, what are the problems?[00:40:44] Uh, the problem is that in order to train these language models, we are scraping the vast majority of the internet. And as time passes, the. Of previous runs of our tests will be pasted on the internet, and they will go into the corpus and the leg model will be memorizing them rather than reasoning them from first principles.[00:41:02] So in, in the machine, classic machine learning parlance, this would be overfitting mm-hmm. Uh, to the test rather than to the generalizing to the, uh, the results that we really want. And so there's an example of, uh, code forces as well also discovered on GPT4. So Code Forces has annual vintages and there was this guy, uh, C H H Halle on Twitter who ran GPT4 on pre 2021 problems, solved all of them and then ran it on 2022 plus problems and solved zero of them.[00:41:31] And we know that the cutoff for GPT4 was 2021. Mm-hmm. So it just memorized the code forces problems as far as we can tell. And it's just really bad at math cuz it also failed the mc 10 stuff. Mm-hmm. It's actually. For some subset of its capabilities. I bet if you tested it with GPT3, it might do better, right?[00:41:50] Yeah. I mean, this is the, you know, when you think about models and benchmarks, you can never take the benchmarks for what the number says, you know, because say, you know, you're focusing on code, like the benchmark might only include the pre 2021 problems and it scores great, but it's actually bad at generalizing and coming up with new solutions.[00:42:10] So, yeah, that, that's a. Big problem.[00:42:13] Other Issues: Benchmark Data Quality and the Iris data set[00:42:13] Yeah. Yeah. So bias, data quality, task specificity, reproducibility, resource requirements, and then calibrating confidence. So bias is, is, is what you might think it is. Basically, there's inherent bias in the data. So for example, when you think about doctor, do you think about a male doctor, a female doctor, in specifically an image net?[00:42:31] Businessmen, white people will be labeled businessmen, whereas Asian businessmen will be labeled Asian businessmen and that can reinforce harmful serotypes. That's the bias issue. Data quality issue. I really love this one. Okay, so there's a famous image data set we haven't talked about called the pedals or iris.[00:42:47] Iris dataset mm-hmm. Contains measurements of, uh, of, uh, length with petal length and petal with, uh, three different species of iris, iris flowers, and they have labeling issues in. So there's a mini, there's a lowest level possible error rate because the error rate exists in the data itself. And if you have a machine learning model that comes out with better error rate than the data, you have a problem cuz your machine learning model is lying to you.[00:43:12] Mm-hmm. Specifically, there's, we know this for a fact because especially for Iris flowers, the length should be longer than the, than the width. Um, but there. Number of instances in the data set where the length was shorter than the, than the width, and that's obviously impossible. So there was, so somebody made an error in the recording process.[00:43:27] Therefore if your machine learning model fits that, then it's doing something wrong cuz it's biologically impossible. Mm-hmm. Task specificity basically if you're overfitting to, to one type of task, for example, answering questions based on a single sentence or you're not, you know, facing something real world reproducibility.[00:43:43] This one is actually, I guess, the fine details of machine learning, which people don't really like to talk about. There's a lot. Pre-processing and post-processing done in I Python notebooks. That is completely un versions untested, ad hoc, sticky, yucky, and everyone does it differently. Therefore, your test results might not be the same as my test results.[00:44:04] Therefore, we don't agree that your scores are. The right scores for your benchmark, whereas you're self reporting it every single time you publish it on a, on a paper. The last two resource requirements, these are, these are more to do with GPTs. The larger and larger these models get, the harder, the more, more expensive it is to run some.[00:44:22] And some of them are not open models. In other words, they're not, uh, readily available, so you cannot tell unless they run it themselves on, on your benchmark. So for example, you can't run your GPT3, you have to kind of run it through the api. If you don't have access to the API like GPT4, then you can't run it at all.[00:44:39] The last one is a new one from GPT4's Paper itself. So you can actually ask the language models to expose their log probabilities and show you how confident they think they are in their answer, which is very important for calibrating whether the language model has the right amount of confidence in itself and in the GPT4 people. It. They were actually very responsible in disclosing that They used to have about linear correspondence between the amount of confidence and the amount of times it was right, but then adding R L H F onto GPT4 actually skewed this prediction such that it was more confident than it should be. It was confidently incorrect as as people say.[00:45:18] In other words, hallucinating. And that is a problem. So yeah, those are the main issues with benchmarking that we have to deal with. Mm-hmm. Yeah, and a lot of our friends, our founders, we work with a lot of founders. If you look at all these benchmarks, all of them just focus on how good of a score they can get.[00:45:38] They don't focus on what's actually feasible to use for my product, you know? So I think.[00:45:44] Tradeoffs of Latency, Inference Cost, Throughput[00:45:44] Production benchmarking is something that doesn't really exist today, but I think we'll see the, the rise off. And I think the main three drivers are one latency. You know, how quickly can I infer the answer cost? You know, if I'm using this model, how much does each call cost me?[00:46:01] Like is that in line with my business model I, and then throughput? I just need to scale these models to a lot of questions on the ones. Again, I just do a benchmark run and you kind of come up. For quadrants. So if on the left side you have model size going from smallest to biggest, and on the X axis you have latency tolerance, which is from, I do not want any delay to, I'll wait as long as I can to get the right answer.[00:46:27] You start to see different type of use cases, for example, I might wanna use a small model that can get me an answer very quickly in a short amount of time, even though the answer is narrow. Because me as a human, maybe I'm in a very iterative flow. And we have Varun before on the podcast, and we were talking about a kind of like a acceleration versus iteration use cases.[00:46:50] Like this is more for acceleration. If I'm using co-pilot, you know, the code doesn't have to be a hundred percent correct, but it needs to happen kind of in my flow of writing. So that's where a model like that would be. But instead, other times I might be willing, like if I'm asking it to create a whole application, I'm willing to wait one hour, you know, for the model to get me a response.[00:47:11] But you don't have, you don't have a way to choose that today with most models. They kind of do just one type of work. So I think we're gonna see more and more of these benchmark. Focus on not only on the research side of it, which is what they really are today when you're developing a new model, like does it meet the usual standard research benchmarks to having more of a performance benchmark for production use cases?[00:47:36] And I wonder who's gonna be the first company that comes up with, with something like this, but I think we're seeing more and more of these models go from a research thing to like a production thing. And especially going from companies like. Google and Facebook that have kinda unlimited budget for a lot of these things to startups, starting to integrate them in the products.[00:48:00] And when you're on a tight budget paying, you know, 1 cent per thousand tokens or 0.10 cent for a thousand tokens, like it's really important. So I think that's, um, that's what's missing to get a lot of these things to productions. But hopefully we, we see them.[00:48:16] Yeah, the software development lifecycle I'm thinking about really is that most people will start with large models and then they will prototype with that because that is the most capable ones.[00:48:25] But then as they put more and more of those things in production, people always want them to run faster and faster and faster and cheaper. So you will distill towards a more domain specific model, and every single company that puts this into production, we'll, we'll want something like that, but I, I think it's, it's a reasonable bet because.[00:48:41] There's another branch of the AI builders that I see out there who are build, who are just banking on large models only. Mm-hmm. And seeing how far they can stretch them. Right. With building on AI agents that can take arbitrarily long amounts of time because they're saving you lots of, lots of time with, uh, searching the web for you and doing research for you.[00:48:59] And I think. I'm happy to wait for Bing for like 10 seconds if it does a bunch of searches for median. Mm-hmm. Just ends with, ends with the right, right result. You know, I was, I was tweeting the other day that I wanted an AI enabled browser because I was seeing this table, uh, there was an image and I just needed to screenshot an image and say, plot this on a chart for me.[00:49:17] And I just wanted to do that, but it would have to take so many steps and I would be willing to wait for a large model to do that for me. Mm-hmm. Yeah. I mean, web development so far has been, Reduce, reduce, reduce the loading times. You know, it's like first we had the, I don't know about that. There, there are people who disagree.[00:49:34] Oh. But I, I think, like if you think about, you know, the CDN and you think about deploying things at the edge, like the focus recently has been on lowering the latency time versus increasing it.[00:49:45] Conclusion[00:49:45] Yeah. So, well that's the, that's Benchmark 1 0 1. Um. Let us know how we, how you think we did. This is something we're trying for the first time.[00:49:52] We're very inspired by other podcasts that we like where we do a bunch of upfront prep, but then it becomes a single topical episode that is hopefully a little bit more timeless. We don't have to keep keeping up with the news. I think there's a lot of history that we can go back on and. Deepen our understanding of the context of all these evolutions in, uh, language models.[00:50:12] Yeah. And if you have ideas for the next, you know, 1 0 1 fundamentals episode, yeah, let us know in the, in the comments and we'll see you all soon. Bye. Get full access to Latent Space at www.latent.space/subscribe

Minden, ami az eredeti AI-os adásunkba nem fért bele. +90 perc tömör mesterséges intelligencia gyönyör. ChatGPT, SEO, kódolás, stb, stb. Linkek: - Biznisz Boyz: https://soundcloud.com/bizniszboyz/ - Data36 course: data36.com/jds - Gigabrief: gigabrief.com - Lex Fridman podcast: https://lexfridman.com/podcast/ Említett eszközök és kutatások: - Cialdini befolyásolási alapelvek: https://en.wikipedia.org/wiki/Influence:_Science_and_Practice - Inversion szabály: https://en.wikipedia.org/wiki/Charlie_Munger#Principle_of_inversion - Chat GPT: https://chat.openai.com - Github Copilot: programozási segéd https://github.com/features/copilot - 8Base: automatikus API és frontend generálás adatbázishoz https://www.8base.com/ - Constellation Software: $31mrd-os M&A cég https://colinkeeley.com/blog/mark-leonard-constellation-software-operating-manual - Amazon Mechanical Turk: ismétlődő munka kiszervezése https://www.mturk.com/ - GPT is all you need for the backend: 65 sor python tetszőleges backend szimuláláshoz, legyen az todo vagy sakk program https://github.com/TheAppleTucker/backend-GPT - Attention is all you need: eredeti Transformer paper https://arxiv.org/abs/1706.03762 - Google érzékeny témák: Your Money Your Life https://triggerfishwriting.medium.com/what-are-googles-your-money-or-your-life-ymyl-pages-ffc53d159a55 - Andrej Karpathy https://karpathy.ai/ - Antrophic AI Constitutional AI: https://arxiv.org/abs/2212.08073 - AI implicit megtanulja az emberi értékeket ha tőlük kap visszajelzést és elkezd félni hogy leállítják https://arxiv.org/abs/2212.09251 - Ex Machina: kiváló dráma AI-ról https://www.imdb.com/title/tt0470752/ Mikről van szó: 00:02:24 AI-alapú nyelvi modellek áttörése: instant válaszok végtelen guglizás helyett 00:05:25 Zéró shot promptok: az AI példa nélkül válaszol 00:06:41 Jobb válaszok struktúrált Chat GPT promptokkal 00:09:51 Híres írók és emberek stílusának másolása Chat GPT-vel 00:13:25 Vázlattal a személyre szabott tartalom generálásához 00:15:12 A GPT megtanítása a szerző stílusának másolására 00:17:19 Jobb eredmények mentális modellekkel és AB tesztelés az iteratív javításhoz 00:26:21 E-mailes kommunikáció megkönnyítése 00:27:14 Segítség az ötletelésben 00:31:00 Paradigmaváltás a szoftverfejlesztésben a GPT-vel és a GitHub Copilottal 00:35:24 SQL generálása a GPT Codex segítségével megjegyzésekből és az adatbázis szerkezetéből 00:36:32 Program kód fordítás más nyelvekre 00:40:35 Növekvő absztrakciós szint a technológiában és a tervezésben, no-code mozgalom és új adatbázis paradigmák 00:43:48 Ez már a mátrix? Gyors tudás letöltése az agyunkba bármilyen területen 00:56:24 A mesterséges intelligencia etikája és Andrej Kárpáty AI-guru 00:58:56 Anthropic AI cég 1 milliárd dolláros befektetési köre, és az alkotmányos AI 01:01:33 Egy általunk nem értett rendszer létrehozásának veszélye 01:05:13 Harvard kutatók bebizonyították, hogyan lehet visszafordítani a biológiai öregedést 01:10:38 A szűk AI jelentősége a nagy technológiai cégek számára, valamint a Google bevétele és keresődominanciája 01:13:20 Google stratégiai kihívása 01:16:41 AI nyelvi modellek hatása a keresésre és a ChatGPT növekvő felhasználói bázisára és fejlődésére 01:19:21 Google befektetése az Anthropic AI-ba és a ChatGPT hatása a SEO-ra 01:21:29 A nagy nyelvi modellek jelenlegi integrációja a mindennapi életben és az AI emberi inputra való támaszkodással és a rossz mérőszámokra való optimalizálással kapcsolatos aggodalmak. 01:27:11 A TikTok mint propagandaeszköz veszélyei 01:28:09 Open Assistant projekt: egy chat GPT alternatíva 750 GB-nyi adaton trenírozva 01:29:00 A digitális heroinba csúszás veszélye 01:31:00 Hallgasd meg a korábbi 113-as AI részt, illetve kövesd be a Biznisz Boyz-t

Wat als er bekend wordt dat buitenaards leven een feit is. Hoe zou de mens reageren? Hoe zou u reageren? Het artikel waar deze podcast over gaat: https://www.frontiersin.org/articles/10.3389/fpsyg.2017.02308/full (https://www.frontiersin.org/articles/10.3389/fpsyg.2017.02308/full) Amazon Mechanical Turk: https://www.mturk.com/ Craig Venter's werk is echt heel interessant: https://www.ted.com/talks/craig_venter_watch_me_unveil_synthetic_life?language=nl Biologos artikel over aliens en de christelijke visie: https://biologos.org/articles/what-would-life-beyond-earth-mean-for-christians The Muslim Vibe over aliens en de Moslim visie: https://themuslimvibe.com/faith-islam/science/what-does-islam-say-about-aliens-a-look-at-quranic-verses-and-hadith U kunt trouwens stemmen op de met ZIS bevriende podcast Sterrenstof: https://podcastawards.nl/stem#wetenschap-educatie  

In this podcast today, I will discuss the company Amazon Mechanical Turk! Listen to the podcast for details! --- Support this podcast: https://anchor.fm/thressa-sweat/support

Sociocultural anthropologist and assistant professor at SFU's School for International Studies, Darren Byler joins Am Johal to speak about his latest book, “Terror Capitalism: Uyghur Dispossession and Masculinity in a Chinese City.” Darren describes how China surveilles and dispossesses Uyghur populations through a mass digital surveillance system, connecting it to the war on terror. Darren and Am also discuss the similarities and differences between the colonialism of China with India, Israel, and other Western countries. Finally, the conversation goes into how Uyghur men protect their wellbeing by developing anti-colonial friendships. The conversation also highlights how many Han Chinese people are building a community of inter-ethnic solidarity to refuse the colonial structures of the state system. Full episode details: https://www.sfu.ca/vancity-office-community-engagement/below-the-radar-podcast/episodes/157-darren-byler.html Read the transcript: https://www.sfu.ca/vancity-office-community-engagement/below-the-radar-podcast/transcripts/157-darren-byler.html Resources: — Terror Capitalism: Uyghur Dispossession and Masculinity in a Chinese City: https://www.dukeupress.edu/terror-capitalism — In the Camps: China's High-Tech Penal Colony: https://www.penguinrandomhouse.com/books/696114/in-the-camps-by-darren-byler/ — Glen Coulthard on Below the Radar: https://www.sfu.ca/vancity-office-community-engagement/below-the-radar-podcast/episodes/37-glen-coulthard.html — Black Skin, White Masks by Frantz Fanon: http://abahlali.org/files/__Black_Skin__White_Masks__Pluto_Classics_.pdf — Justice for “Data Janitors by Lilly Irani: https://www.publicbooks.org/justice-for-data-janitors/ — Amazon Mechanical Turk: https://www.mturk.com/ — Digitize and Punish: Racial Criminalization in the Digital Age by Brian Jordan Jefferson: https://blackwells.co.uk/bookshop/product/Digitize-and-Punish-by-Brian-Jordan-Jefferson-author/9781517909239

Episode: 2765 The Mechanical Turk.  Today, the chess playing automaton.

We have another Ask me a Question session. I get hundreds and hundreds of questions a week and hundreds of thousands over the course of my career. So I've got plenty of material for a lot of little things that can make a big difference in your success online or just in business in general. Screw The Commute Podcast Show Notes Episode 471 How To Automate Your Business - https://screwthecommute.com/automatefree/ Internet Marketing Training Center - https://imtcva.org/ Higher Education Webinar – https://screwthecommute.com/webinars See Tom's Stuff – https://linktr.ee/antionandassociates 00:23 Tom's introduction to Ask me a Question 02:59 Using a paper calendar 04:19 Taking backups and where to do them 08:41 Recording in hot climates 11:21 Selling free webinars 12:59 Making an expensive video for pennies on the dollar 18:00 Using headshots on your website and bio 20:13 Using subdomains to help with SEO 21:51 Getting return visits to your website 26:24 Retargeting ads Entrepreneurial Resources Mentioned in This Podcast Higher Education Webinar - https://screwthecommute.com/webinars Screw The Commute - https://screwthecommute.com/ Screw The Commute Podcast App - https://screwthecommute.com/app/ College Ripoff Quiz - https://imtcva.org/quiz Know a young person for our Youth Episode Series? Send an email to Tom! - orders@antion.com Have a Roku box? Find Tom's Public Speaking Channel there! - https://channelstore.roku.com/details/267358/the-public-speaking-channel How To Automate Your Business - https://screwthecommute.com/automatefree/ Internet Marketing Retreat and Joint Venture Program - https://greatinternetmarketingtraining.com/ Disabilities page - https://imtcva.org/disabilities/ Amazon Mechanical Turk - https://www.mturk.com/ Beginner's Guide to a Customer Loyalty Program - https://blog.hubspot.com/service/customer-loyalty-program Glue Loyalty - https://glueloyalty.com/pricing/ Candy Bar Loyalty - https://www.candybar.co/pricing/ Smile Loyalty - https://smile.io/pricing Belly Loyalty Program - https://www.bellycard.com/ LoyalZoo Loyalty Program - https://www.loyalzoo.com/pricing/ Internet Marketing Training Center - https://imtcva.org/ Related Episodes Julio Gonzalez - https://screwthecommute.com/470/ More Entrepreneurial Resources for Home Based Business, Lifestyle Business, Passive Income, Professional Speaking and Online Business I discovered a great new headline / subject line / subheading generator that will actually analyze which headlines and subject lines are best for your market. I negotiated a deal with the developer of this revolutionary and inexpensive software. Oh, and it's good on Mac and PC. Go here: http://jvz1.com/c/41743/183906 The Wordpress Ecourse. Learn how to Make World Class Websites for $20 or less. https://screwthecommute.com/wordpressecourse/ Join our Private Facebook Group! One week trial for only a buck and then $37 a month, or save a ton with one payment of $297 for a year. Click the image to see all the details and sign up or go to https://www.greatinternetmarketing.com/screwthecommute/ After you sign up, check your email for instructions on getting in the group.

Mike and Ting talk about NFTs, Avatar retaking the crown, and stool gazing. NOTE: this is the pre-episode warm up chat for Death Stranding - Part 1. Contact us: @lostlevelsclub or mike.and.ting@lostlevels.club Show Notes: Pre-Chat Non-fungible token CryptoKitties What are NFTs and why are some worth millions? Beeple sold an NFT for $69 million Artificial scarcity China Box Office: ‘Avatar’ Leads With a Further $14 Million An Exhaustive Timeline of All the Avatar Sequel Announcements Everything we actually know about Matrix 4 Pre-Chat: Subscription Toilet (podcast episode) Going through the motions: the rise and rise of stool-gazing This AI app lets you scan your poop for science. Does it work? Amazon Mechanical Turk

Oggi vedremo cos’è Amazon Mechanical Turk, come funziona e come può essere utile agli imprenditori per smaltire il lavoro o per guadagnare qualcosa lavorando da casa.Link al video nel primo commento.OTTIENI GRATIS IL LIBRO "Vendere su Amazon dalla A alla Z":►https://www.scuolaecommerce.com/libro-fb.Iscriviti al nostro corso gratuito sulla vendita su Amazon:►https://www.scuolaecommerce.com/webinar-fb

What is microwork? Thanks for asking!Microwork is paid work which usually involves short and repetitive tasks carried out on a smartphone or computer. It could be identifying objects shown in an image, watching videos, labelling data, translating short sentences or recording one’s voice for example. Charging electric scooters or taking photos of products for an app could also be considered microwork.That sounds simple enough; what is life like as a microworker then? Generally speaking, each task is paid at a rate of a few cents so microwork is rather unstable. On the other hand, this kind of work is available to all as it doesn’t require specific qualifications. Another benefit is flexibility, with microworkers able to work when and where they want, as long as tasks aren’t time-sensitive. It’s as simple as registering on a platform which acts as the middle man between workers and companies. Amazon Mechanical Turk is an example of one such platform. Many companies use microwork to develop technology like artificial intelligence. To educate machines, we have to talk to them. For example, Alexa and Siri learn to understand our voices thanks to microworkers who record themselves saying all kinds of phrases, each with their own accent and sound environment of course.And driverless cars are able to recognise trees and pedestrians thanks to humans identifying them on millions of photos. This form of work is relatively recent, having emerged in the United States in the 2000s. Back in 2011, it was estimated that microwork contributed $375M to the world economy. However, 22% of microworkers live under the poverty line. And there are other drawbacks too, in addition to the lack of economic security. Some may be demotivated by the apparent lack of meaning in their tasks. Often, microworkers don’t know the name of the company they’re working for, or anything about the project to which they’re contributing.So are we saying robots aren’t yet ready to replace humans in the workplace? In under 3 minutes, we answer your questions!To listen the last episodes, you can click here: What is retrospective contact tracing? What is the Iranian nuclear program?What is cultured meat?A podcast written and realised by Joseph Chance. See acast.com/privacy for privacy and opt-out information.

Screaming Frog tweeted some disturbing information about crawl reports of the Google Search Console. There was up to 50% difference between the crawl data returned in the Search Console and the real log files. This isn't good… Also in this week's news, Barry Schwartz corrected me about last week's Search Engine Journal flawed info on desktop pages for the future mobile index. Some interesting facts about the huge evolution of mobile usage. Videos ranking on Google  SERP improved the identification of key moments. The infamous blue checkmark of Twitter is coming back. Google is launching an Amazon Mechanical Turk alternative in India. A complete guide about how to get rid of URLs in Google index just got published. A new script came out to scrap Archive.org The official Google blog is publishing a post to explain how the Search experience greatly improved lately. John Mueller talks about anchor text.   As always, this is only my opinion. Please don't hesitate to share your point of view on the topics covered in this video.   I never take for granted the time you spend to watch our content.   --- The program of the SEO Conspiracy Podcast is the following: Monday ➡ SEO Myth-busting with my exclusive co-host, the one and only Dixon Jones;  Tuesday ➡ Your fix of Alternative SEO News. I'm reviewing every important news about the Search/Digital Marketing industry from the week before; Wednesday ➡ SEO Stories. With or without a guest, I will take the time to dissects Search Engine Optimization and Digital Marketing topics; Thursday ➡ this day is reserved to talk about Semantic SEO and my strategy called the Topical Mesh. In a series of 52 videos, I lay out the complete plan. This is the most advanced free SEO tutorial in the World; Friday ➡ Q&A. I have tons of questions in stock, asked by my students and clients.  To start off the series, I will dig into this pool of SEO and Digital  Marketing related questions. To continue the series, please contact me (contact info in the about page on the Youtube channel) or via social medias (links below). Ask me any questions. My answer will be 100% BS Free Guaranteed or your money back (just kidding, I'm giving out everything for free); Week-ends ➡ and/or sometimes during the week, live sessions will take place. Among other ideas, I will be performing live SEO audits.   I want to help you achieve better results; I don't want to hold back anything.  I've always been known to lay it all out like it is.   There is way too much BS talk in the SEO industry. Let's cut throughout the noise to have a real conversation.   Thank you very much for watching Laurent Bourrelly https://www.seoconspiracy.com/   ----------- Laurent's Stuff : https://www.topicalmesh.com/ https://www.frenchtouchseo.com/ https://rank4win.com/ https://twitter.com/laurentbourelly   ----------- SEO Conspiracy Social Media  : - Facebook: https://www.facebook.com/seoconspiracy/ - Instagram: https://www.instagram.com/seoconspiracy/ - Twitter: https://twitter.com/seoconspiracy   #SEO #Google #DigitalMarketing

Tekpili. Nuevas tendencias, nuevos dilemas. Con Juanito Pereira. Episodio 56. Esta vez la charla va de Amazon Mechanical Turk, videojuegos de nueva generación, Spotify y más.

Tonight you will be listening to Descript's Terms of Service. Narrated by a AI clone of Scott Elchison voice. Descript’s Terms of Service Highlights: * Word Count: 5712 * Arbitration Agreement: Yes * *Overdub Highlight:* By submitting Training Audio to Descript, you consent (and represent and warrant that you have obtained the consent of any third-party Consenting Speakers) to our use and storage of the audio recordings and voices you submit as follows: * (a) to use your voice, and the voice of any third-party Consenting Speakers, to train an Overdub Voice, to synthesize and otherwise use such voices as describe herein, and to otherwise operate the Descript Service; * (b) as part of our research datasets to analyze, maintain, and improve our voice technology, and other technology, and for other research and development and data analysis purpose, provided that if we add your Training Audio to our research datasets, we first anonymize the data so it is no longer associated with your account; * (c) Descript employees, contractors and Amazon Mechanical Turk workers may listen to audio samples of your Training Audio and your synthesized audio in order to test the quality of your Overdub Voice; and * (d) Descript employees may use your Overdub Voice to create a series of non-defamatory utterances, solely for internal quality assurance purposes. * Other than as described above, only you, and people you explicitly share access with, will have access to generate synthesized audio using your Overdub Voice. Descript will not share your Training Audio or your Overdub Voice with third parties except as expressly described above and in our Privacy Policy ( https://www.descript.com/privacy ). Support this podcast at — https://redcircle.com/ts-and-zzz/donations Advertising Inquiries: https://redcircle.com/brands

Què poc ens pensàvem al Febrer que dies més tard ens tancarien a casa, milers de negocis baixarien la persiana, o veuríem gent amb mascareta pel carrer. Parlem de les tendències que han vingut per quedar-se i què passarà quan hagi minvat la tempesta.Show notes:El nou podcast de Ben Thompson i John Gruber és Dithering.El podcast de John Gruber és The Talk Show.Quan el Marc parlava d'una força laboral atomitzada, posava com exemple el Amazon Mechanical Turk.La idea de que a Internet, amb infinits nínxols disponibles, només necessitarem convèncer a 1.000 seguidors surt de 1.000 True Fans.

CrowdFlower was a company started in 2007 by Lukas Biewald, an entrepreneur and computer scientist. CrowdFlower solved some of the data labeling problems that were not being solved by Amazon Mechanical Turk. A decade after starting CrowdFlower, the company was sold for several hundred million dollars.Today, data labeling has only grown in volume and scope. But Lukas has moved on to a different part of the machine learning stack: tooling for hyperparameter search and machine learning monitoring.Lukas Biewald joins the show to talk about the problems he was solving with CrowdFlower, the solutions that he developed as part of that company, and the efforts with his current focus: Weights and Biases, a machine learning tooling company.

CrowdFlower was a company started in 2007 by Lukas Biewald, an entrepreneur and computer scientist. CrowdFlower solved some of the data labeling problems that were not being solved by Amazon Mechanical Turk. A decade after starting CrowdFlower, the company was sold for several hundred million dollars. Today, data labeling has only grown in volume and The post Machine Learning Labeling and Tooling with Lukas Biewald appeared first on Software Engineering Daily.

CrowdFlower was a company started in 2007 by Lukas Biewald, an entrepreneur and computer scientist. CrowdFlower solved some of the data labeling problems that were not being solved by Amazon Mechanical Turk. A decade after starting CrowdFlower, the company was sold for several hundred million dollars. Today, data labeling has only grown in volume and The post Machine Learning Labeling and Tooling with Lukas Biewald appeared first on Software Engineering Daily.

CrowdFlower was a company started in 2007 by Lukas Biewald, an entrepreneur and computer scientist. CrowdFlower solved some of the data labeling problems that were not being solved by Amazon Mechanical Turk. A decade after starting CrowdFlower, the company was sold for several hundred million dollars. Today, data labeling has only grown in volume and The post Machine Learning Labeling and Tooling with Lukas Biewald appeared first on Software Engineering Daily.

Image annotation is necessary for building supervised learning models for computer vision. An image annotation platform streamlines the annotation of these images. Well-known annotation platforms include Scale AI, Amazon Mechanical Turk, and Crowdflower.There are also large consulting-like companies that will annotate images in bulk for you. If you have an application that requires lots of annotation, such as self-driving cars, then you might be compelled to outsource this annotation to such a company.SuperAnnotate is an image annotation platform that can be used by these image annotation outsourcing firms. This episode explores SuperAnnotate, and the growing niche of image annotation. Vahan and Tigran Petrosyan are the founders of SuperAnnotate, and join the show for today's interview.

Image annotation is necessary for building supervised learning models for computer vision. An image annotation platform streamlines the annotation of these images. Well-known annotation platforms include Scale AI, Amazon Mechanical Turk, and Crowdflower. There are also large consulting-like companies that will annotate images in bulk for you. If you have an application that requires lots The post SuperAnnotate: Image Annotation Platform with Vahan and Tigran Petrosyan appeared first on Software Engineering Daily.

Image annotation is necessary for building supervised learning models for computer vision. An image annotation platform streamlines the annotation of these images. Well-known annotation platforms include Scale AI, Amazon Mechanical Turk, and Crowdflower. There are also large consulting-like companies that will annotate images in bulk for you. If you have an application that requires lots The post SuperAnnotate: Image Annotation Platform with Vahan and Tigran Petrosyan appeared first on Software Engineering Daily.

Image annotation is necessary for building supervised learning models for computer vision. An image annotation platform streamlines the annotation of these images. Well-known annotation platforms include Scale AI, Amazon Mechanical Turk, and Crowdflower. There are also large consulting-like companies that will annotate images in bulk for you. If you have an application that requires lots The post SuperAnnotate: Image Annotation Platform with Vahan and Tigran Petrosyan appeared first on Software Engineering Daily.

How To Make Money Online FAST With Amazon Mechanical Turk!***Click below to subscribe to my email list and learn how to make 6-7 figures online! https://drewbusinessaccelerator.com/freedom**

Welcome!   For being locked down do to this Pandemic there is certainly a lot of technology in the news this week.  So let's get into it.  President Trump issued an Executive Order to protect our Electric Grid from using equipment not manufactured in the US, Microsoft Teams is under attack, Phishing and Ransomware are in the News and What will Post-COVID Business look like? So sit back and listen in.  For more tech tips, news, and updates visit - CraigPeterson.com --- Automated Machine Generated Transcript: Craig Peterson: Hey everybody, welcome Craig Peterson here on WGAN. It is quite a week. I just can't believe how fast time is going. So many people are at home with nothing much to do, they're watching Netflix, et cetera, and I am busier than ever just trying to help people out and I'm going to be doing more free training and stuff over the next couple of weeks. Now I've just been so, so busy. I don't know if you've heard any of my features here on the radio station. They're supposed to have started airing, I guess we'll see if they do air, but I'm putting together these kinds of filler things that are a couple of minutes long. The whole idea behind them is to really help. People with just various technology issues. You know, me, I'm focusing on security because that is what seems to be lacking the most, and especially when we're seeing what we're seeing right [00:01:00] now, which is all kinds of people. Just getting everything stolen from them. It Is crazy what's happening.  You know, we're all working at home right now to some degree. Many of us, obviously you still have to go in and. You know, in foodservice and manufacturing, et cetera. But even with that, the bosses aren't necessarily all there. Some people are getting sick and are staying at home for very good reasons. I think we'll see more of that in the future. Someone gets sick instead of the old American worth work ethic of going in and getting everybody else sick. I think we're going to see a lot more of the, Hey, I'm going to stay home because I'm not feeling well. This is going to be interesting because so many companies have these sick policies, sick day policies that I've never liked particularly. I think some of those will change too, but what is going to happen here in our post-COVID world, right? We've got this COVID-19 of [00:02:00] course the Wuhan virus causes the disease. it's also called, what is it, C O V I D SARS-2? Remember SAR. SARS had a much, much higher death rate than COVID-19 is turning out to have. But there are many, many people that have this. And we've seen some statistics now coming out saying that even people that are staying home, this one hospital this week did some, a little bit of research and found that 60% of their patients had quarantined in themselves at home. Now that tells you something too. We, we still don't know enough about this whole WuHan virus and the diseases that it might cause. Some of the symptoms we kind of know, obviously when it comes to respiratory problems, is an acute respiratory disease, which is what SARS is. Yeah, we know the basics of that, but man, the stuff we've been hearing about people having circulation problems, having legs amputated, even people who are [00:03:00] in good shape, you know, I hate to see it, but I can understand a diabetic having problems, right. And maybe ultimately having a leg amputated because of circulatory problems that come with diabetes or circulatory problems that come with being morbidly obese or even just obese. Those all kind of make sense to me, but. I don't know there's just so much we don't know. One of the things we're trying to figure out is what does the business looks like?  What is going to happen? And there's a great article that came out in the computer world just this last week that is talking about telecommuting. I think it's really kind of an interesting thing because what we're talking about is a disease that's going to be affecting us probably for the next 18 months to two years now. I don't mean like the whole country or world is shut down for that period of time. Obviously that would be catastrophic to everyone. We would have people dying of starvation if that were to happen, but what I'm talking about is really kind of like what happened with the Spanish flu. You know, every last one of us has had that flu that happened in 1918 and unless you've been an absolute hermit that I've never had any food, you didn't grow, et cetera, right? It just sticks around. And that's going to happen with his WuHan virus. Well, it is going to be around forever, frankly, now that it's been thrust upon us, however, that came to be. Depending on whether or not we've got a vaccine. We've got some really good treatment when they're in place. That's really going to be the point where we try and get back to usual. I don't know. It's so many businesses are doing layoffs. One of my sons. His boss was just furloughed and a couple of his team members were furloughed. He's [00:05:00] kind of low end to management. He has a team that he supervises, and so the supervisor, one of the supervisors of the team supervisors got laid off. So when the business gets back going again, are they still going to have that extra layer of management in the middle? I don't think so. And some of these team members that were laid off are not necessary, you know, not, not talking about my son here, but just in general. But some of these team members that have been laid off in businesses are not necessarily the best of employees. So what does that mean? The owners and executives and businesses are going to have to find themselves running businesses in very different ways. I talked this week a little bit with Matt. Of course, I'm on the radio pretty much every morning during the week on different stations, but I was talking about what is [00:06:00] happening. What are we looking at? Where's this going? And one of the things that came up was, Hey, listen, we have these executives at the C-level. We have all of these people down, the front end, is that going to change the way most businesses work? And obviously I think the answer to that is yes, right? Absolutely. Yes. The vast majority of the burden to put together these new businesses and new operations is going to fall to the people in information technology. That's exactly what we are doing. So we've got to have it, executives, starting to talk about what does the business look like going forward? What should they be doing? How can they have an infrastructure that works for the employees and that is safe and secure because the bad guys have [00:07:00] redoubled their efforts and there are so many opportunities to them now because there are fewer eyes watching everything? Right now. Working from home is a term. That many people are using. And frankly, if you want to guarantee that the business change is going to fail, maybe you just call it working from home. Telecommuting on a corporate basis can work, but that's not everybody. That's not where we're all going to be here when we're talking about these multibillion-dollar companies. Barely any of them had true corporate work at home or telecommuting pre-COVID-19 now, some of them did in some cases, but frankly, the big distinction between work from home and corporate telecommuting is that [00:08:00] they thought work at home was an occasional thing for convenience. So, or you're not feeling well today. There's a blizzard, there's a big storm out there, or there's a power outage at the main office because they're, they're doing some construction. Some businesses also said, Hey, listen, every Friday during the summer, you know, you want to stay home once a month or whatever, just go ahead and do it and work from home. That's not corporate telecommuting. Telecommuting is where the employee or the contractor, these people who are working on a gig basis are based at the remote location full time. Now I've talked a bit about the gig economy. And gig workers before on this show, and I've talked about it many times on, on the radio and TV, but in case you don't know what that is, the gig economy is a major change. We started to see a few years ago where people, particularly businesses, were looking and saying, Hey, listen, we don't need to have all of these people on the payroll. Because in reality, this job is part-time. So why would we pay someone full time when it's a part-time job? And why would I have one person working at it when I could have three, four, or five people working at it when necessary. So all of a sudden there's an uptick in my business. Instead of having to try and find someone else, hire someone else, bring them in or, or turn down the work because I can't possibly handle it because I only have this one person who was part-time before. What we ended up doing is saying, Hey, How bout we just find people to do this one narrow thing, and the more narrowly the task can be defined, the better of the businesses because the cost goes down. [00:10:00] The more complex a task is, the more expensive it is. And you look at something like Amazon Mechanical Turk in case you're not familiar with that service. Amazon has, there are people who maybe some of you guys are doing this, who sit there and do very small, very narrow tasks for typically a fixed price. So it might be, get me the phone number and name of this doctor in this town. And you're paid a penny or whatever, 5 cents for doing that very, very narrow task. So they can go ahead and they have someone else saying, find me the name of all of the doctors that meet this criterion in this town and get me their names, their phone numbers, and their addresses. Much, much cheaper to break all of that down to the business. So they're looking at things like Mechanical Turk, but they're also looking at sites like Fiverr, which I've [00:11:00] used before as well. F I V E R R.com and if you go to fiverr.com in fact, let me go there right now while we're talking, you can find people to do almost. Anything for you. It says right on their homepage here, find the perfect freelance services for your business. And most of these are very narrow tasks. And their original idea is you, you know, five bucks, they discharged five bucks for it. And, you know, isn't that. or more reasonable thing than having to have an employee and having to have all of the expenses involved. All right, so I'll stick around. I wanted to finish this up here. A little bit of wandering and meandering as we're talking about. What does the post-WuHan virus world look like in the business space? You're listening to Craig Peterson, on W G A N and online at Craig Peterson dot com. Craig Peterson: Hi guys. Craig Peterson here on WGAN and of course online at craigpeterson.com. We were talking before the break, a little bit about the post-Covid 19 world. And I started talking about the gig economy and what it really is, what does it really mean to us? And I was just talking about a website called fiverr.com which kind of defined the whole gig economy for a while, frankly, for a number of years. And now there are more sites out there as well. But really Fiverr is the place to go online. So they have things like design a logo. Customize your WordPress website, doing voiceover whiteboard work for people. SEO, which is search engine optimization, illustration, translation, data entry. Those are kind [00:01:00] of their top categories, and you can go there. You can find what people are doing, what they're offering, what's the best thing for you, for your business? What might you want to consider? If it is really quite good and there are a lot of true experts that are making there. Their talents available to businesses now it's not just five bucks to do something. Some of these are a lot more expensive and some go on an hourly basis and, and I've used a number of other websites in the past in order to get people to hire people to do things. Upwork is one of the other big ones. U P W O R K.com. Check that one out as well. Whether you're looking for help or you want to provide help and sell some help. But upwork.com is another good one that I've used. And in both cases, I can go and post something and say, Hey, this is what I'm interested in. Having done and people will bid on it [00:02:00] for you. Now, a little inside tip here you might not be aware of in that is if you want people to bid on it, they have to be aware of it, and the only way they're going to be aware of it usually is if you reach out to them. So you have to do a bunch of studying and research and advance so that you know who looks like they might fit for you, and then you have to send them an invite directly because most of these people, especially the good ones, are not sitting there just waiting for a general. Query to come in, Hey, I need somebody to do a logo. Now they don't pay attention to that because they are in demand. So you have to find the people that you want to do. For instance, your logo, whatever the work is. So you'll go online, you'll look around, you'll look at their samples, they've posted, you'll find a few people, and I've found usually in order to find somebody that's good. I have to reach out to as many as 50 five [00:03:00] zero people on these websites to get the attention of somebody I really want. So if you are top-rated, it's phenomenal. They have ratings like at Upwork they have really great ratings and stuff for who some of the better people are. It really helps you with your decision. So when we're talking about the future, it's not just telecommuting. Or you might have lost your job. So what do you do now? I know, for instance, one of our listeners here, Linda, she reached out to me and I helped her with some, or actually one of my techs helped her out with some of the problems she was having. because she has lost her business actually, I think it was, and she's trying to start another one by doing website evaluation. You know, that's a perfect opportunity for somebody. To go to Fiverr or Upwork and see if they can't dig up a little bit of work as well. Now when you're first starting out, you're going to have to look at [00:04:00] those main feeds and you're going to have to comb through them and approach people. And you'd probably have to do stuff for really cheap until you develop a reputation. Cause you have to have people giving you those five-star reviews. But it's going to take a little bit of time. Now, one of the big questions that come up is payroll taxes. And when we're talking about the gig economy, the IRS has a set of standards that are in place that help you evaluate whether someone should be treated as a contractor or if they should be treated as an employee. And there's quite a bit of IRS case law if you want to call it that, IRS rules and regulations that have come out of the IRS courts that are paid by the IRS and judges work for the IRS and they get to decide what's right or wrong with you, right? But, there have been a lot of cases that say, Hey, listen to, here's where the line is drawn between a [00:05:00] contractor that you can pay 1099 and somebody who's W2. And that line that we're talking about is, is not just, Hey, they're working at home. Yeah. They're working from home. Well, do you supervise them? Do you give them the work that needs to be done? Are you setting deadlines? Are you telling them what equipment or software to use? You know, you need to talk to your attorneys, reach out to your accountant to figure out what all of those rules are and how they apply to you. But it, this adds yet another little twist to it. You know, it's one thing if you have just this limited task and you hire them once to do the task, like, okay, I need a logo design, or I need to have this changed on my website, or. Whatever it might be, and that's all well and good and that probably fits the contractor definition. Probably don't even have to 1099 them if you're using one of these sites like Fiverr or [00:06:00] Upwork because they're going to take care of it for you. Some of these sites will do tax withholdings for people and there's a lot of things they'll do, but where they are living also now. Will it affect your payroll taxes? So let's say that you're going to keep people on as employees and your businesses in New Hampshire, but they're living or switch it around here cause it doesn't work for New Hampshire. Right? But let's say they're living in a different state with a different tax jurisdiction. And you are your businesses in a state that has income tax provisions. I know in the Northeast we have some agreements between the States because of, of course, New Hampshire has no income tax and they're the ones that are always used for these things. But, there was an agreement between the state saying, Hey, listen, if they live in mass, you have to pay mass taxes. If you live in New Hampshire and you work in mass, you have to pay mass taxes. If you never ever stepped foot in mass, you have to pay mass. No, you don't. But did you see what happened in New York where? The governor of New York has come out and said, Oh yeah, by the way, all of you people that volunteered your time, if you stayed in New York for more than two weeks, you need to pay us income tax even though you were a volunteer. It just gets crazy. Right? So how do you keep track of all these jurisdictions? And if you're hiring people that live in some other state, they're in Illinois, they're in California, they're in one of these blue States that has crazy regulations and high taxes. Now you have to worry about all of that sort of stuff. Okay. It is really going to be difficult. The employee's home is in Atlanta. The company needs to treat that is an Atlanta office or Bureau in every way. If what's the legal [00:08:00] nexus? I've seen cases where just having a phone number from a state was enough to say, yeah, you are a resident of that state. It's really kind of crazy and not just a resident. I'm talking about businesses here. You have a business nexus there, so you have an Atlanta phone number and you don't have an office there, et cetera, but somebody answers that phone. Even if it's not in Georgia, you could get nailed you. Do you see what I'm talking about? This is absolutely going to be a huge, huge different corporate telecommuting is going to just drive us all crazy. Frankly, and in some states, you have not just the state tax, but you have a County tax, you have a city tax, all kinds of different local taxes at different percentages. I remember I had some stuff going on in Washington state, and it was different [00:09:00] tax rates, even for sales tax. You've been on the County, you were in. It, it kind of gets crazy. So, you're going to have to change their tax status if they're doing a hundred percent of their work in that other jurisdiction. And I think that's going to end up being a problem for a lot of people. So keep, keep an eye on that one is, well, ultimately this is going to lead to I think, nothing but confusion. Anyways, we'll move on to another topic when we get back enough about all of the taxes and things you're going to have to worry about with people working from home. But boy, there are a lot, no time to let your guard down because of Corona fraud. Is a huge threat. And what's we'll talk about what those real-world threats are. So stick around. We'll be right back. You're listening to Craig Peterson on WGAN online, Craig peterson.com Craig Peterson: Hello everybody. Welcome back. Craig Peterson here, WGAN, and of course online at craigpeterson.com. Talking a little bit, of course, it is hard to avoid this, how it got into the post-COVID world out there. What does it really mean? We're just talking. In the above telecommuting and how it's really going to cause some stresses on businesses. And you know, we've already talked in weeks past about how it's going to help businesses with a number of different things, including helping them with their ability to cut costs on, on travel and office space, et cetera. But there are a lot of other things to consider as you just went over. Oh, now we got to talk about what is happening to us at our homes and our businesses from, of course, the security side,  because it's no time to let your guard down. Coronavirus fraud is a huge threat and it's been growing. We're seeing constant warnings about it from the FBI and from. These are various security companies that are out there. Certainly, we're getting all kinds of alerts from Microsoft and from also the Cisco people, but the scammers, the bad guys out there are just constantly reusing old ways of hacking us. And they're using scams that they've used forever as well. And that's part of the reason why I always talk about making sure you stay up to date. It's more important to stay up to date right today than it ever has been before. And scammers are rehashing. Some of these campaigns, kind of like the, remember the Nigerian [00:02:00] scams way back when? Some of those are back now in a bit of a different way. So we've got countries now, and of course, our States are starting to try and get a little bit back to normal here that got some paths to recovery. And in many cases, they're trying to get rid of some of these lockdown restrictions. But meanwhile, the crisis has brought out the worst in these con artists out there. And there's a great article by Ammar over at, we live security talking about some of this thing because. Really, they're exploiting every trick in their book when it comes to trying to defraud people. They've been trying to impersonate legitimate sources of information on a pandemic. We've talked about that where they'll send out an email saying, click here to look at this map of the pandemic, and there might be ads on that or might even be worse. Various types of spyware, obviously the that they're trying to put on there, but they're trying to defraud people and they've got also these fraudulent online marketplaces set up where they're offering deals on everything from hand sanitizer through toilet paper, eh, some of the masks and things. In fact, we just saw it was like a, what was it, $250 million, or maybe it was $25 million, refund from the Chinese for some state that had ordered some of these N95 masks that, that did not meet the standards. So. The scams are everywhere, and as I said, States are getting nailed in this as well. And the most popular, by the way, COVID 19 map. If you really want to see what's going on, you should go to Johns Hopkins University and there's a professor over there by the name of Lauren Gardner at civil and systems engineering, a professor who's working with some of her graduate students. To keep this up to date. So you can go there right now. and it says it's Coronavirus dot EDU, which is, of course, John Hopkins University, which is one of these teaching universities, that is a teaching hospital, but they're showing how many deaths globally, more than a quarter-million. Oh, almost what is getting close to 80,000 deaths in the United States. I also saw some really interesting numbers that were published this week in a scientific journal about how, you know, we're, we're looking at these number of deaths and we say, okay, 80,000 deaths, which is always horrific, but a. Normal flu year would get us what, 40,000 to [00:05:00] maybe 80,000 right? We had a really bad flu year a couple of years ago, but they delved into the statistics behind it. Now, this is where it's really kind of gets interesting because when you look at those statistics behind the normal. Flu, the flu pandemic, I guess they really are. it turns out that the statistics are heavily inflated and they, it's done because we don't track flu deaths like we're tracking the COVID 19 nowhere near as much detail. People that might have died of bacterial pneumonia in years past who were to be counted as a flu death. Now that is a bit of a problem. Right? So what do you do when you have these bad statistics? They're saying that some of these years where we reported 20,000 or more flu deaths, [00:06:00] actually may have been a thousand deaths in reality. So, Right. Any, anyway, so I'm kind of rambling a little bit here, but that brought it up when I was looking at this Johns Hopkins map here in front of me, how many people have died? How many people have recovered? It turns out that at this point that this COVID 19 flu is definitely more fatal. Then the normal flu season and the article I was reading in the journal were saying it could be as much as 44 times more fatal than an average flu year. Now that's really bad, isn't it? When you get right down to it, 44 times more fatal. but we don't know yet. Right. That's kind of a bottom line on all of this. We just really don't know and we're not going to know for a while. Anyways, back to it. [00:07:00] These maps, and I'm looking at a picture of one right now that was in, we live security.com, which is a map. It looks a lot like the John Hopkins map, and it probably is actually, and on top of that, it's got an ad for, you might need disposable coveralls with a hood protective suit. Now. Is this good? Is this not a good suit? They say, click on that to see it on Amazon. And Amazon certainly could have these for, for sale, but are they really sending you to Amazon or are they sending you to some other site out there? Right. What are they doing? They've got a live chat. They've set up. It's, it's really kind of amazing what the bad guys have done. They put a lot of work into this. The world health organization. you know, I don't know, the bigger, the higher up a government or non-governmental entity is in the food chain, [00:08:00] the less I like them, but they do have their own dashboard showing you what they think is going on. With the Coronavirus, so you'll find them at who dot I N T, which is the world health organization international, and they've got a big warning right on their homepage. Beware of criminals pretending to be the world health organization. they will, they're saying they will never, they, the world health organization will never ask for your username or password to access safety information. They'll never send email attachments you didn't ask for. They'll never ask you to visit a link outside of. Who dot I. N. T. They'll never charge money to apply for a job, register for a conference, or reserve a hotel, and they'll never conduct lotteries or offer prizes, grants, certificates, or funding through email. So that gives you an idea of the scams that are being pulled [00:09:00] right now when it comes to the world health organization. So don't let your guard down everybody, these emails that are going out are a real problem. They've got fake one-stop shops for all of your pandemic needs. That's a problem as well. Just just be very careful where you go. I'm looking at some emails as well. They've got tricks and there are many of them are the same old tricks they've always been using. Don't fall for the tricks. All right. Stick around. When we get back, we're moving on again. We're going to talk about this new executive order from President Trump. Is it going to make us safer? You're listening to Craig Peterson here on WGAN and online Craig peterson.com. Craig Peterson: Hello everybody. Welcome back Craig Peterson here. You can find me on pretty much any podcast platform that's out there. One of the easiest ways is to go to Craig peterson.com/whatever your favorite podcast mechanism is. iTunes is kind of the 500-pound gorilla. They're not the 800 anymore. They're just 500 and you can get there by Craigpeterson.com/itunes. Craig peterson.com/spotify Craig peterson.com/tunein whatever your favorite might be, you'll find me right there. So let's get into our next kind of controversial topic. And this has to do with President Trump's ban. Now it went into effect on May 1st, so it's been around for a couple of weeks. It seemed to be something that was released kind of at the spur of the moment. And it has to do with cybersecurity and the critical infrastructure. Now, you probably know that I ran for a couple of years, the FBI's InfraGard webinar training programs, and we did a whole bunch of training on critical infrastructure stuff. That's really kind of the mandate for InfraGard, but critical infrastructure. Now, just look at all of the jobs with Colvid 19 that were considered critical. The critical infrastructure really encompasses most of the economy nowadays. Even law offices are considered critical infrastructure. He said with a chuckle. Now that can be a problem. It can be good. It can be bad. It really kind of all depends, right? But bottom line, when I'm talking about critical infrastructure, I'm talking about the infrastructure that literally runs the country. There's one of the most overused words in the English language, literally, but in this case, [00:02:00] it really does. We're talking about the infrastructure that controls our electric grid, the infrastructure that controls our telephones, our smart devices. Obviously the infrastructure that controls the internet, the infrastructure that controls our sewage systems, our water systems, the whole electric grid, all the way up to our houses. That is the major part of critical infrastructure. Obviously our roads are considered critical infrastructure and the bridges and, and all of the ways of maintaining them. That's all pretty darn critical because without those commerce comes to a slowdown, dramatic and maybe a grinding halt and people die. Think about what happens if a whole region loses power, which happened here, went back in Oh four, I guess, and I think that was the most recent time. It happened in a very big way in, [00:03:00] was it 86 up in Quebec? And the one in Quebec was because of a bit of solar activity and the one here, you know, I've seen attributed to a bunch of things. The most recent one was that. Our power outage was probably done because of a probe into our electric grid, looking to see if they could potentially hack it and it ended up tripping one of these sites, one of these major sites that are used for distributing electricity, and then that tripped another, tripped to another, tripped to another and before we know it, we had a major cascade failure. So all of that stuff is very, very critical. If, if you've been in a hospital, you know how much they eat electricity. Now, hospitals, of course, have generators for the most part, and that's an important thing for them to have, right? You want to be able to have power if the power [00:04:00] goes out. So, okay, I get that, and that's a very good thing. But at some point, if you don't have access to, let's say, the diesel to run the generators, or maybe they're natural gas generators and you can't run those. What ultimately can you trust if you're a hospital. Because if the whole region loses power, so let's say New England, we lost power in all of the new England states, including New York State, New York City, maybe New Jersey. So we're talking about a five-hour car ride in order to get beyond where this particular power outage occurred. That means even people that have generators are going to run out of fuel because they, the gas stations aren't going to work. Most of them don't have. Pumps. So the trucks can't really deliver it cause the gas station doesn't have electricity. They can't be on, they just don't know what's happening. So they're going to have to send trucks to New Jersey or someplace to try and pick up diesel. And if it's even broader to say we had another Carrington event, like what happened in the mid 18 hundreds where there was a major solar flare that knocked out everything in the country. Now back in the mid 18 hundreds that weren't such a big deal. Today it would be huge. So between those two, obviously having a more localized power failure is better. How about the sewage where it all backs up maybe into the streets? How about the water supply where we just can't get water. Because it shut down. So many of these devices are now part of our internet of things, and that's a real problem.  So President Trump signed this executive order that prohibits operators of the United States power grid to buy and to install any electrical equipment that has been manufactured outside of the US they're even going so far as to provide funding and finances to remove some of this equipment from our electrical infrastructure. You probably already know that we are not allowing these Chinese firms to build our new five G infrastructure or any of the equipment that's in it either. Then here's the code from the order. I further find that unrestricted acquisition or use in the United States of the bulk power system, electric equipment designed, developed, manufactured, or supplied by persons owned by controlled by or subject to the jurisdiction or direction of foreign adversaries augments the ability of the foreign adversaries to create an exploit vulnerabilities in bulk power system, electric equipment with potentially catastrophic effect.  I think he's right. We're seeing these power grids, water grids, et cetera, being attacked. And much of it's coming through the internet of things like keep warning people about, it's, it's really, it's just absolutely amazing. So let's go back. I went and checked in the news, cause I had heard about what had happened over in Israel. And this is May 7th okay, so this week, this is very, very recent. Israel is blaming the US for Iran causing a widespread cyberattack on Israeli water and sewage facilities during April. This was a report that came out from Fox News on Thursday, and according to the report, [00:08:00] Iran used American servers to hack into the facilities. A I've talked about this now for 20 years, and, this whole part of it just really bothers me. They used American servers. Most of the time when the bad guys are using American servers using American computers, what they've actually done is they have compromised a server. 20 years ago we were talking about how Al Qaeda was videotaping the beheading of Americans and distributing them worldwide using American servers. Isn't that amazing? It's shocking. It shouldn't be shocking anyways to all of us, but that's what they were doing. They were using servers that they had hijacked. Now here we are 20 years later and Iran is using these servers to attack. [00:09:00] We know that our servers here, our desktops are being used, they're being compromised and then use to do denial of service attacks. Many other types of attacks out there. So it looks like President Trump might have been a little bit ahead of the game here. I'm looking at, the article here that I'm seeing on the Jerusalem Post. Prime minister Benjamin Netanyahu addressed the issue at last year's cyber tech conference in Televiv saying that Iran is attacking Israel on a daily basis. We monitor it and prevent it every day. They are threatening and other ways. What is important is that every country can be attacked and each country needs a combination of defense and attack capabilities. Israel has such an ability. So think that through a little bit. I know here in the US we have the ability to attack back, no question about that. Now, I also found [00:10:00] online over at, Analytics India magazine online, and this is from a couple of weeks back, three weeks ago, cyberattacks on the critical infrastructure of India is a worrying trend. So let's see, we've got the US that we know has had the critic, our critical infrastructure tack. We know your Iran appears to be responsible for Israeli. Critical infrastructure attacks, and according to the prime minister, they're being attacked daily. We've got India, and here's another one. This is the Czech Republic. This is just a quick search that I did online to find out who's been attacked lately. And this is from April 20th so what about three weeks ago? Attempted cyber attacks against several hospitals and an airport in the Czech Republic show. The coronavirus pandemic has not slowed down the West digital adversaries. So the leaders over in the Czech Republic are saying that they were able to stop these attacks, but they're getting more highly sophisticated attacks all of the time. Czech's top cybersecurity agency has warned, expected imminent serious cyberattacks against us healthcare sector aimed at disabling computers and destroying data. So in many cases, it's ransomware. In fact, that's the number one threat right now against our businesses in the US, it's still ransomware. Can you believe it? It is still ransomware. We are still not protecting ourselves and our business. It just drives me nuts. And that's our, we'll do some more training about this in the next few weeks here. This is particularly problematic right now because we're, we are in the middle of a pandemic. We do have hospitals trying to treat patients and they are under attack and they are getting ransomware and some of these big ransomware bad guys out there. I've said, Oh, no, no, no, we're not. Going to, Hey, if we do take control accidentally of the hospital's computers, we're just going to release it right away. We're not going to hold them ransom, and yet they have been, so be very careful. Everybody, this is, this is not going away anytime soon. They are going to continue to attack us. So when we get back, let's talk about something fun here. Let's talk about what the James Dyson Foundation is doing for our kids. You're listening to Craig Peterson here on W G A N and online CraigPeterson.com/subscribe make sure you get my weekly newsletter so you keep on top of all of these new stories for the week, and I'll be on with Matt Wednesday at seven 30. Craig Peterson: Hey everybody, welcome back. Craig Peterson here on WGAN. I'm on every Saturday from one til three and I am so grateful you guys have joined me today and all of the people that have been signing up today from my newsletter, by the way, when you sign up, I've got. Three little special surprises that only don't even mention when you sign up. So we'll be getting those over the course of the next week or so. Some really great tip sheets, some tools that you can use in order to help make sure your home and your business is properly secured. And hopefully by now. they've started running my little features and those are going to be fantastic. I'm trying to generate a couple of weeks so we can put them up and keep them fresh. But it, it kinda goes into some details of, you know what you should do. So let me, I'm going to put one in here right now. Play one of these features. This one's on passwords. Just give an idea of what these are so you can kind of keep an eye, an ear out for them. I was going to say an eye, but it's obviously an ear. Have you ever heard the term poned? While you might have been poned? Hi, this is Craig Peterson here with a security blink about something known as powning. Poned means that your account has been the victim of a data breach. Your username and password have been stolen from a third party. Now there's an easy way to find out if your account login has been stolen. Troy hunt started and still maintained a website called have I been postponed? He's collected the records of almost 10 billion user accounts from the dark web. Think about that for a minute. If you have an online user account, the odds are that your account data is online, out in the dark web, and the bad guys are using the same information they're finding on the dark web to send you phishing emails recently that's included scareware emails that are threatening to release some information about you. If you don't pay a Bitcoin ransom to prove their point, they're including your email address and password they found online. I'm contacted by listeners every week because these emails truly are scary, but are best ignored. How do you find out if you've been a victim of a data breach? Although it's safe to assume that you have been, you can just go online to have I been poned.com. Troy will let you enter your email address and he will search his database to see if your account information has been stolen. So what should you do? Get one password. It's the best password manager I've ever found. Use it to automatically generate a new password for you. For every online account, you have. One password will also automatically check to see if your account is listed on have I been pwned. To find out more about pwned accounts and password management and to find out how best to use them. Visit Craig peterson.com/compromised. So that's what we're doing, putting them out. I think that sounds pretty good. I heard it sounds really good. I'm thinking of the future ones, I'm going to do it a little bit less scripted. It just sounds too highly produced. I don't know what you guys think. Let me know. Just email me@craigpeterson.com I love to get a little bit of feedback from you. Well, let's get into our friend here, James Dyson. Now, in case you don't know who this is, James Dyson, that's spelled D. Y. S. O. N. He's a British inventor, and you probably know him best via his vacuum, the Dyson vacuum. It's really kind of a cool thing. Definitely overkill, but this thing works on the principle of cyclonic separation. And they used some of the similar technology too that Dyson did in order to make some very cool bladeless of fans that you can get. I really liked these things. They're absolutely amazing. He has designed a whole bunch of things. I'm looking right now at his Wikipedia page, and of course, they've got a picture of his bagless Dyson vacuum cleaner, which is really what got him into most homes, most people to understand, but he has been very, very big in inventing things over the years. I like his air blade hand dryer, which you will see at many bathrooms, probably more of them as you go forward. It does use ultraviolet light in order to clean the air. It doesn't spray it all around. I do not like and I have never liked the air dryers and bathrooms. It makes the spread of germs inevitable. It is a very, very bad idea and yet. So many people just think it's fantastic, right? So much easier. We don't have, to use paper towels, which are frankly much better. They spread the disease a lot less. So the Dyson air blade is a very, very cool, hand dryers, kind of like a squeegee. Air to remove water rather than trying to just blow it all away or evaporated with heat very fast drying, a lot less energy and safer too for us in this COVID-19 day. Anyways, let's get into what he's done right now. He's trying to encourage kids to do a little bit of experimentation. He has this fantastic PDF that you can download by going to the James Dyson Foundation website that you can just search for online, James Dyson, DYSON foundation. Now a few, our parent, [00:06:00] grandparent, if you're homeschooling because there's no more school for the year, or you're homeschooling because it's just a great thing to do. You're gonna want to check this out. It would have been handy when my wife and I were homeschooling all of our kids as well, but he's got these challenge cards is what he's calling them, and there are a total of 22 science challenges and 22 engineering challenges. Yeah. It's just so cool. One of these, the first one reminds me of when I was a kid, cause I remember doing this in school and this is how to get an egg to fit into a bottle without breaking it. Now, back then when I was in school, of course, it was a milk bottle, but what they're doing is they want you to get a glass bottle that has a mouth that smaller than the egg. You're going to put that egg into a glass of vinegar and make sure it's completely covered.  So within two days, that egg is going to be very rubbery. Do you remember doing this? You guys ever done this? Then you heat the bottle in hot water. Obviously make sure that you remember a taut, okay. Use a tea towel and your handle it, and then rest the egg on the neck of the bottle. You don't want to put it so the narrow end is down over the mouth of the bottle. Then as the Air inside cools down, it's going to contract. Right. Expand contract, right as you heat and cool. So. The bottle is going to contract a little bit. The air is going to contract a lot. And you're going to have a vacuum inside this bottle, so it's going to suck the egg inside. So cool. And then the card goes into some detail. How does it work? It talks about the protein and what kind of acid is in the vinegar and what ends up happening. It actually [00:08:00] changes the chemical compound of the egg, which is what makes it rubbery. They've got this underwater volcano thing, which is so cool. This is a colorful underwater volcano that you can make very simple, again, ping pong balls and making them float using a hairdryer. It talks about the Bernoulli Bernoulli effect, which is, you remember I first learned about when I was starting to work on these new hard drives that had just come out and how har, how the heads floated using. Bernoulli a fact, a balloon, kebabs. Can you put a skewer into a balloon without popping it? So they explain how that works, what to do, what not to do. Liquid densities, just a whole ton of them. A geodesic dome is their first engineering challenge. Let me see if I can pull that up on my screen because this is pretty cool to make. Make sure you grab this, send it to your kids, grandkids. Use it yourself. Measuring the speed of light weather balloon. How to make a paperclip float. Yeah. Surface tension. Right. Skipped, fire extinguisher, scared pepper, dancing raisins that so many cool things. A lava lamp. I've always thought those were the coolest things. Did you know that some of the best random number generators out there right now are actually using lava lamps? A whole bunch of them. The visible link and then the Geodesic dome is you're using these jelly sweets and cocktail sticks and putting them all together. And how is it done? Talks about Buckminster fuller. I just love this stuff. I don't know about you guys, but it's so simple. Marble runs the kids can make, and it's where marble is running down the outside of a box and how you guided spaghetti bridges. See, all of these are cheap, strong as this drinking [00:10:00] straw. Not the crappy paper ones, but a real drinking straw. Electric motors. Yeah. Anyhow, check it out online. Of course, there's a link to it as well @craigpeterson.com you can go there. You can see all of this week's articles, and if you are a subscriber to my email list. You will already have it in your mailbox, should have gone out to this morning. So double-check your email. If you did not get it, just send me an email to me@craigpeterson.com that's Peterson with an S O N.com and just ME. Right. Me, it's me and Craig peterson.com and I'll be glad to double-check as to why you didn't get it. Hopefully, I didn't get caught in a spam box somewhere cause we send out thousands of these things every week. And you never know if someone, if people don't open them, I don't know if he knew how this works, but if people don't open them, like on Gmail, Google mail, if they're not, people don't open them. They assume, Oh, nobody's interested in this. And so it gets a lower priority until all of a sudden Google thinks, Oh well. This must be spam because people aren't opening it. So make sure you open it and download any graphics that are in there. Cause that tells Google and everybody else that, Hey, you care about this email. If you turn off the remote images, which is what I normally do personally. but when I get a newsletter, I always make sure to turn it back on. so if you got the images, then Google or AOL or Hotmail or office who 65 whatever you're using will know that it is a good email. It's valid. All right. Stick around. When we get back, we're moving to be on we're going to talk a little bit about Microsoft teams and some phishing that's been going on. You're listening to Craig Peterson here on W G A N. Craig Peterson: Hello everybody. Welcome back. Craig Peterson here on WGAN online and craigpeterson.com. We've been covering a lot of stuff this show today. We just talked about these challenge cards and if you're interested, if you didn't get that URL, I'm going to give it to you again. I love these things are great for your kids, grandkids coming over for the day, whatever it might be. Go online and go to either look for James Dyson's foundation or just go to my website craigpeterson.com. You'll find it there under the radio show, but the James Dyson Foundation is who published these things they're absolutely phenomenal. We also talked about President Trump's executive order banning foreign electrical equipment from getting into our grid. Looks like they're trying to remove equipment that's already there. After the attacks that have been mounted all around the world against different [00:01:00] countries is no time to let your guard down. We've got Corona fraud in a very, very big way still, so we talked about some of that, what that's all about, and telecommuting in a post-COVID 19 world, what does that look like? How is that going to affect our businesses, our lives, our jobs, et cetera? So if you missed any of that, you can just go online to Craig peterson.com check the podcast and you can listen to it right there. I've also been trying to put them up over on YouTube and put them up on Facebook from time to time. I'm going to get better about that. I absolutely have to because we've got to get this message out to everybody, and if you have shared my newsletter with friends or some of these webinars I did. Two dozen over the course of a couple of weeks if you shared any of them. I just want to thank you guys so much for doing that. This is such an important thing for me to get the word out. That's what I've been trying to do for. Decades now because I got nailed as a small business owner by one of these pieces of nasty where there was out circulating at the time, and I really don't want it to happen to you or anybody else. And it really upsets me when I see some of these advertisers who are deceiving people. Just this week I broke down one of these ads I was hearing for VPNs. And every word they were saying was correct. But if you get into like the legal definition, if you're sworn in, it's the truth, the whole truth and nothing but the truth, right? It's not what it's supposed to be. What does that mean? Well, the truth, you know? Okay. So did you rob that store? No. Okay. That's the truth of the whole truth might be, no, I did not Rob that store, but I heard Jane robbed the store, or I know Jane robbed this store or that would be the whole truth. So they, they're talking about their VPN product. And they're talking about how it can keep your data away from prying eyes. Well, yeah, it's kind of true, but it also exposes you to even more prying eyes. You see what I'm talking about when I say not the whole truth. So that's why I've been doing all of these free little training and also been doing lots of stuff for some of the paid courses and training too, because we've got to help people understand, and that leads us to what we're going to talk about right now, which is Microsoft teams. And now Microsoft teams are not bad. It's software that you can get as part of your now called, [00:04:00] Microsoft three 65 subscriptions, which can be good, right? And teams are what you need in order to have collaborative work and to be able to do collaborative work. But just as a quick word of warning, the only collaboration system out there right now that has full-audibility and all of the features that are required by some of the more advanced regulations is WebEx teams. But anyways, on all of these fronts from the Microsoft teams through, you might be using Slack, which is another very popular one, and even WebEx, but we're seeing a whole lot of phishing emails, and there's a warning that just came out here this last week that. People, particularly people who are working in industries such as energy, retail, and hospitality. There are some hackers out there right now that are attacking people specifically pretending they are from Microsoft teams. So they're trying to steal the access credentials of employees who are working from home. And what we've been finding is that many of the people who are working from home right now are. You know, they're, they're not being supervised by the security people. They're using a home computer. It may or may not be up to date. It may or may not have reasonable security precautions on it. It can be a real problem. And when they are getting an email like this, if you ever get an email that looks like it's from Microsoft or looks like it from a vendor that you've been using. If you're in the office, you might lean over to somebody else and say, what do you think of this email? Do you think this is legitimate? Or you might report it to your people, your security people, et cetera. But we're finding with people working from home that they're not double-checking it. And so they're clicking on a [00:06:00] link. They think, Oh my gosh, I'm not using Microsoft teams properly, or I mess something up and there's something I have to do. I got to recover this. I got to figure this out. And in fact, what it is, is that the bad guys out there that are trying to hack you realize what it is that you're trying to do, which is get, just get my work done, right? Just get the software working. So they have been directing attacks to the people. That is a little bit more ignorant in some of these ways. All right. Now at this point, it looks like most of these attacks are not highly targeted. In other words, it's not spearphishing. So it goes right back to what I was talking about earlier. Those emails that we were getting from the Nigerian Prince, right? They are general. So they're unlikely to mention your username and Microsoft teams, even your company. They are just generic and they can be sent to anybody. And so the hackers have taken a list of different companies and what businesses they're in and have been trying to direct them to those businesses. Now, the URLs that are in these, oftentimes we're finding that they. Are using multiple levels of URL redirect, and the idea behind that is to throw off these malicious link detection tools that are out there and to hide the actual URL of the final domain that's being used to host the ultimate attack. Isn't this something. These people are doing. So I did some training here on using Cisco Umbrella, which is a product that we sell, but you can buy directly from Cisco. It is specifically designed to help prevent these types of attacks, and I think it's really important that everybody use that installs it right. Get the free version if that is what you need. If you're a business, you should talk with me because there are special business levels that are not offered on the umbrella website, but special business versions that allow a lot more tracking and a lot more granular control. But make sure you have this in place because even with the multiple redirects, the odds are high that Cisco umbrella is going to be able to attack that. All right. So one message is impersonating the notification that's received when a coworker is trying to connect with you or contact you via teams. The other one is claiming that the recipient has a file waiting for them on Microsoft teams, and the email footer even has legitimate links to. The Microsoft websites, you know, Microsoft teams, application downloads, et cetera. And in one of the attacks, these phishing emails containing a link to a document hosted on a site used by an email marketing company. So we have to be very, very careful. And especially now we're, we're working more at home. We are going to be continuing to work more at home, move most of us anyway, and we are using these collaboration tools and maybe you don't have access to your normal texts of people that you would text support people that you would have access to. So double-check all of that. Well, when we come back, we're going to talk about the biggest threat. To the small, medium enterprise space. You're a small business, your small office, your home office, what it is, what those numbers look like, and what you can do about it. And we will be back in just a couple of minutes here. This is Craig Peterson, you are listening to me on W G A N or online at Craig, Peterson.com stick around. We'll be right back. Hey, welcome back everybody. Craig Peterson here. So glad to have you guys. I really enjoy helping out and I love getting those emails you guys send to me. You're so kind. They're just on some of the compliments and some of your suggestions. It's just fantastic and you can reach me directly. By sending an email to me@craigpeterson.com now, I get a lot of emails, particularly lately, so if it takes me a little bit to get back to you, I apologize in advance, but we do try and get back to all of the people who reach out, but you know, that's not always possible. Just a matter of life, I guess, in this day and age. All right, so let's move on to our next topic for today, and that has to do with the biggest threat out there right now for the small business space. And I was looking at some numbers here during the break. I'm trying to [00:01:00] figure out, so, so what is. Going on. We, we've talked a lot about phishing. We talked about what was just happening here in some of the online space. Things you need to look out for and what, what we're really talking about here when we call talk about small business, the biggest threat is. Ransomware to realize that. How long has ransomware been along? Been around? Excuse me. How long has it been out there? How long has it been attacking us? We have some statistics out there. I'm looking at right now from health net security saying that 46% are small, medium businesses have been targeted by ransomware, and 73% have. Paid the ransom. Now, paying the ransom can be cheap. It can be expensive. It really depends. Of course, the FBI suggests you don't pay a ransom because of two reasons. One, it doesn't guarantee you'll get your data [00:02:00] back. In fact, half of the time when a Ransom's paid all of the data is not. Recovered. And the other reason is it shows the bad guys who will pay ransoms, which means, Hey, listen, guys, you guys are paying a ransom.  Maybe we should go after you again because unfortunately, many of the businesses that have been hit by this stuff don't properly update. their security and those are the companies that ended up coming to me. Right? They should have come before the ransomware hit, not after the ransomware hit and not after they had a second problem. You know, if, if you've got somebody who's providing you with its services. And you have been, you know, ransomed. Don't go back to them to try and fix the problem. It's like, well, who was it Einstein that said that the same thinking that created a problem cannot solve the problem. And we've seen that again and again and again, but paying the ransoms. Here's what it costs right now. 43% of SMBs said they've paid between 10,000 and 50,000 to ransomware attackers. 13% said they were forced to pay more than $100,000 now, I can guarantee you any SMB out there, well, if you're like 500 employees. Huh? It's going to cost you more than a hundred thousand. But, uh, you know, if you are a company that has less than a hundred employees, it's not going to cost you more than that. Not even close to it, but paying the ransom doesn't guarantee anything. If you are a bigger company, we're seeing the average cost of one of these attacks being over a million dollars, because if you're trying to recover, you're trying to do the. Great. You got to notify all of your customers, your customers, find out that you've been hacked and that you had ransomware, you had the lost business while you were down. You [00:04:00] have a lost reputation after you get back. Okay. It's just absolutely amazing. Now. Businesses that are in the B to B space like mine, right? I'm, I'm a business to business. In other words, my services, my security services, the hardware, everything. We're selling to businesses. I really don't deal with consumers, although we've certainly helped a lot of consumers out there, listen to the radio show, but the businesses that are in the B2B space are. Saying that about 80% of them, this is self-evaluation. 80% of them are prepared for an attack to some degree or another. They've at least taken some preparatory steps. People, these businesses that are selling to individuals. In other words, B to C, business to consumer, it's about 20% less. All right? It's crazy. 28% of SMBs admitted that they do not have a plan to mitigate a ransomware attack. So it's very important to get all of this stuff together because the bad guys are coming after us. You've got to have a plan. You've got to prevent the attack. So what do you do? Since ransomware. It is right now really the top threat it gets in via phishing attacks. It gets in a lot of different means, but it's really a saran somewhere. That's the bottom line. I would suggest something here because I know you guys. It is so frustrating trying to do updates. It's even more frustrating when you install an update and it breaks something. Right. And frankly, the update thing comes up in the middle of doing something. You say, Oh, I'll do this later. So you put it off. Hopefully, you're running the pro version of Microsoft Windows, not the home version that doesn't let you do much of them put off. And then they'd remind you the next day, Oh, I gotta do this. I gotta remember to do [00:06:00] this. And then you delay it. And in my training, I talk about what the best delays are to use, depending on what kind of business you are, but you gotta kind of figure that out. What are the best delays, uh, between the time Microsoft tells you that you should do it and, and when you absolutely need to do it? So you're sitting there and saying, ah, last time I did this, I had problems and took me a day to recover and I lost all of that work and I don't really know what I'm doing right. I don't know if I should legitimately install it or not. Right? Have you guys had those questions? Yeah, I bet you have. Send me an email me@craigpeterson.com if you've ever had any of those types of questions go through your mind because I think it's normal. Those are the same questions that go through my mind, my team's mind. So what we end up doing, of course, is doing a bunch of online research, at least we understand a little bit about what needs to [00:07:00] be done and how to do that sort of evaluation, right? We're kind of security professionals, so I get it, right? You're sitting there wondering, what should I do? So because of that, let me tell you the secret. Cause it really is a secret. Obviously try and stay up to date. Obviously have windows defender turned on and UpToDate, as UpToDate as you can get it, but I mentioned it in the last segment and if you want more details, go back to the last segment. You can find that online@craigpeterson.com under my radio show. But listen to what I had to say there because probably the best thing you can do. It installs and uses Umbrella. Cisco umbrella is available for free. There are home versions, there are family versions, there are paid versions. They do not sell any of the, you know, the real business versions on their website, and you can always email me@craigpeterson.com if you have some questions about which one's best for you. But what we deal with typically is the enterprise versions. I'm even using the enterprise umbrella. That my company sells at my house, right. In order to protect everything appropriately. But what happens with ransomware is it has to call home. Usually, when malware gets onto your computer and it establishes a foothold, one of the first things that do is call home. So it calls home and says, okay, I've got this computer. What do you want me to do? And the more modern ransomware will give lists of the files that you have on your computer. He liked that. And so it asks, Hey, listen, the files on your computer are this, that, and the other thing. So a bad guy, I'll look at the names of the files on your computer, and if it's interesting, they'll get on your computer. They'll poke around a little bit. And that's why there's such a variant in how much the ransom is. Sometimes they'll demand multimillion-dollar ransoms for the data if they think that you might be worth it. If you are a town, for instance, you're a city like Atlanta. Look at this. They've been ransomed what, two or three times we know of. So the first thing it tries to do is call home. The first thing some of this phishing email does is try and get you to one of these sites where you can get the ransomware. Umbrella, Cisco Umbrella is designed to stop both. It's available for free. Install it. Now I have a course on it and I may be giving that course again. An absolutely free course. We'll see soon, so I'll make sure on my email list so you get it, Craig peterson.com/subscribe. Craig Peterson: Hey, welcome back everybody. Craig Peterson here. Hard to believe the time is almost up, but you know, because that's the way that

After we discuss about how important a vision to start new year in last episode, this podcast will talk about our process to achieve that long goal in a year. According to Personal and Psychology bulletin (2016), they have recruited 242 Amazon Mechanical Turk workers and recorded their New Year’s Resolution. They found 34,4% from survey had talked related to work issue for resolution, second highest result after health issue resolution. In this podcast, we will talk about how we can build our commitment in short term and long term for our resolution and implement it. Let’s walk our talk together to know more about implementation of our resolution.. Let’s #FeelTheBridge to walk our talk together in this year. Visit our website: www.stevlandbridge.com contact us: contact@stevlandbridge.com

  This is the fourth and last episode of mini series "The dark side of AI". I am your host Francesco and I'm with Chiara Tonini from London. The title of today's episode is Bias in the machine      C: Francesco, today we are starting with an infuriating discussion. Are you ready to be angry?    F: yeah sure is this about brexit?  No, I don't talk about that. In 1986 the New York City's Rockefeller University conducted a study on breast and uterine cancers and their link to obesity. Like in all clinical trials up to that point, the subjects of the study were all men.  So Francesco, do you see a problem with this approach?    F: No problem at all, as long as those men had a perfectly healthy uterus. In medicine, up to the end of the 20th century, medical studies and clinical trials were conducted on men, medicine dosage and therapy calculated on men (white men). The female body has historically been considered an exception, or variation, from a male body.    F: Like Eve coming from Adam's rib. I thought we were past that... When the female body has been under analysis, the focus was on the difference between it and the male body, the so-called “bikini approach”: the reproductive organs are different, therefore we study those, and those only. For a long time medicine assumed this was the only difference.    Oh good ... This has led to a hugely harmful fallout across society. Because women had reproductive organs, they should reproduce, and all else about them was deemed uninteresting. Still today, they consider a woman without children somehow to have betrayed her biological destiny. This somehow does not apply to a man without children, who also has reproductive organs.    F: so this is an example of a very specific type of bias in medicine, regarding clinical trials and medical studies, that is not only harmful for the purposes of these studies, but has ripple effects in all of society Only in the 2010 a serious conversation has started about the damage caused by not including women in clinical trials. There are many many examples (which we list in the references for this episode).    Give me one Researchers consider cardiovascular disease a male disease - they even call it “the widower”. They conduct studies on male samples. But it turns out, the symptoms of a heart attack, especially the ones leading up to one, are different in women. This led to doctors not recognising or dismissing the early symptoms in women.    F: I was reading that women are also subject to chronic pain much more than men: for example migraines, and pain related to endometriosis. But there is extensive evidence now of doctors dismissing women's pain, as either imaginary, or “inevitable”, like it is a normal state of being and does not need a cure at all.    The failure of the medical community as a whole to recognise this obvious bias up to the 21st century is an example of how insidious the problem of bias is.   There are 3 fundamental types of bias:    One: Stochastic drift: you train your model on a dataset, and you validate the model on a split of the training set. When you apply your model out in the world, you systematically add bias in the predictions due to the training data being too specific Two: The bias in the model, introduced by your choice of the parameters of your model.   Three: The bias in your training sample: people put training samples together, and people have culture, experience, and prejudice. As we will see today, this is the most dangerous and subtle bias. Today we'll talk about this bias.   Bias is a warping of our understanding of reality. We see reality through the lens of our experience and our culture. The origin of bias can date back to traditions going back centuries, and is so ingrained in our way of thinking, that we don't even see it anymore.    F: And let me add, when it comes to machine learning, we see reality through the lens of data. Bias is everywhere, and we could spend hours and hours talking about it. It's complicated.    It's about to become more complicated.    F: of course, if I know you… Let's throw artificial intelligence in the mix.    F: You know, there was a happier time when this sentence didn't fill me with a sense of dread...  ImageNet is an online database of over 14 million photos, compiled more than a decade ago at Stanford University. They used it to train machine learning algorithms for image recognition and computer vision, and played an important role in the rise of deep learning. We've all played with it, right? The cats and dogs classifier when learning Tensorflow? (I am a dog by the way. )   F: ImageNet has been a critical asset for computer-vision research. There was an annual international competition to create algorithms that could most accurately label subsets of images. In 2012, a team from the University of Toronto used a Convolutional Neural Network to handily win the top prize. That moment is widely considered a turning point in the development of contemporary AI. The final year of the ImageNet competition was 2017, and accuracy in classifying objects in the limited subset had risen from 71% to 97%. But that subset did not include the “Person” category, where the accuracy was much lower...  ImageNet contained photos of thousands of people, with labels. This included straightforward tags like “teacher,” “dancer” and “plumber”, as well as highly charged labels like “failure, loser” and “slut, slovenly woman, trollop.”   F: Uh Oh.  Then “ImageNet Roulette” was created, by an artist called Trevor Paglen and a Microsoft researcher named Kate Crawford. It was a digital art project, where you could upload your photo and let the classifier identify you, based on the labels of the database. Imagine how well that went.    F: I bet it did't work Of course it didn't work. Random people were classified as “orphans” or “non-smoker” or “alcoholic”. Somebody with glasses was a “nerd”. Tabong Kima, a 24-year old African American, was classified as “offender” and “wrongdoer”.    F: and there it is.  Quote from Trevor Paglen: “We want to show how layers of bias and racism and misogyny move from one system to the next. The point is to let people see the work that is being done behind the scenes, to see how we are being processed and categorized all the time.”   F: The ImageNet labels were applied by thousands of unknown people, most likely in the United States, hired by the team from Stanford, and working through the crowdsourcing service Amazon Mechanical Turk. They earned pennies for each photo they labeled, churning through hundreds of labels an hour. The labels were not verified in any way : if a labeler thought someone looks “shady”, this label is just a result of their prejudice, but has no basis in reality. As they did, biases were baked into the database. Paglen quote again: “The way we classify images is a product of our worldview,” he said. “Any kind of classification system is always going to reflect the values of the person doing the classifying.” They defined what a “loser” looked like. And a “slut.” And a “wrongdoer.”   F: The labels originally came from another sprawling collection of data called WordNet, a kind of conceptual dictionary for machines built by researchers at Princeton University in the 1980s. But with these inflammatory labels included, the Stanford researchers may not have realized what they were doing. What is happening here is the transferring of bias from one system to the next.    Tech jobs, in past decades but still today, predominantly go to white males from a narrow social class. Inevitably, they imprint the technology with their worldview. So their algorithms learn that a person of color is a criminal, and a woman with a certain look is a slut.    I'm not saying they do it on purpose, but the lack of diversity in the tech industry translates into a narrower world view, which has real consequences in the quality of AI systems.    F: Diversity in tech teams is often framed as an equality issue (which of course it is), but there are enormous advantages in it: it allows to create that cognitive diversity that will reflect into superior products or services. I believe this is an ongoing problem. In recent months, researchers have shown that face-recognition services from companies like Amazon, Microsoft and IBM can be biased against women and people of color.    Crawford and Paglen argue this: “In many narratives around AI it is assumed that ongoing technical improvements will resolve all problems and limitations. But what if the opposite is true? What if the challenge of getting computers to “describe what they see” will always be a problem? The automated interpretation of images is an inherently social and political project, rather than a purely technical one. Understanding the politics within AI systems matters more than ever, as they are quickly moving into the architecture of social institutions: deciding whom to interview for a job, which students are paying attention in class, which suspects to arrest, and much else.”   F: You are using the words “interpretation of images” here, as opposed to “description” or “classification”. Certain images depict something concrete, with an objective reality. Like an apple. But other images… not so much?    ImageNet contain images only corresponding to nouns (not verbs for example). Noun categories such as “apple” are well defined. But not all nouns are created equal. Linguist George Lakoff points out that the concept of an “apple” is more nouny than the concept of “light”, which in turn is more nouny than a concept such as “health.” Nouns occupy various places on an axis from concrete to abstract, and from descriptive to judgmental. The images corresponding to these nouns become more and more ambiguous. These gradients have been erased in the logic of ImageNet. Everything is flattened out and pinned to a label. The results can be problematic, illogical, and cruel, especially when it comes to labels applied to people.    F: so when an image is interpreted as Drug Addict, Crazy, Hypocrite, Spinster, Schizophrenic, Mulatto, Red Neck… this is not an objective description of reality, it's somebody's worldview coming to the surface. The selection of images for these categories skews the meaning in ways that are gendered, racialized, ableist, and ageist. ImageNet is an object lesson in what happens when people are categorized like objects. And this practice has only become more common in recent years, often inside the big AI companies, where there is no way for outsiders to see how images are being ordered and classified.    The bizarre thing about these systems is that they remind of early 20th century criminologists like Lombroso, or phrenologists (including Nazi scientists), and physiognomy in general. This was a discipline founded on the assumption that there is a relationship between an image of a person and the character of that person. If you are a murderer, or a Jew, the shape of your head for instance will tell.    F: In reaction to these ideas, Rene' Magritte produced that famous painting of the pipe with the tag “This is not a pipe”.   You know that famous photograph of the soldier kissing the nurse at the end of the second world war? The nurse came public about it when she was like 90 years old, and told how this total stranger in the street had grabbed her and kissed her. This is a picture of sexual harassment. And knowing that, it does not seem romantic anymore.    F: not romantic at all indeed Images do not describe themselves. This is a feature that artists have explored for centuries. We see those images differently when we see how they're labeled. The correspondence between image, label, and referent is fluid. What's more, those relations can change over time as the cultural context of an image shifts, and can mean different things depending on who looks, and where they are located. Images are open to interpretation and reinterpretation. Entire subfields of philosophy, art history, and media theory are dedicated to teasing out all the nuances of the unstable relationship between images and meanings. The common mythos of AI and the data it draws on, is that they are objectively and scientifically classifying the world. But it's not true, everywhere there is politics, ideology, prejudices, and all of the subjective stuff of history.    F: When we survey the most widely used training sets, we find that this is the rule rather than the exception. Training sets are the foundation on which contemporary machine-learning systems are built. They are central to how AI systems recognize and interpret the world. By looking at the construction of these training sets and their underlying structures, we discover many unquestioned assumptions that are shaky and skewed. These assumptions inform the way AI systems work—and fail—to this day. And the impenetrability of the algorithms, the impossibility of reconstructing the decision-making of a NN, hides the bias further away from scrutiny. When an algorithm is a black box and you can't look inside, you have no way of analysing its bias.    And the skewness and bias of these algorithms have real effects in society, the more you use AI in the judicial system, in medicine, the job market, in security systems based on facial recognition, the list goes on and on.    Last year Google unveiled BERT (Bidirectional Encoder Representations from Transformers). It's an AI system that learns to talk: it's a Natural Language Processing engine to generate written (or spoken) language.    F: we have an episode  in which we explain all that   They trained it from lots and lots of digitized information, as varied as old books, Wikipedia entries and news articles. They baked decades and even centuries of biases — along with a few new ones — into all that material. So for instance BERT is extremely sexist: it associates with male almost all professions and positive attributes (except for “mom”).    BERT is widely used in industry and academia. For example it can interpret news headlines automatically. Even Google's search engine use it.    Try googling “CEO”, and you get out a gallery of images of old white men.   F: such a pervasive and flawed AI system can propagate inequality at scale. And it's super dangerous because it's subtle. Especially in industry, query results will not be tested and examined for bias. AI is a black box and researchers take results at face value.    There are many cases of algorithm-based discrimination in the job market. Targeting candidates for tech jobs for instance, may be done by algorithms that will not recognise women as potential candidates. Therefore, they will not be exposed to as many job ads as men. Or, automated HR systems will rank them lower (for the same CV) and screen them out.    In the US, algorithms are used to calculate bail. The majority of the prison population in the US is composed of people of colour, as a result of a systemic bias that goes back centuries. An algorithm learns that a person of colour is more likely to commit a crime, is more likely to not be able to afford bail, is more likely to violate parole. Therefore, people of colour will receive harsher punishments for the same crime. This amplifies this inequality at scale.    Conclusion   Question everything, never take predictions of your models at face value. Always question how your training samples have been put together, who put them together, when and in what context. Always remember that your model produces an interpretation of reality, not a faithful depiction.  Treat reality responsibly.   

Andrew Glass is a Brooklyn based Rubyist operating a small independent devshop called Bang Equals. He has held many ‘enrichment jobs’, including being a ball person at US Open for 5 years, traveling for judging Guinness World Record attempts, and will be a balloon holder in the Macy’s Thanksgiving Day Parade this year. Today the panel is discussing his about his 2018 RailsConf talk, Human Powered Rails: Automated Crowdsourcing In Your Ruby on Rails App. In his talk, he shows the audience how to use Amazon Mechanical Turk. Amazon Mechanical Turk lets you post tasks, set a price point, and then people can go and complete the task. This is often done with tasks that can’t be done with machine learning and to train machine learning algorithms. In his talk he goes into What it is, how it’s used, and how we can use Ruby to automate the process. In his apps, he uses it for lead generation, qualification, enrichment, and some video and photo tagging. More specific uses include recording items from a picture of a shopping list, identifying specific things in a video, categorizing businesses and items, sentiment analysis of text or image. Overall, Mechanical Turk is used for things that machine learning can’t handle yet. The panel discusses some different uses for crowdsourcing and how to submit something to Mechanical Turk. There are multiple ways to ensure accuracy in your surveys, including setting up multiple stages to your task, having more than one person complete your task, and creating a qualified worker pool based on tests to determine their aptitude and skill.  The panel discusses some of the controversy surrounding Mechanical Turk, citing an article in the New York Times (see links). The big issue is wages and worker rights. Wages can be very low, and it is ripe for abuse by companies as they could easily refuse all work and withhold pay. It is also important for the companies to give an accurate time estimate for the task and a reasonable reimbursement. Mechanical Turk attracts a variety of people, from people that do it for fun to people to actually do it for a living, so it is vital that companies use the tool responsibly.  Andrew talks more about how his app works. His apps are built on RTurk, Turkee, and Mechanical Turk, and he talks about how they work. The tricky part is figuring out the logic for what answers they will accept. Andrew talks about how to get started with Mechanical Turk and how to validate the work you get back. To ensure you get accurate information, he suggest that you make it happy for your users, make the UX simple and usable, and use a lot of formatting in your forms so that you get good information in. They preface their results with an accuracy score to help determine what is true. Andrew talks about where he wants to go from he. His Turking days are behind him, but his days of coordinating the efforts of many using software show promise.  Panelists Dave Kimura Charles Max Wood Guest Andrew Glass Sponsors Sentry | Use the code “devchat” for $100 credit Cloud 66 - Pain Free Rails Deployments Try Cloud 66 Rails for FREE & get $100 of free credits with promo code RubyRogues-19 RedisGreen Links Human Powered Rails: Automated Crowdsourcing In Your RoR App by Andrew Glass Amazon Mechanical Turk AWS Transcribe I Found Work on an Amazon Website.  I Made 97 Cents an Hour.  RTurk Turkee AWS SDK Turk Picks Dave Kimura: HatchBox Charles Max Wood: The MaxCoders Guide to Finding Your Dream Developer Job White Christmas Andrew Glass: Foragoodstrftime.com Follow Andrew @andrewglass1 on Twitter and Instagram and andyglass.co

Andrew Glass is a Brooklyn based Rubyist operating a small independent devshop called Bang Equals. He has held many ‘enrichment jobs’, including being a ball person at US Open for 5 years, traveling for judging Guinness World Record attempts, and will be a balloon holder in the Macy’s Thanksgiving Day Parade this year. Today the panel is discussing his about his 2018 RailsConf talk, Human Powered Rails: Automated Crowdsourcing In Your Ruby on Rails App. In his talk, he shows the audience how to use Amazon Mechanical Turk. Amazon Mechanical Turk lets you post tasks, set a price point, and then people can go and complete the task. This is often done with tasks that can’t be done with machine learning and to train machine learning algorithms. In his talk he goes into What it is, how it’s used, and how we can use Ruby to automate the process. In his apps, he uses it for lead generation, qualification, enrichment, and some video and photo tagging. More specific uses include recording items from a picture of a shopping list, identifying specific things in a video, categorizing businesses and items, sentiment analysis of text or image. Overall, Mechanical Turk is used for things that machine learning can’t handle yet. The panel discusses some different uses for crowdsourcing and how to submit something to Mechanical Turk. There are multiple ways to ensure accuracy in your surveys, including setting up multiple stages to your task, having more than one person complete your task, and creating a qualified worker pool based on tests to determine their aptitude and skill.  The panel discusses some of the controversy surrounding Mechanical Turk, citing an article in the New York Times (see links). The big issue is wages and worker rights. Wages can be very low, and it is ripe for abuse by companies as they could easily refuse all work and withhold pay. It is also important for the companies to give an accurate time estimate for the task and a reasonable reimbursement. Mechanical Turk attracts a variety of people, from people that do it for fun to people to actually do it for a living, so it is vital that companies use the tool responsibly.  Andrew talks more about how his app works. His apps are built on RTurk, Turkee, and Mechanical Turk, and he talks about how they work. The tricky part is figuring out the logic for what answers they will accept. Andrew talks about how to get started with Mechanical Turk and how to validate the work you get back. To ensure you get accurate information, he suggest that you make it happy for your users, make the UX simple and usable, and use a lot of formatting in your forms so that you get good information in. They preface their results with an accuracy score to help determine what is true. Andrew talks about where he wants to go from he. His Turking days are behind him, but his days of coordinating the efforts of many using software show promise.  Panelists Dave Kimura Charles Max Wood Guest Andrew Glass Sponsors Sentry | Use the code “devchat” for $100 credit Cloud 66 - Pain Free Rails Deployments Try Cloud 66 Rails for FREE & get $100 of free credits with promo code RubyRogues-19 RedisGreen Links Human Powered Rails: Automated Crowdsourcing In Your RoR App by Andrew Glass Amazon Mechanical Turk AWS Transcribe I Found Work on an Amazon Website.  I Made 97 Cents an Hour.  RTurk Turkee AWS SDK Turk Picks Dave Kimura: HatchBox Charles Max Wood: The MaxCoders Guide to Finding Your Dream Developer Job White Christmas Andrew Glass: Foragoodstrftime.com Follow Andrew @andrewglass1 on Twitter and Instagram and andyglass.co

Andrew Glass is a Brooklyn based Rubyist operating a small independent devshop called Bang Equals. He has held many ‘enrichment jobs’, including being a ball person at US Open for 5 years, traveling for judging Guinness World Record attempts, and will be a balloon holder in the Macy’s Thanksgiving Day Parade this year. Today the panel is discussing his about his 2018 RailsConf talk, Human Powered Rails: Automated Crowdsourcing In Your Ruby on Rails App. In his talk, he shows the audience how to use Amazon Mechanical Turk. Amazon Mechanical Turk lets you post tasks, set a price point, and then people can go and complete the task. This is often done with tasks that can’t be done with machine learning and to train machine learning algorithms. In his talk he goes into What it is, how it’s used, and how we can use Ruby to automate the process. In his apps, he uses it for lead generation, qualification, enrichment, and some video and photo tagging. More specific uses include recording items from a picture of a shopping list, identifying specific things in a video, categorizing businesses and items, sentiment analysis of text or image. Overall, Mechanical Turk is used for things that machine learning can’t handle yet. The panel discusses some different uses for crowdsourcing and how to submit something to Mechanical Turk. There are multiple ways to ensure accuracy in your surveys, including setting up multiple stages to your task, having more than one person complete your task, and creating a qualified worker pool based on tests to determine their aptitude and skill.  The panel discusses some of the controversy surrounding Mechanical Turk, citing an article in the New York Times (see links). The big issue is wages and worker rights. Wages can be very low, and it is ripe for abuse by companies as they could easily refuse all work and withhold pay. It is also important for the companies to give an accurate time estimate for the task and a reasonable reimbursement. Mechanical Turk attracts a variety of people, from people that do it for fun to people to actually do it for a living, so it is vital that companies use the tool responsibly.  Andrew talks more about how his app works. His apps are built on RTurk, Turkee, and Mechanical Turk, and he talks about how they work. The tricky part is figuring out the logic for what answers they will accept. Andrew talks about how to get started with Mechanical Turk and how to validate the work you get back. To ensure you get accurate information, he suggest that you make it happy for your users, make the UX simple and usable, and use a lot of formatting in your forms so that you get good information in. They preface their results with an accuracy score to help determine what is true. Andrew talks about where he wants to go from he. His Turking days are behind him, but his days of coordinating the efforts of many using software show promise.  Panelists Dave Kimura Charles Max Wood Guest Andrew Glass Sponsors Sentry | Use the code “devchat” for $100 credit Cloud 66 - Pain Free Rails Deployments Try Cloud 66 Rails for FREE & get $100 of free credits with promo code RubyRogues-19 RedisGreen Links Human Powered Rails: Automated Crowdsourcing In Your RoR App by Andrew Glass Amazon Mechanical Turk AWS Transcribe I Found Work on an Amazon Website.  I Made 97 Cents an Hour.  RTurk Turkee AWS SDK Turk Picks Dave Kimura: HatchBox Charles Max Wood: The MaxCoders Guide to Finding Your Dream Developer Job White Christmas Andrew Glass: Foragoodstrftime.com Follow Andrew @andrewglass1 on Twitter and Instagram and andyglass.co

Show introduction.An interrupted summary of this special spooky episode's topics, along with discussion between Teh Dŭk!tər and FrEd-rEkw' on Friedrich Nietzsche's Twilight of the Idols, or How to Philosophize with a Hammer, in honor of which a new addition to the show's segments is entitled: Phenomenologizing with a New Hammer.Being there.Guilt and anxiety.The writings of Irving Yalom and Rollo May are integrated into discussion and practical advice on guilt and anxiety, as these experiences are typically understood by existential-phenomenological psychologists.Music for our Non-Corporeal DescendantsThe Lesson of AnxietySung by Sim-own. Inspired by Jean Piaget’s theory on accommodation, and incorporating existential perspectives on anxiety. But, see also Klaus Fiedler’s discussion on the emotional correlates of accommodation and assimilation in his Affective Influences on Social Information Processing, a chapter in the Handbook of Affect and Social Cognition by Joseph P. Forgas. On the psychopathology of everyday life specifically, see the work of the same name by Sigmund Freud, and William Barrett’s book entitled Irrational Man. Being here.Results from a A U.S. nationwide survey on nightmares and worries. A nationwide online survey of Amazon Mechanical Turk workers was conducted October 15th-19th, 2019. We asked participants their age, gender, and state of residence, and then asked them to tell us about their most recent nightmare, in their own words over the course of 2-3 sentences. We then asked them to explain—again, over 2-3 sentences of their own words—what most worries them in everyday life. For both of these questions, in turn, we asked them to rate the impact of their nightmare and worries on their everyday lives. Phenomenologizing with a New Hammer.On nightmares, worries, and Emmanuel Levinas.Dr. David R. Harrington returns to discuss the application of Levinasian concepts to the most popular themes revealed in our nationwide survey on nightmares and worries.

Niemal sto lat temu Karel Čapek w sztuce „R.U.R. (Roboty Uniwersalne Rossuma)” przewidywał, że roboty zastąpią nas w codziennych zadaniach. W odcinku zastanawiamy się, czy jego wizja się sprawdza i czy rzeczywiście chciałybyśmy oddać naszą pracę maszynom. Czy asystenci głosowi faktycznie nam pomagają? A właściwie asystentki, bo Siri Apple czy Cortana Microsoftu w domyślnych ustawieniach mają kobiece głosy. Alexa Amazona nosi nawet popularne żeńskie imię. Rozmawiamy z trzema kobietami o imieniu Alexa: z Alexą Brand, terapeutką, o przywiązaniu ludzi do asystentów głosowych; z Alexą Steinbrück, która studiowała w Amsterdamie sztuczną inteligencję, o jej własnej relacji z urządzeniem Amazona; oraz Alexą Garin-Fernandez, mikrobiolożką z Chile, o Amazonii i zawłaszczaniu przestrzeni publicznej przez firmę Amazon. Rozmawiamy też z Siri. Jak sztuczna inteligencja może pomóc w pracy dziennikarza dużego dziennika lub redaktora magazynu kulturalnego? Słuchamy Jeremy’ego Gilberta, dyrektora strategii w „Washington Post” (należącym zresztą do szefa Amazona) i rozmawiamy z Maciejem Jakubowiakiem z „Dwutygodnika”. Czy SI jest rzeczywiście taka mądra i czy przejmie naszą pracę? Za jej działaniem często stoją rzesze ludzi, których niewidzialna praca pozwala utrzymać rozwój technologiczny, na przykład pracujący na platformie Amazon Mechanical Turk. Co to jest fauxtomation? I na czym polega ostatnia mila automatyzacji? Oraz dlaczego dziewiętnastowieczni luddyści obrali sobie za wroga krosno tkackie? Muzyka Wykorzystałyśmy utwory autorstwa Kevina MacLeoda (incompetech.com) Pamgaea oraz Cheery Monday na licencji Creative Commons Uznanie autorstwa 3.0 Licencja Podkast na licencji Creative Commons Uznanie autorstwa (3.0). Audycję można kopiować i rozpowszechniać w dowolnym miejscu i na wybrany przez siebie sposób oraz remiksować i używać w ramach własnych utworów, także komercyjnych, o ile zaznaczy się autorki i tytuł oryginału. Linki do tekstów i projektów wymienionych w odcinku na stronie: https://www.dwutygodnik.com/artykul/8454-odbiornik-20-romantyzm-dla-ludzi-robota-dla-bota.html

Épisode 3 de l'événement "Les plateformes de micro-travail : enjeu pour l’intelligence artificielle, enjeu pour l’emploi ?" France Stratégie, en collaboration avec la MSH Paris-Saclay, a organisé une conférence internationale à Paris, le 13 juin 2019, suivie d’un séminaire fermé de International Network on Digital Labor (INDL) le 14 juin. Elle a été l’occasion d’entendre des témoignages de micro-travailleurs et d’entreprises / plateformes qui les recrutent, de présenter les résultats des enquêtes nationales et internationales sur ces formes émergentes d’emploi et d’en débattre avec des experts académiques et institutionnels, français et étrangers. Après Uber, Deliveroo et autres services à la demande, le micro-travail est une nouvelle facette du travail intermédié par les plateformes numériques. Des services sur Internet ou sur mobile proposent à des foules d’individus de réaliser, pour le compte de commanditaires, des petites tâches standardisées et répétitives, en contrepartie d'une rémunération allant de quelques centimes à quelques euros par tâche. Celles-ci nécessitent en général de faibles qualifications : prendre une photo dans un magasin, reconnaître et classer des images, transcrire des bouts de texte, mettre en forme un fichier électronique… Malgré leur simplicité apparente, ces micro-tâches réalisées par des millions de personnes dans le monde, servent notamment à créer les bases de données nécessaires au calibrage et à l’« entraînement » d’algorithmes et d’intelligences artificielles. Au niveau international Amazon Mechanical Turk est la plus connue des plateformes de micro-travail. En France et dans les pays francophones d’Afrique, d’autres plateformes attirent un nombre croissant de travailleuses et travailleurs, pour compléter leur revenu primaire, voire pour y suppléer. Quelle est l’ampleur du phénomène ? Comment reconnaître, organiser et réguler cette nouvelle forme de travail ? Comment, finalement, s’articule-t-elle avec les formes traditionnelles de l’emploi ? Plus d'informations sur l'événement : cutt.ly/JFMdy6

Épisode 4 de l'événement "Les plateformes de micro-travail : enjeu pour l’intelligence artificielle, enjeu pour l’emploi ?" France Stratégie, en collaboration avec la MSH Paris-Saclay, a organisé une conférence internationale à Paris, le 13 juin 2019, suivie d’un séminaire fermé de International Network on Digital Labor (INDL) le 14 juin. Elle a été l’occasion d’entendre des témoignages de micro-travailleurs et d’entreprises / plateformes qui les recrutent, de présenter les résultats des enquêtes nationales et internationales sur ces formes émergentes d’emploi et d’en débattre avec des experts académiques et institutionnels, français et étrangers. Après Uber, Deliveroo et autres services à la demande, le micro-travail est une nouvelle facette du travail intermédié par les plateformes numériques. Des services sur Internet ou sur mobile proposent à des foules d’individus de réaliser, pour le compte de commanditaires, des petites tâches standardisées et répétitives, en contrepartie d'une rémunération allant de quelques centimes à quelques euros par tâche. Celles-ci nécessitent en général de faibles qualifications : prendre une photo dans un magasin, reconnaître et classer des images, transcrire des bouts de texte, mettre en forme un fichier électronique… Malgré leur simplicité apparente, ces micro-tâches réalisées par des millions de personnes dans le monde, servent notamment à créer les bases de données nécessaires au calibrage et à l’« entraînement » d’algorithmes et d’intelligences artificielles. Au niveau international Amazon Mechanical Turk est la plus connue des plateformes de micro-travail. En France et dans les pays francophones d’Afrique, d’autres plateformes attirent un nombre croissant de travailleuses et travailleurs, pour compléter leur revenu primaire, voire pour y suppléer. Quelle est l’ampleur du phénomène ? Comment reconnaître, organiser et réguler cette nouvelle forme de travail ? Comment, finalement, s’articule-t-elle avec les formes traditionnelles de l’emploi ? Plus d'informations sur l'événement : cutt.ly/JFMdy6

Épisode 2 de l'événement "Les plateformes de micro-travail : enjeu pour l’intelligence artificielle, enjeu pour l’emploi ?" France Stratégie, en collaboration avec la MSH Paris-Saclay, a organisé une conférence internationale à Paris, le 13 juin 2019, suivie d’un séminaire fermé de International Network on Digital Labor (INDL) le 14 juin. Elle a été l’occasion d’entendre des témoignages de micro-travailleurs et d’entreprises / plateformes qui les recrutent, de présenter les résultats des enquêtes nationales et internationales sur ces formes émergentes d’emploi et d’en débattre avec des experts académiques et institutionnels, français et étrangers. Après Uber, Deliveroo et autres services à la demande, le micro-travail est une nouvelle facette du travail intermédié par les plateformes numériques. Des services sur Internet ou sur mobile proposent à des foules d’individus de réaliser, pour le compte de commanditaires, des petites tâches standardisées et répétitives, en contrepartie d'une rémunération allant de quelques centimes à quelques euros par tâche. Celles-ci nécessitent en général de faibles qualifications : prendre une photo dans un magasin, reconnaître et classer des images, transcrire des bouts de texte, mettre en forme un fichier électronique… Malgré leur simplicité apparente, ces micro-tâches réalisées par des millions de personnes dans le monde, servent notamment à créer les bases de données nécessaires au calibrage et à l’« entraînement » d’algorithmes et d’intelligences artificielles. Au niveau international Amazon Mechanical Turk est la plus connue des plateformes de micro-travail. En France et dans les pays francophones d’Afrique, d’autres plateformes attirent un nombre croissant de travailleuses et travailleurs, pour compléter leur revenu primaire, voire pour y suppléer. Quelle est l’ampleur du phénomène ? Comment reconnaître, organiser et réguler cette nouvelle forme de travail ? Comment, finalement, s’articule-t-elle avec les formes traditionnelles de l’emploi ? Plus d'informations sur l'événement : cutt.ly/JFMdy6

Épisode 1 de l'événement "Les plateformes de micro-travail : enjeu pour l’intelligence artificielle, enjeu pour l’emploi ?" France Stratégie, en collaboration avec la MSH Paris-Saclay, a organisé une conférence internationale à Paris, le 13 juin 2019, suivie d’un séminaire fermé de International Network on Digital Labor (INDL) le 14 juin. Elle a été l’occasion d’entendre des témoignages de micro-travailleurs et d’entreprises / plateformes qui les recrutent, de présenter les résultats des enquêtes nationales et internationales sur ces formes émergentes d’emploi et d’en débattre avec des experts académiques et institutionnels, français et étrangers. Après Uber, Deliveroo et autres services à la demande, le micro-travail est une nouvelle facette du travail intermédié par les plateformes numériques. Des services sur Internet ou sur mobile proposent à des foules d’individus de réaliser, pour le compte de commanditaires, des petites tâches standardisées et répétitives, en contrepartie d'une rémunération allant de quelques centimes à quelques euros par tâche. Celles-ci nécessitent en général de faibles qualifications : prendre une photo dans un magasin, reconnaître et classer des images, transcrire des bouts de texte, mettre en forme un fichier électronique… Malgré leur simplicité apparente, ces micro-tâches réalisées par des millions de personnes dans le monde, servent notamment à créer les bases de données nécessaires au calibrage et à l’« entraînement » d’algorithmes et d’intelligences artificielles. Au niveau international Amazon Mechanical Turk est la plus connue des plateformes de micro-travail. En France et dans les pays francophones d’Afrique, d’autres plateformes attirent un nombre croissant de travailleuses et travailleurs, pour compléter leur revenu primaire, voire pour y suppléer. Quelle est l’ampleur du phénomène ? Comment reconnaître, organiser et réguler cette nouvelle forme de travail ? Comment, finalement, s’articule-t-elle avec les formes traditionnelles de l’emploi ? Plus d'informations sur l'événement : https://cutt.ly/JFMdy6

Info: https://www.mturk.com/ Become a worker: https://www.mturk.com/worker Sign up to be a worker: https://www.mturk.com/get-started Cool! Have fun guys and hopefully I helped! Share some love at my Patreon --- Send in a voice message: https://anchor.fm/gennohope/message

Podcast de Tecnología y Filosofía Pop con @fergarco, @vctr_ y @shaqkharisteas. - Ubercopter, ya puedes pedir aventón por los aires - AMLO de Santa Anna quiere ser amiguis de Mark Zuckerberg - Libra, la nueva criptomoneda de Facebook - ¿Cuál fue la primera red social? - La Inteligencia Artificial conecta voces y rostros - Amazon Mechanical Turk, la IA es un señor en una caja - Nos están creciendo cuernos por el uso del smartphone - ¿Somos esclavos del tiempo? - El negocio de la nostalgia y el Golden Age Thinking www.fergarco.com

Assistant Professor Kotaro Hara Crowd work or crowdsourcing, an emerging form of online contracting work, is growing. About 600,000 workers participate in the online gig economy annually and the number is growing rapidly. Crowdsourcing facilitates new ways of working. Its remote and asynchronous work style – unbounded by time and location – could enable people with disabilities and stay-at-home parents to work. On the other hand, many are concerned that workers in crowdsourcing markets are treated unfairly. Researchers are particularly concerned about low wage of crowd work. For example, past research has found that workers typically earn US$2 an hour. In this podcast, Assistant Professor Kotaro Hara from SMU School of Information Systems discusses his research on why crowdsourcing platforms for work, which includes Amazon Mechanical Turk, contribute to low wages for workers in the gig economy, and what could potentially be done to improve the efficacy of these platforms.

Plus artificielle qu’on pense. Même, pas très fûtée, pour le moment. Avec Charles Trahan

American Greed Factory-Episode 316: Obsolescence is Forever, Anti-Drug PSA, Amazon Mechanical Turk, Joe Rogan v Alex Jones, Liam Neeson’s unadvised confession, Trumps Best shit Ever, 1999’s Sixth Sense, Model Talk. 

Amazon Mechanical Turk operates a marketplace for crowdsourcing, and developers can build human intelligence directly into their applications through a simple API. With access to a diverse, on-demand workforce, companies can leverage the power of the crowd for a range of tasks, from ML training and automating manual tasks to generating human insights. In this session, we cover key concepts for Mechanical Turk, and we share best practices for how to integrate and scale your crowdsourced application. By the end of this session, expect to have a general understanding of Mechanical Turk and know how to get started harnessing the power of the crowd.

点击每期节目可以看到具体文稿内容What are the implications of believing it's impossible to alter other people's beliefs?By Alex FraderaWhat makes us stand up and advocate for what we believe? Whether denouncing the tyranny of taxation or making a plea for the necessity of universal health care, we're surely driven by our conviction and the urgency of the situation. But how about what we believe about belief itself, whether it is fixed or malleable? Work in the Journal of Personality and Social Psychology untangles the previously invisible effect of our belief in human certainty.This is a tricky topic to study. People who believe attitudes are set in stone are more likely be more motivated to stand up for their own, thanks to a heightened certainty and faith in their own position. But at the same time, believing attitudes are fixed means the views of your adversaries will be hard to shift, making it less worthwhile to try to change them. In other words, if there's an effect of people's beliefs about human certainty on their willingness to advocate (to attempt to persuade others), it's likely to play out in opposing directions, making it difficult to uncover.Undaunted, Omair Akhtar, who works for Apple, and S. Christian Wheeler of Stanford University recruited 82 participants from Amazon Mechanical Turk and asked half to read a scientific article that reported that attitudes are fixed, and the others to read a different version that stated they are easily changeable. Next, the researchers surveyed the participants' opinions about the death penalty.Those in the “attitudes are fixed” condition expressed both more certainty in their own attitude (whether pro or anti), and a stronger sense that others were unlikely to be persuadable. But they were no more or less likely to say they would try to persuade someone else about the death penalty – an apparently null effect on willingness to advocate, just as we suspected might happen.This might seem to imply that our beliefs about human certainty are irrelevant to our willingness to advocate. But that's not the case. Akhtar and Wheeler were able to penetrate the fog using powerful advances in statistical analysis, showing that believing in the fixed nature of attitudes both tips us toward convincing others, thanks to increasing the certainty of our own attitudes, and also deters us from trying to convince them, thanks to increasing our belief in the non-persuadability of others. The two contrasting effects, normally invisible, were now apparent.968重庆之声每周一至周五8点56分每天三分钟养成良好英语听说习惯

Eric Bolo is the CTO of Batvoice Technologies, a speech analytics startup based in Paris, France. Eric talks about building a custom speech-to-text system for their flagship product, Call Watch. He introduces us to speech analytics and audio-mining, and describes some typical applications. We go into detail about speech-to-text (STT) technologies, and discuss the pros and cons of using cloud STT services such as Google speech versus building a custom STT system yourself. Eric tells us about the latest open source tools and frameworks for building STT systems, and how to get that precious voice data to train our models. We learn how to build and annotate a custom voice dataset ourselves, and hear his advice on starting a voice first company. This is a great first episode to kick off the series! Eric is super smart, with excellent technical skills and a real passion for voice technology. We already know each other quite well, so I couldn't think of anyone I'd rather have as my first guest on the show. I know you're gonna enjoy hearing what he had to say! This is a time-limited preview. To hear the full episode, and access the full catalogue of episodes and bonus content, become a Voice Tech Pro https://voicetechpodcast.com/proLinks from the show:Batvoice / Callwatch: http://www.batvoice.com Google speech API: https://cloud.google.com/speech-to-text/ Microsoft Translator Speech API: https://www.microsoft.com/en-us/translator/speech.aspx IBM Watson: https://www.ibm.com/watson/services/text-to-speech/ Kaldi toolkit for speech recognition: http://kaldi-asr.org/doc/about.html EESEN speech recognition framework: https://github.com/srvk/eesen European Language Resources Association: http://www.elra.info/en/ Mozilla Common Voice Project: https://voice.mozilla.org/en TEDlium English speech recognition training corpus from TED talks: http://www.openslr.org/7/ Voxforge GPL Transcribed Speech corpus: http://www.voxforge.org/ Amazon Mechanical Turk: https://www.mturk.com/ Subscribe to get future episodes:Apple iTunes : https://apple.co/2LqW4olGoogle Podcasts : http://bit.ly/voicetechpodcast-google Google Android : http://bit.ly/voicetechpodcast-android Stitcher : http://bit.ly/voicetechpodcast-stitcher Spotify : https://spoti.fi/2IZr5hm Alexa : http://bit.ly/voicetechpodcast-alexaNewsletter : http://bit.ly/voicetechpodcast-newsletter Website : http://bit.ly/voicetechpodcastHow to support the Voice Tech Podcast:Tell a friend about us or share on social media!Leave a 5 star review on iTunes:

Back in late 2017, we did a show about expensify and how the organization was using a service called 'Amazon Mechanical Turk' (MTurk) to process receipts and to help train their Machine Learning Algorithms. You can download that show and listen to it here:  2017-040 #infosec people on Twitter and elsewhere were worried about #privacy issues, as examples of receipts on MTurk included things like business receipts, medical invoices, travel receipts and the like. One of our Slack members (@nxvl) came on our #Slack channel after the show reached out and said that his company uses services like these at their company. They use these services to test applications, unit testing, and creation of test cases for training and refinement of their own applications and algorithms. We discuss the privacy implications of employing these services, how to reduce the chances of data loss, the technology behind how they make the testing work, and what other companies should do if they want to employ the Mturk, or other 3rd parties. Direct Show Download:   http://traffic.libsyn.com/brakeingsecurity/2018-003-MTurk-NXVL-privacy_issues_using_crowdsourced_applications.mp3   ANNOUNCEMENTS: Ms. Amanda Berlin is running 4 session of her workshop "Disrupting the Killchain" starting on the 4th of February at 6:30pm Pacific Time (9:30 Eastern Time)  If you would like to sign up, the fee is $100 and you can send that to our paypal account at https://paypal.me/BDSPodcast  Course Syllabus:   https://docs.google.com/document/d/12glnkY0nxKU9nAvekypL4N910nd-Nd6PPvGdYYJOyR4/edit     If you have an interesting security talk and fancy visiting Amsterdam in the spring, then submit your talk to the Hack In The Box #HITB Amsterdam conference, which will take place between 9 and 13 April 2018. Tickets are already on sale,  And using the checkout code 'brakeingsecurity' discount code gets you a 10% discount". Register at https://conference.hitb.org/hitbsecconf2018ams/register/     #Spotify: https://brakesec.com/spotifyBDS RSS: https://brakesec.com/BrakesecRSS #Youtube Channel:  http://www.youtube.com/c/BDSPodcast #iTunes Store Link: https://brakesec.com/BDSiTunes #Google Play Store: https://brakesec.com/BDS-GooglePlay Our main site:  https://brakesec.com/bdswebsite   Join our #Slack Channel! Email us at bds.podcast@gmail.com or DM us on Twitter @brakesec #iHeartRadio App:  https://brakesec.com/iHeartBrakesec #SoundCloud: https://brakesec.com/SoundcloudBrakesec Comments, Questions, Feedback: bds.podcast@gmail.com Support Brakeing Down Security Podcast by using our #Paypal: https://brakesec.com/PaypalBDS OR our #Patreon https://brakesec.com/BDSPatreon #Twitter: @brakesec @boettcherpwned @bryanbrake @infosystir #Player.FM : https://brakesec.com/BDS-PlayerFM #Stitcher Network: https://brakesec.com/BrakeSecStitcher #TuneIn Radio App: https://brakesec.com/TuneInBrakesec         Show Notes:     Mr. Boettcher gave a talk (discuss) http://DETSec.org  Brakeing Down Incident Response Podcast   Amanda’s class (starts 4 february, $100 for 4 sessions, $50 for early video access)   I need to mention HITB Amsterdam David’s Resume Review -- Bsides Nash Resume Review  SANS SEC504 Mentor course Guest: Nicolas Valcarcel Twitter: @nxvl   Possible News to discuss: https://www.reddit.com/r/sysadmin/comments/7sn23c/oh_security_team_how_i_loathe_you_meltdown/   Mechanical Turk https://www.mturk.com/ Figure Eight (was CrowdFlower) https://www.figure-eight.com   CircleCi 2.0 https://circleci.com/docs/2.0/   TaskRabbit https://www.taskrabbit.com/   Historically:  https://en.wikipedia.org/wiki/The_Turk   Expensify using Amazon Mechanical Turk https://www.theverge.com/2017/11/28/16703962/expensify-receipts-amazon-turk-privacy-controversy   https://www.wired.com/story/not-always-ai-that-sifts-through-sensitive-info-crowdsourced-labor/ FTA: “"I wonder if Expensify SmartScan users know MTurk workers enter their receipts. I’m looking at someone’s Uber receipt with their full name, pick up, and drop off addresses," Rochelle LaPlante, a Mechanical Turk worker who is also a co-administrator of the MTurk Crowd forum, wrote on Twitter.”   https://www.dailydot.com/debug/what-is-amazon-mechanical-turk-tips/ “About those tasks, they’re called HITs, which is short for Human Intelligence Tasks. A single HIT can be paid as low as a penny but may take only a couple seconds to complete. Requesters often list how long a task is supposed to take, along with the nature of the work and the requirements for completing the work.”   “Since mTurk has been around for over a decade, Amazon has created a special class of workers called Masters Qualification. Turkers with masters have usually completed over 1,000 HITs and have high approval ratings.” Kind of like a Yelp for HIT reviewers?   Are companies like expensify aware of the data that could be collected and analyzed by 3rd parties? Is it an acceptable risk?   Privacy questions to ask for companies that employ ML/AI tech? Are they using Mturk or the like for training their algos? Are they using Master level doers for processing?   Nxvl links: Securely Relying on the Crowd (paper Draft): https://github.com/nxvl/crowd-security/blob/master/Securely%20relying%20on%20the%20Crowd.pdf How to Make the Most of Mechanical Turk: https://www.rainforestqa.com/blog/2017-10-12-how-to-make-the-most-of-mechanical-turk/ How We Maintain a Trustworthy Rainforest Tester Network: https://www.rainforestqa.com/blog/2017-08-02-how-we-maintain-a-trustworthy-rainforest-tester-network/ The Pros and Cons of Using Crowdsourced Work: https://www.rainforestqa.com/blog/2017-06-06-the-pros-and-cons-of-using-crowdsourced-work/ How We Train Rainforest Testers: https://www.rainforestqa.com/blog/2016-04-21-how-we-train-rainforest-testers/ AWS re:Invent: Managing Crowdsourced Testing Work with Amazon Mechanical Turk: https://www.rainforestqa.com/blog/2017-01-06-aws-re-invent-crowdsourced-testing-work-with-amazon-mturk/ Virtual Machine Security: The Key Steps We Take to Keep Rainforest VMs Secure: https://www.rainforestqa.com/blog/2017-05-02-virtual-machine-security-the-key-steps-we-take-to-keep-rainforest-vms/

This planet is a big place filled with amazing and unusual things. Understanding every object, location, and action on this pale blue dot is an enormous challenge. As the world's leading provider of high-resolution Earth imagery, data, and analysis, DigitalGlobe faces this challenge every day. They use automatic computer vision and machine learning where possible, but so far, the only true solution requires the most powerful information processing machine we know: the human brain. Scaling this solution to work on the trillions of satellite pixels collected by DigitalGlobe every day requires thousands of brains, all working in harmony. To address this, DigitalGlobe | Radiant's (now Radiant Solutions) Tomnod service uses Amazon Mechanical Turk, a crowdsourcing internet platform, to identify small objects appearing in large areas of new satellite imagery. Tomnod is heavily used for commercial and humanitarian purposes. In this session, you hear how Radiant Solutions uses crowdsourcing to help solve large-scale computer vision and machine learning problems.

Building a conversational AI experience that can respond to a wide variety of inputs and situations depends on gathering high-quality, relevant training data. Dialog with humans is an important part of this training process. In this session, learn how researchers at Facebook use Amazon Mechanical Turk within the ParlAI (pronounced “parlay”) framework for training and evaluating AI models to perform data collection, human training, and human evaluation. Learn how you can use this interface to gather high-quality training data to build next-generation chatbots and conversational agents.

Lilly Irani discusses the human labor behind artificial intelligence technology. Irani helped create a platform called Turkopticon to support workers on Amazon Mechanical Turk, a website that outsources micro data processing work. Irani also talks about her current book project on entrepreneurialism and national development in India.

Lilly Irani discusses the human labor behind artificial intelligence technology. Irani helped create a platform called Turkopticon to support workers on Amazon Mechanical Turk, a website that outsources micro data processing work. Irani also talks about her current book project on entrepreneurialism and national development in India.

Key Quote “This is the last episode that we will be recording outside of a professional recording studio.” Time-Stamped Notes 00:03 – Introduction to OAO 00:12 – This is the last episode outside a studio 00:28 – Ari and Nick offers their thanks to their listeners 01:02 – EloBot is a bot for your Facebook Messenger that acts as your personal assistant 01:52 – Nick mentions his friend who also built a bot for Facebook 02:06 – Temp File allows you to share files less than 1GB temporarily and for free 02:35 – ai provides services to help you classify data 03:00 – Nick explains further how they use Mighty.ai for Get Leverage 03:44 – Amazon Mechanical Turk is similar to Mighty.ai except they do not allow as much personalization 05:07 – Ari shares a hilarious article from Business Insider regarding the craziest requests concierges have ever received 07:17 – Ari shares an article about what successful people, like Elon Musk, ask job candidates in interviews 08:15 – BeLive allows you to broadcast on Facebook Live with multiple people on one stream 08:53 – End of today’s podcast

With Amazon Mechanical Turk (MTurk), you can leverage the power of the crowd for a host of tasks ranging from image moderation and video transcription to data collection and user testing. You simply build a process that submit tasks to the Mechanical Turk marketplace and get results quickly, accurately, and at scale. In this session, Russ, from Rainforest QA, shares best practices and lessons learned from his experience using MTurk. The session covers the key concepts of MTurk, getting started as a Requester, and using MTurk via the API. You learn how to set and manage Worker incentives, achieve great Worker quality, and how to integrate and scale your crowdsourced application. By the end of this session, you will have a comprehensive understanding of MTurk and know how to get started harnessing the power of the crowd.

Jump-start your machine learning project by using the crowd to build your training set. Before you can train your machine learning algorithm, you need to take your raw inputs and label, annotate, or tag them to build your ground truth. Learn how to use the Amazon Mechanical Turk marketplace to perform these tasks. We share Amazon's best practices, developed while training our own machine learning algorithms, and walk you through quickly getting affordable and high-quality training data.

In this episode, we talk with Justin Picket of SUNY-Albany about using web-based surveys for public opinion polling and experiments. He provides guidance, tips, and tricks for using services like Amazon Mechanical Turk. “A lot of people have great ideas, and they just don’t have the resources to go out and go a longitudinal study. […]

  Hello, welcome to episode 24 of The Bitcoin Game, I'm Rob Mitchell. Earlier this month I got to speak with Andrew Lee from Purse, a company that allows people to use Bitcoin to buy stuff on Amazon, and at pretty large discounts. My own first thought about Purse was that it was some strange company trying to shoehorn its way between Bitcoiners and Amazon. But after talking to Andrew, I now see Purse as a company that is helping to spread the use and adoption of Bitcoin throughout the world. There's a lot more to Purse than I realized! Hope you enjoy this interview. TWITTER GIVEAWAY I will be posting a question or two about the podcast on Twitter. First to tweet back the correct answer will win either a Bitcoin Keychain or the brand new Bitcoin Fork Pen (both pictured below). So follow me at this URL: https://twitter.com/theBTCgame MAGIC WORD Listen for the magic word, and submit it to your LetsTalkBitcoin.com account to claim a share of this week's distribution of LTBcoin. Listeners now have a full week from the release date to claim a magic word. The magic word for this episode must be submitted by 5am Pacific Time on September 2, 2015. SHOW LINKS Purse https://purse.io LG http://www.lg.com Mad Bitcoins http://www.madbitcoins.com Stripe https://stripe.com Lids http://www.lids.com Merrill Lynch https://www.ml.com Dwolla https://www.dwolla.com Mt. Gox https://en.wikipedia.org/wiki/Mt._Gox Amazon http://www.amazon.com Plug & Play http://plugandplaytechcenter.com Andreas Antonopoulos https://en.wikipedia.org/wiki/Andreas_Antonopoulos Blog post on Purse's new Non-Custodial Multisig Wallet https://blog.purse.io/new-multisignature-wallets-to-improve-security-and-privacy Coinbase https://www.coinbase.com Amazon Mechanical Turk https://www.mturk.com Roger Ver http://rogerver.com Reddit Discussion about Purse Pros & Cons https://www.reddit.com/r/Bitcoin/comments/3f36s5 MUSIC All the music in this episode of The Bitcoin Game was created by me! If you're curious, the music was created in GarageBand (by Apple), Animoog (by Moog Music), and Figure (by Propellerhead). Please contact me if you'd like more info about any music you hear on the podcast. STAY IN TOUCH https://Twitter.com/TheBTCGame http://TheBitcoinGame.com Email me at Rob at TheBitcoinGame.com Thanks for listening! Bitcoin tip address: 1G8HDg5EsPQpamKYS2bDya9Riv9xv1nVo5 The Bitcoin Game box artwork created from an illustration by Rock Barcellos.

Hello, welcome to episode 20 of The Bitcoin Game, I'm Rob Mitchell. On April 18th, 2015, Los Angeles Bitcoiners were treated to our city's first Bitcoin conference, called The State Of Digital Money 2015 (or SODM15). I recorded a lot of great content at this conference, and in today's episode, I bring you the first of these recordings. Presentation: The Transformation of Global Commerce by Steve Beauregard, CEO & Founder at GoCoin @GoCoinCEO | @GoCoin https://www.gocoin.com Panel: Merchant & Mass Adoption of Digital Currency Moderated by Steve Beauregard with Justin Newton, CEO of Netki @NetkiCorp https://www.netki.com Paul Puey, CEO and Co-Founder of Airbitz @paullinator | @Airbitz https://airbitz.co Nick Sullivan, Founder and CEO of ChangeTip @gorillamania | @ChangeTip https://www.changetip.com Andrew Lee, CEO of Purse @2drewlee | @PurseIO https://purse.io Connie Chung, Senior Payments Product Manager at Expedia @Expedia http://www.expedia.com Below are YouTube videos of both the presentation and the panel. The videographer who recorded these videos works relentlessly on the front lines, recording cryptocurrency and P2P content from all over the country. Check out all his videos at https://www.youtube.com/channel/UCxfh-2aOR5hZUjxJLQ2CIHw/feed?activity_view=3. MAGIC WORD Listen for the magic word, and submit it to your LetsTalkBitcoin.com account to claim a share of this week's distribution of LTBcoin. Listeners now have a full week from the release date to claim a magic word. The magic word for this episode must be submitted by 8:00am Pacific Time on July 9, 2015. SHOW LINKS State Of Digital Money http://www.cureativ.com/sodm/#digitalmoney 500 Startups http://500.co Plug And Play Tech Center http://www.plugandplaytechcenter.com Bitcoin Syndicates https://angel.co/bitcoin/syndicates Amazon Mechanical Turk https://www.mturk.com UnoCoin https://www.unocoin.com Zero To One by Peter Thiel http://zerotoonebook.com Primer on the Greek financial crisis http://faculty.chicagobooth.edu/anil.kashyap/research/papers/A-Primer-on-the-Greek-Crisis_june29.pdf SPONSOR Bitcoin Keychains by Bkeychain You've seen these keychains on dozens and dozens of websites, it's about time you had one of your own! These substantial metal keychains make great conversation starters, and they also make great gifts to or from Bitcoiners. You can find a list of online retailers at Bkeychain.com, and several support Bitcoin so much, they don't even accept fiat currency. So what are you waiting for? http://Bkeychain.com MUSIC All music in this episode was created by me, or with friends and family. Ganesh Painting Company is the name of one of the jam bands I feature live recordings of regularly. Some of the musicians you're hearing are Mike Coleman and Steve Lunn. Feel free to contact me if you want more info about any music you hear on the podcast. Thanks for listening! STAY IN TOUCH https://Twitter.com/TheBTCGame http://TheBitcoinGame.com Email me at Rob at TheBitcoinGame.com BOUNTY FOR TYPOS AND ERRORS Found an error or typo on this page? Is the magic word not working for you? Be the first to let me know privately (such as sending me an email or private message), and I'll send you some LTBcoin. The Bitcoin Game box artwork created from an illustration by Rock Barcellos.

Fumiaki Yoshimatsu さんをゲストに迎えて、電王戦、Apple Watch, Context Aware, Facebook Messenger, Crashlytics, Android Studio などについて話しました。 Show Notes 将棋:電王戦棋士側勝利呼んだ「わざと隙見せる作戦」 山本 一成@Ponanza After Seven Hours and 19 Innings, One Hit Sinks the Yankees Google Now ColorSync Ticketing Reimagined | Peatix Daring Fireball: The Apple Watch Apple Watch: the definitive review | The Verge 林信行による世界先行レビュー：Apple Watchが腕時計とウェアラブルの概念を変える (1/5) - ITmedia Apple Watch Debut Tests Cook’s No-Lines Gadget-Shopping Revamp Facebook Launches Messenger for Web Browsers Facebook's Messenger platform will let you download apps, message businesses Facebook shuts down XMPP Chat API LINE、ついにiPad用アプリケーションを投入 Path Talk lets you text businesses instead of calling, and it actually works TaskRabbit Amazon Mechanical Turk Outbox Shuts Down Its Mail Digitizing Service Crashlytics Fabric - Twitter's Mobile Development Platform Flock: Bringing Fabric to a city near you Welcome to Fabric! The Pants Build Tool at Twitter Pants Build IntelliJ IDEA — The Most Intelligent Java IDE Download Android Studio and SDK Tools 第14回　エディタの話［その6］─インスペクション：Android Studio最速入門

We need validation. Leave a review: iTunes | Stitcher John Pienta, Aline Sandouk, and Kaci McCleary (Ethan Forsgren joined in later) debate the merits of Iowa's recently defeated measure that would have allowed PhD psychologists to prescribe psych meds.  Would they be able to deal with co-morbidities? Would an education course be enough to cope with the complexities of psychiatric medications?  Do psych meds function at a level so fundamental to the operation of the human brain that allowing people without a certain basic level of psychiatric education would be too dangerous, or are prescribing algorithms enough? Then, a Canadian researcher comes up with a topical cream that eats tattoos.  Fun science fact: people can only see about 30 shades of grey, which leads to a discussion of one man's job to cull porn from a social networking site, Amazon's Mechanical Turk service, and vigilantism.   Exploding Head Syndrome and Alien Hand Syndrome are explored.   News shows try, as they do, to create a trend out of nothing, in this case about bodybuilders using breast milk as a supplement. Science discovers a bit of DNA that leads to bigger brains. Humans Can Only Distinguish Between about 30 Shades of Grey Alec Falkenham, Dalhousie student, develops tattoo removal cream Why Body Builders Are Pounding Down Breast Milk Amazon Mechanical Turk Exploding Head Syndrome – Overview & Facts Just A Bit Of DNA Helps Explain Humans' Big Brains [huge_it_gallery id=”19″] Listen to more great shows for medical students on The Vocalis Podcast Network. The opinions expressed in this feed and podcast are not those of the University of Iowa or the Roy J. and Lucille A. Carver College of Medicine; nor do they reflect the views of anyone other than the people who expressed them.  If you have feedback on anything you hear on the show, positive or not, let us know.…

In this episode Terry Lamb talks about using Amazon Mechanical Turk to outsource simple tasks online on the cheap, and discusses an example that can help your YouTube marketing and increase online engagement.

Imagine having an army of workers ready to do your bidding, no matter how menial and trivial, as long as you pay them some semblance of a decent wage. That's Amazon Mechanical Turk in a nutshell, with which people are doing strange and wonderful things. To discuss this and other randomness, we had the pleasure […] The post Episode 077: The Untimely Death of Stephen first appeared on Bad Philosophy.

Imagine having an army of workers ready to do your bidding, no matter how menial and trivial, as long as you pay them some semblance of a decent wage. That’s Amazon Mechanical Turk in a nutshell, with which people are doing strange and wonderful things. To discuss this and other randomness, we had the pleasure […]