Machine Learning - Introduction

What is machine learning?

A learning machine, broadly defined, is any device whose actions are influenced by past experiences.

— Nils J. Nilsson

Tom M. Mitchell provided a widely quoted, more formal definition of the algorithms studied in the machine learning field: "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E."[Mitchell, T. (1997). Machine Learning. McGraw Hill. p. 2.]

Machine Learning is using generic algorithms to tell you something interesting about your data without writing any code specific to the problem you are solving.

The below is a similar explanation. You need not write the program to do the work required.

Machine learning, as a type of artificial intelligence (AI), enables computers to learn without being explicitly programmed, and to improve their functions when exposed to new data. By analyzing patterns in this data, the machine learning algorithms are self-adjusting based on a set of design rules.

http://www.softvision.com/blog/what-is-machine-learning/

Machine learning was initially part of AI. But AI abandoned it. Machine learning (ML), reorganized and recognized as its own field, started to flourish in the 1990s. The field changed its goal from achieving artificial intelligence to tackling solvable problems of a practical nature. It shifted focus away from the symbolic approaches it had inherited from AI, and toward methods and models borrowed from statistics, fuzzy logic, and probability theory.

Machine learning and Data mining

Machine learning and data mining often employ the same methods and overlap significantly, but while machine learning focuses on prediction, based on known properties learned from the training data, data mining focuses on the discovery of (previously) unknown properties in the data (this is the analysis step of knowledge discovery in databases).

In machine learning, performance is usually evaluated with respect to the ability to reproduce known knowledge, while in knowledge discovery and data mining (KDD) the key task is the discovery of previously unknown knowledge. 

https://en.wikipedia.org/wiki/Machine_learning

Statistics

Machine learning and statistics are closely related fields in terms of methods, but distinct in their principal goal: statistics draws population inferences from a sample, while machine learning finds generalizable predictive patterns. According to Michael I. Jordan, the ideas of machine learning, from methodological principles to theoretical tools, have had a long pre-history in statistics. He also suggested the term data science as a placeholder to call the overall field.

https://web.archive.org/web/20171018192328/https://www.reddit.com/r/MachineLearning/comments/2fxi6v/ama_michael_i_jordan/ckelmtt/?context=3

Articles in Medium by Adam Geitgey

1
https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec3c471#.w55suff6b

Machine Learning is using generic algorithms to tell you something interesting about your data without writing any code specific to the problem you are solving.

2

https://medium.com/@ageitgey/machine-learning-is-fun-part-2-a26a10b68df3


3

https://medium.com/@ageitgey/machine-learning-is-fun-part-3-deep-learning-and-convolutional-neural-networks-f40359318721

Deep Learning and Convolutional Neural Networks

4

https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-recognition-with-deep-learning-c3cffc121d78

A method invented in 2005 called Histogram of Oriented Gradients — or just HOG for short.

Algorithm - face landmark estimation.

5

https://medium.com/@ageitgey/machine-learning-is-fun-part-5-language-translation-with-deep-learning-and-the-magic-of-sequences-2ace0acca0aa

Sequence-to-sequence learning.

Statistical machine translation systems perform much better than rule-based systems if you give them enough training data. Franz Josef Och improved on these ideas and used them to build Google Translate in the early 2000s. Machine Translation was finally available to the world.

A recurrent neural network (or RNN for short) is a slightly tweaked version of a neural network where the previous state of the neural network is one of the inputs to the next calculation. This means that previous calculations change the results of future calculations!

The idea of turning a face into a list of measurements is an example of an encoding. We are taking raw data (a picture of a face) and turning it into a list of measurements that represent it (the encoding).

6

https://medium.com/@ageitgey/machine-learning-is-fun-part-6-how-to-do-speech-recognition-with-deep-learning-28293c162f7a

The algorithm (roughly) described here to deal with variable-length audio is called Connectionist Temporal Classification or CTC. 

7

The new system is called Deep Convolutional Generative Adversarial Networks (or DCGANs for short).

How DCGANs work

To build a DCGAN, we create two deep neural networks. Then we make them fight against each other, endlessly attempting to out-do one another. In the process, they both become stronger.

---------------------

Machine Learning Explained
https://blog.dataiku.com/machine-learning-explained-algorithms-are-your-friend

Machine Learning MIT Course - Course Materials

http://www.ai.mit.edu/courses/6.867-f04/lectures.html

Machine Learning Cheatsheet - SAS

https://blogs.sas.com/content/subconsciousmusings/2017/04/12/machine-learning-algorithm-use/



How To Become A Machine Learning Engineer: Learning Path
Aug 19, 2017
https://hackernoon.com/learning-path-for-machine-learning-engineer-a7d5dc9de4a4


Machine learning - Notes
http://www.holehouse.org/mlclass/


Updated 18.11.2023,  21 July 2021,  15 July 2018,   24 June 2018,  6 October 2017,  23 August 2017, 30 July 2016

Data Science - Books - Bibliography - Introduction

 

https://towardsdatascience.com/learn-on-towards-data-science-52245bc91451

Practitioner’s Guide to Data Science

By Hui Lin, Ming Li

1st Edition

First Published 2023

eBook Published 24 May 2023

Based on industry experience, this book outlines real-world scenarios and discusses pitfalls that data science practitioners should avoid. It also covers the big data cloud platform and the art of data science, such as soft skills. The authors use R as the primary tool and provide code for both R and Python.　

This book is for readers who want to explore possible career paths and eventually become data scientists. This book comprehensively introduces various data science fields, soft and programming skills in data science projects, and potential career paths. Traditional data-related practitioners such as statisticians, business analysts, and data analysts will find this book helpful in expanding their skills for future data science careers. Undergraduate and graduate students from analytics-related areas will find this book beneficial to learn real-world data science applications. Non-mathematical readers will appreciate the reproducibility of the companion R and python codes.

Key Features:

• It is hands-on. We provide the data and　repeatable　R and Python code in notebooks. Readers can repeat the analysis in the book using the data and code provided. We also suggest that readers modify the notebook to perform analyses with their data and problems, if possible. The best way to learn data science is to do it!

TABLE OF CONTENTS

Chapter 1|28 pages

Introduction

 

Chapter 2|18 pages

Soft Skills for Data Scientists

 

Chapter 3|8 pages

Introduction to the Data

 

Chapter 4|22 pages

Big Data Cloud Platform

 

Chapter 5|26 pages

Data Pre-processing

 

Chapter 6|22 pages

Data Wrangling

 

Chapter 7|26 pages

Model Tuning Strategy

 

Chapter 8|16 pages

Measuring Performance

 

Chapter 9|20 pages

Regression Models

 

Chapter 10|30 pages

Regularization Methods

 

Chapter 11|42 pages

Tree-Based Methods

 

Chapter 12|78 pages

Deep Learning

 

https://linhui.org/hui's_files/datascientist1#(20)

https://scholar.google.com/citations?user=PAArLQIAAAAJ&hl=en&oi=sra

https://scholar.google.com/citations?user=PAArLQIAAAAJ&hl=en

https://linhui.org/

https://github.com/happyrabbit

https://scientistcafe.com/

A Tour of Data Science: Learn R and Python in Parallel

Nailong Zhang

CRC Press, 11-Nov-2020 - Computers - 216 pages (C) 2021.

A Tour of Data Science: Learn R and Python in Parallel covers the fundamentals of data science, including programming, statistics, optimization, and machine learning in a single short book. It does not cover everything, but rather, teaches the key concepts and topics in Data Science. It also covers two of the most popular programming languages used in Data Science, R and Python, in one source.

Key features:

Allows you to learn R and Python in parallel

Cover statistics, programming, optimization and predictive modelling, and the popular data manipulation tools – data table and pandas

Provides a concise and accessible presentation

Includes machine learning algorithms implemented from scratch, linear regression, lasso, ridge, logistic regression, gradient boosting trees, etc.

Appealing to data scientists, statisticians, quantitative analysts, and others who want to learn programming with R and Python from a data science perspective.

https://books.google.co.in/books?id=zVYAEAAAQBAJ

A Hands-On Introduction to Data Science

Chirag Shah

Cambridge University Press, 02-Apr-2020 - Business & Economics - 424 pages

This book introduces the field of data science in a practical and accessible manner.

The foundational ideas and techniques of data science are provided  allowing students to easily develop a firm understanding of the subject. The material that will have continual relevance even after tools and technologies change. 

Using popular data science tools such as Python and R, the book offers many examples of real-life applications, with practice ranging from small to big data. A suite of online material for both instructors and students provides a strong supplement to the book, including datasets, chapter slides, solutions, sample exams and curriculum suggestions. This entry-level textbook is ideally suited to readers from a range of disciplines wishing to build a practical, working knowledge of data science.

https://books.google.co.in/books?id=rljPDwAAQBAJ

Data Science Job: How to become a Data Scientist

Przemek Chojecki, 31-Jan-2020 - Computers - 100 pages

Data Scientist is one of the hottest job on the market right now. Demand for data science is huge and will only grow, and it seems like it will grow much faster than the actual number of data scientists. So if you want to make a career change and become a data scientist, now is the time.

This book will guide you through the process. From my experience of working with multiple companies as a project manager, a data science consultant or a CTO, I was able to see the process of hiring data scientists and building data science teams. I know what’s important to land your first job as a data scientist, what skills you should acquire, what you should show during a job interview.

https://books.google.co.in/books?id=h0PZDwAAQBAJ

Foundations of Data Science

Avrim Blum, John Hopcroft, Ravindran Kannan

Cambridge University Press, 23-Jan-2020 - Computers - 432 pages

This book provides an introduction to the mathematical and algorithmic foundations of data science, including machine learning, high-dimensional geometry, and analysis of large networks. 

Topics include the counterintuitive nature of data in high dimensions, important linear algebraic techniques such as singular value decomposition, the theory of random walks and Markov chains, the fundamentals of and important algorithms for machine learning, algorithms and analysis for clustering, probabilistic models for large networks, representation learning including topic modelling and non-negative matrix factorization, wavelets and compressed sensing. 

Important probabilistic techniques are developed including the law of large numbers, tail inequalities, analysis of random projections, generalization guarantees in machine learning, and moment methods for analysis of phase transitions in large random graphs. Additionally, important structural and complexity measures are discussed such as matrix norms and VC-dimension. This book is suitable for both undergraduate and graduate courses in the design and analysis of algorithms for data.

https://books.google.co.in/books?id=koHCDwAAQBAJ

Data Science and Intelligent Applications: Proceedings of ICDSIA 2020

Ketan Kotecha, Vincenzo Piuri, Hetalkumar N. Shah, Rajan Patel

Springer Nature, 17-Jun-2020 - Technology & Engineering - 576 pages

This book includes selected papers from the International Conference on Data Science and Intelligent Applications (ICDSIA 2020), hosted by Gandhinagar Institute of Technology (GIT), Gujarat, India, on January 24–25, 2020. The proceedings present original and high-quality contributions on theory and practice concerning emerging technologies in the areas of data science and intelligent applications. The conference provides a forum for researchers from academia and industry to present and share their ideas, views and results, while also helping them approach the challenges of technological advancements from different viewpoints.

The contributions cover a broad range of topics, including: collective intelligence, intelligent systems, IoT, fuzzy systems, Bayesian networks, ant colony optimization, data privacy and security, data mining, data warehousing, big data analytics, cloud computing, natural language processing, swarm intelligence, speech processing, machine learning and deep learning, and intelligent applications and systems. Helping strengthen the links between academia and industry, the book offers a valuable resource for instructors, students, industry practitioners, engineers, managers, researchers, and scientists alike.

p.217 Human activity recognition

https://books.google.co.in/books?id=eSbsDwAAQBAJ

© 2020

Data Science and Productivity Analytics

Editors: Charles, Vincent, Aparicio, Juan, Zhu, Joe (Eds.)

Table of contents (15 chapters)

Data Envelopment Analysis and Big Data: Revisit with a Faster Method Pages 1-34

Khezrimotlagh, Dariush (et al.)

Data Envelopment Analysis (DEA): Algorithms, Computations, and Geometry Pages 35-56

Dulá, José H.

An Introduction to Data Science and Its Applications  Pages 57-81

Rabasa, Alex (et al.)

Identification of Congestion in DEA Pages 83-119

Mehdiloo, Mahmood (et al.)

Data Envelopment Analysis and Non-parametric Analysis Pages 121-160

Villa, Gabriel (et al.)

The Measurement of Firms’ Efficiency Using Parametric Techniques Pages 161-199

Orea, Luis

Fair Target Setting for Intermediate Products in Two-Stage Systems with Data Envelopment Analysis

Pages 201-226

An, Qingxian (et al.)

Fixed Cost and Resource Allocation Considering Technology Heterogeneity in Two-Stage Network Production Systems Pages 227-249

Ding, Tao (et al.)

Efficiency Assessment of Schools Operating in Heterogeneous Contexts: A Robust Nonparametric Analysis Using PISA 2015 Pages 251-277

Cordero, Jose Manuel (et al.)

A DEA Analysis in Latin American Ports: Measuring the Performance of Guayaquil Contecon Port 

Pages 279-309

Morales-Núñez, Emilio J. (et al.)

Effects of Locus of Control on Bank’s Policy—A Case Study of a Chinese State-Owned Bank 

Pages 311-335

Xu, Cong (et al.)

A Data Scientific Approach to Measure Hospital Productivity Pages 337-358

Daneshvar Rouyendegh (B. Erdebilli), Babak (et al.)

Environmental Application of Carbon Abatement Allocation by Data Envelopment Analysis Pages 359-389

Yu, Anyu (et al.)

Pension Funds and Mutual Funds Performance Measurement with a New DEA (MV-DEA) Model Allowing for Missing Variables Pages 391-413

Badrizadeh, Maryam (et al.)

Sharpe Portfolio Using a Cross-Efficiency Evaluation Pages 415-439

Landete, Mercedes (et al.)

https://www.springer.com/gp/book/9783030433833

Special Issue on Data Science for Better Productivity

Data science for better productivity

Vincent Charles,Juan Aparicio &Joe Zhu 

Journal of the Operational Research Society 

Volume 72, 2021 - Issue 5: Special Issue Data Science for Better Productivity

Afsharian, M. (2019). A frontier-based facility location problem with a centralised view of measuring the performance of the network. Journal of the Operational Research Society, 72(5), 1058–1074. https://doi.org/10.1080/01605682.2019.1639476   

Bougnol, M.-L., & Dulà, J. (2020). Improving productivity using government data: The case of US Centers for Medicare & Medicaid's ‘Nursing Home Compare. Journal of the Operational Research Society, 72(5), 1075–1086. https://doi.org/10.1080/01605682.2020.1724056   

Del Vecchio, M., Kharlamov, A., Parry, G., & Pogrebna, G. (2020). Improving productivity in Hollywood with data science: Using emotional arcs of movies to drive product and service innovation in entertainment industries. Journal of the Operational Research Society, 72(5), 1110–1137. https://doi.org/10.1080/01605682.2019.1705194   

Grimaldi, D., Fernandez, V., & Carrasco, C. (2019). Exploring data conditions to improve business performance. Journal of the Operational Research Society, 72(5), 1087–1098. https://doi.org/10.1080/01605682.2019.1590136   

Ihrig, S., Ishizaka, A., Brech, C., & Fliedner, T. (2019). A new hybrid method for the fair assignment of productivity targets to indirect corporate processes. Journal of the Operational Research Society, 72(5), 989–1001. https://doi.org/10.1080/01605682.2019.1639477   

Jiang, R., Yang, Y., Chen, Y., & Liang, L. (2019). Corporate diversification, firm productivity and resource allocation decisions: The data envelopment analysis approach. Journal of the Operational Research Society, 72(5), 1002–1014. https://doi.org/10.1080/01605682.2019.1568841   

Li, Y., & Chen, W. (2019). Entropy method of constructing a combined model for improving loan default prediction: A case study in China. Journal of the Operational Research Society, 72(5), 1099–1109. https://doi.org/10.1080/01605682.2019.1702905   

Lin, S.-W., Lu, W.-M., & Lin, F. (2020). Entrusting decisions to the public service pension fund: An integrated predictive model with additive network DEA approach. Journal of the Operational Research Society, 72(5), 1015–1032. https://doi.org/10.1080/01605682.2020.1718011   

Routh, P., Roy, A., & Meyer, J. (2020). Estimating customer churn under competing risks. Journal of the Operational Research Society, 72(5), 1138–1155. https://doi.org/10.1080/01605682.2020.1776166   

Shi, Y., Zhu, J., & Charles, V. (2020). Data science and productivity: A bibliometric review of data science applications and approaches in productivity evaluations. Journal of the Operational Research Society, 72(5), 975–988. https://doi.org/10.1080/01605682.2020.1860661   

Summerfield, N. S., Deokar, A. V., Xu, M., & Zhu, W. (2020). Should drivers cooperate? Performance evaluation of cooperative navigation on simulated road networks using network DEA. Journal of the Operational Research Society, 72(5), 1042–1057. https://doi.org/10.1080/01605682.2019.1700766   

Zhu, J. (2020). DEA under big data: Data enabled analytics and network data envelopment analysis. Annals of Operations Research, 1–23. In press. https://doi.org/10.1007/s10479-020-03668-8 

Zhu, W., Liu, B., Lu, Z., & Yu, Y. (2020). A DEALG methodology for prediction of effective customers of internet financial loan products. Journal of the Operational Research Society, 72(5), 1033–1041. https://doi.org/10.1080/01605682.2019.1700188 [Taylor & Francis On 

https://www.tandfonline.com/doi/full/10.1080/01605682.2021.1892466

Ud. 16.11,2023, 3.45 am Austin, Texas

Pub. 16.7.2021

Top IoT Systems and Components Vendors

IIoT Platforms Gartner

By 2025, 50% of industrial enterprises will use industrial Internet of Things (IIoT) platforms to improve factory operations, up from 10% in 2020.

Market Definition/Description

Gartner defines the IIoT platform market as a set of integrated software capabilities to improve asset management decision making within asset-intensive industries. IIoT platforms also provide operational visibility and control for plants, infrastructure and equipment.

IIoT Platforms

The IIoT platform  cost-effectively collects higher volumes of high-velocity, complex machine data from networked IoT endpoints. The IIoT platform also orchestrates historically siloed data sources to enable better accessibility, and improve insights and actions across a heterogeneous asset group through specialized analysis of the data.

The IIoT platform:

Monitors IoT endpoints and event streams

Analyzes data at the edge and in the cloud

Integrates and engages IT and OT systems in data sharing and consumption

Enables application development and deployment

Can enrich and supplement OT functions for improved asset management life cycle strategies and processes

The IIoT platform, in concert with the IoT edge and through enterprise IT/OT integration, prepares asset-intensive industries to become digital businesses. Digital capabilities are achieved by enhancing and connecting their core business with customers, suppliers and business partners.

The IIoT platform software that resides on and near devices — such as controllers, routers, access points, gateways and edge compute systems — is considered part of the “distributed IIoT platform.”

The platform provider must exhibit demonstrable value in terms of integration and interoperability with such applications, which include:

Enterprise asset management (EAM)

Computerized maintenance management systems (CMMSs)

Fleet management

Condition-based maintenance (CBM)

Manufacturing execution systems (MES)

Maintenance, repair and operations (MRO)

Product life cycle management (PLM)

Application portfolio management (APM)

Field service management (FSM)

Building management systems (BMSs)

IIoT Platform Capabilities

The IIoT platform  is composed of the following technology functions:

Device management — This function includes software that enables manual and automated tasks to create, provision, configure, troubleshoot and manage fleets of IoT devices and gateways remotely, in bulk or individually, and securely.

Integration — This function includes software, tools and technologies, such as communications protocols, APIs and application adapters, which minimally address the data, process, enterprise application and IIoT ecosystem integration requirements across cloud and on-premises implementations for end-to-end IIoT solutions. These IIoT solutions include IIoT devices (for example, communications modules and controllers), IIoT gateways, IIoT edge and IIoT platforms.

Data management — This function includes capabilities that support:

Ingesting IoT endpoint and edge device data

Storing data from edge to enterprise platforms

Providing data accessibility (by devices, IT and OT systems, and external parties, when required)

Tracking lineage and flow of data

Enforcing data and analytics governance policies to ensure the quality, security, privacy and currency of data

Analytics — This function includes processing of data streams, such as device, enterprise and contextual data, to provide insights into asset state by monitoring use, providing indicators, tracking patterns and optimizing asset use. A variety of techniques, such as rule engines, event stream processing, data visualization and machine learning, may be applied.

Application enablement and management — This function includes software that enables business applications in any deployment model to analyze data and accomplish IoT-related business functions. Core software components manage the OS, standard input and output or file systems to enable other software components of the platform. The application platform (for example, application platform as a service [aPaaS]) includes application-enabling infrastructure components, application development, runtime management and digital twins. The platform allows users to achieve “cloud scale” scalability and reliability and deploy and deliver IoT solutions quickly and seamlessly.

Security — This function includes the software, tools and practices facilitated to audit and ensure compliance. This function also establishes preventive, detective and corrective controls and actions to ensure privacy and the security of data across the IIoT solution.

2019

https://www.gartner.com/reviews/market/industrial-iot-platforms

Hitachi Again Named a “Visionary” in Gartner Magic Quadrant for IIoT Platforms 2019

https://www.hitachivantara.com/ext/gartner-magic-quadrant-for-industrial-iot.html

https://www.ptc.com/en/resources/iiot/white-paper/gartner-mq-for-iiot

--------------------------

IBM

Google

Intel

Microsoft

Cisco

Apple

SAP

Oracle

Samsung

Hewlett Packard

Ericson

Amazon.Com

GE

Qualcomm

AT&T

Orange

Blackberry

Facebook

Dell

Verizon

--------------------

News

February 2016

http://www.ecommercetimes.com/story/83088.html


IoT Players

http://electronicsofthings.com/category/industry-players/

http://internetofthingswiki.com/iot-companies-you-must-know/653



5 Oct 2020
25 March 2016

AI, Machine Learning & Deep Learning - Education - Training Programs - USA

https://www.eventbrite.com/e/technical-introduction-to-ai-machine-learning-deep-learning-tickets-34671486349

https://www.facebook.com/events/1729161024041553



Technical Introduction to AI, Machine Learning & Deep Learning
Engineered Education
Friday, July 28, 2017 from 9:00 AM to 7:00 PM (PDT)
San Francisco, CA
TICKET TYPE SALES END PRICE FEE QUANTITY
Registration $495.00 $13.37
Team Discount (4 or more registrations)   $349.00 $9.72



This workshop will arm you with the tools to get started using machine learning in your day job and the resources to find additional help if you want to go deeper.
The course is expertly designed to leave you with the ability to take training data, do feature selection and actually build models for applications like content categorization, sentiment analysis, and image recognition. By the end of the day, students will be able to use models in their day-to-day work. You will also walk away with a high-level understanding of how common models such as Deep Neural Networks, SVMs, Logistic Regression and Naive Bayes work and when to use them.

Technologies Introduced

Intro to Machine Learning
Scikit-learn
Numpy
Pandas

Intro to Deep Learning
TensorFlow
Keras

Intro to Machine Learning Platforms
Google Cloud ML
Azure ML
Amazon ML

Prerequisites
We try to make this class as accessible as possible. Some proficiency with Python is necessary. If you can open up a Jupyter notebook and install requisite software that’s helpful but we’ll also cover how to do that quickly in the beginning.



What you Need to bring

You must also bring your own laptop (don’t forget your charger).

Preparation
It saves a lot of time if you can get your laptop setup in advance.  If you can't get everything setup, try to come early and we'll help you with the installation.

Download code for the class from https://github.com/lukas/ml-class.

There are instructions on this website for how to install all the necessary programs at https://github.com/lukas/ml-class/blob/master/README.md - if you have questions, you can email us or put them in the github issues tracker where they might help another student.

Teacher
Lukas Biewald:  Lukas Biewald is the founder of CrowdFlower, an Artificial Intelligence company that works with data science teams at Google, Bloomberg, Facebook and hundreds of other organizations to make machine learning work in the real world. Prior to that, Lukas was the first data scientist at Powerset (Acquired by Microsoft and rebranded as Bing) and a scientist at Yahoo!, Lukas was shipping machine learning algorithms to hundreds of millions of users.

Lukas frequently teaches invited Machine Learning workshops with Galvanize, O’Reilly and ODSC. He is a frequent contributor to Computerworld, Forbes and O’Reilly and has presented at the machine learning academic conferences such as AAAI, SIGIR, ACL and EMNLP. He was in Inc’s annual 30 under 30 and was also a finalist at TechCrunch Disrupt.

Curriculum
9:00 – 10:00 Breakfast and Intro to Machine Learning
We will assume no knowledge of Machine Learning, so we'll go over terminology and the history of Machine Learning and Artificial Intelligence.  We'll talk about the common use cases and how they fit in with the different Machine Learning algorithms.

10:00 – 12:00 Build a Sentiment Classifier From Scratch
Everyone builds a Twitter sentiment classifier using scikit-learn. We try multiple feature selection approaches and multiple model types. We learn some common tricks for actually making machine learning effective in the real world.

12:00-1:00 Lunch and Overview of State Machine Learning
Eat lunch and for your eating entertainment, Lukas will introduce a little math, stats and history of how machine learning got to where it is today.  We will go over the state of machine learning platforms today and how to get an entry-level job in machine learning for those that are interested.

1:00-2:30 Try the Common Machine Learning Platforms
These days, there are many excellent, scalable, low cost machine learning platforms. We will try rebuilding our sentiment classifier on two of the most common: Microsoft Azure ML and Amazon ML.

2:30-3:00 Break and Q&A
We can discuss other applications of this technology and look at how it might apply to real-world tasks that students may be working on.

3:00-5:00 Introduction to TensorFlow and Deep Neural Networks
We will learn how deep neural networks work and actually build one! If you bring a laptop with a GPU that supports CUDA (for example a MacBook with Mac OS X 10.11 or later), we’ll see if we can make it GPU accelerated.
We’ll all build a network to do handwritten digit recognition.

5:00-5:30 Wrap-up and Q&A
We will finish up and discuss how to apply this knowledge directly to problems that we actually face in our jobs.

5:30-7:00 Drinks & Networking
We’ll bring together top entrepreneurs, tech executives & engineers to connect with and learn from. Plus, this is a chance to meet your classmates and teachers in an informal and fun setting.

Senior Analytics Scientist - Risk Analytics - Job Specification

A Analytics-driven e-commerce company is looking for:

Job Title: Senior Analytics Scientist - Risk Analytics


Role Outline
Senior Analytics Scientist - Risk Analytics and reports to the Sr. Mgr / Director leading the team.

The key requirement for the role is the ability to understand the business, develop data driven solutions to address business problems and provide analytic support to the risk analytics group. The individual will possess the ability to work in teams and display a proactive learning attitude.

Job Description
Job Title : Senior Analytics Scientist - Risk Analytics
Department : Risk Analytics
Reports To : Sr. Manager / Director


Key responsibilities
* Key responsibilities include
o Building models to predict risk and other key metrics
o Coming up with data driven solutions to control risk
o Finding opportunities to acquire more customers by modifying/optimizing existing rules
o Doing periodic upgrades of the underwriting strategy based on business requirements
o Evaluating 3rd party solutions for predicting/controlling risk of the portfolio
o Running periodic controlled tests to optimize underwriting
o Monitoring key portfolio metrics and take data driven actions based on the performance
* Business Knowledge: Develop an understanding of the domain/function. Manage business process (es) in the work area. The individual is expected to develop domain expertise in his/her work area.
* Teamwork: Develop cross site relationships to enhance leverage of ideas. Set and manage partner expectations. Drive implementation of projects with Engineering team while partnering seamlessly with cross site team members.
* Communication: Responsibly perform end to end project communication across the various levels in the organization.

Candidate Specification:
Skills:
* Should have solid understanding of probability and stats; Bayesian methods, probability distributions, Central limit theorem etc.
* Should be familiar with some of the following GLM, logistic regression, Random forest, Gradient boosting trees, CART, Naïve bayes, Linear Program, Mixed Integer program, etc.
* Knowledge of analytical tools such as R/Python/SAS/SQL
* Experience in handling complex data sources
* Dexterity with MySQL, MS Excel
* Strong Analytical aptitude and logical reasoning ability
* Strong presentation and communication skills.
* Strong process/project management skill

Preferred:
* 3 - 5 years of experience in Financial Services/Analytics Industry/ecommerce
* Understanding of the financial services business

Essential Skills for Data Science and Data Analytics Careers

1. Statistics/Mathematics

2. Data Mining/Machine Learning/Artificial Intelligence

Data Analytics and Data Mining - Difference Explained


Machine Learning - Blog

3. Data Frameworks like Hadoop

What is Hadoop? Text and Video Lectures

4. Special programming languages for statistics  - R and Python

5. Database and SQL skills for Structured data.

6. NoSQL skills for unstructured social media data.

NoSQL Databases

http://www.kdnuggets.com/2016/05/10-must-have-skills-data-scientist.html

http://www.kdnuggets.com/2014/11/9-must-have-skills-data-scientist.html

Updated 26 June 2016

NoSQL Databases

The term was started in 2009

10 things you should know about NoSQL databases
By Guy Harrison, August 26, 2010
http://www.techrepublic.com/blog/10-things/10-things-you-should-know-about-nosql-databases/


Introduction - Presentation
http://martinfowler.com/articles/nosql-intro-original.pdf


2013  Introduction to NoSQL - Video Presentation by Martin Fowler

__________________

__________________

GOTO Conferences

http://martinfowler.com/nosql.html

https://www.mongodb.com/

http://www.nosqlweekly.com/  - Subscribe to their weekly. They provide a python weekly also.

Data Science Competitions

https://www.kaggle.com/competitions



Active Competitions on 3 June 2016
_____________________________

• Ultrasound Nerve Segmentation

  Identify nerve structures in ultrasound images of the neck

  2 months

  156 teams

  199 scripts

  $100,000
• 

  Draper Satellite Image Chronology

  Can you put order to space and time?

  24 days

  272 teams

  368 scripts

  $75,000
• 

  State Farm Distracted Driver Detection

  Can computer vision spot distracted drivers?

  59 days

  928 teams

  597 scripts

  $65,000
• 

  Expedia Hotel Recommendations

  Which hotel type will an Expedia customer book?

  7.3 days

  1922 teams

  2220 scripts

  $25,000
• 

  Avito Duplicate Ads Detection

  Can you detect duplicitous duplicate ads?

  38 days

  276 teams

  191 scripts

  $20,000

• 

  Facebook V: Predicting Check Ins

  Machine Learning Software Engineer at 

  Facebook

  Menlo Park, CA or Seattle, WA

  33 days

  589 teams

  1294 scripts

  Jobs

• 

  San Francisco Crime Classification

  Predict the category of crimes that occurred in the city by the bay

  3.3 days

  2274 teams

  2267 scripts

  Knowledge
• 

  Kobe Bryant Shot Selection

  Which shots did Kobe sink?

  10 days

  648 teams

  588 scripts

  Knowledge
• 

  Integer Sequence Learning

  1, 2, 3, 4, 5, 7?!

  3 months

  9 teams

  10 scripts

  Knowledge
• 

  Painter by Numbers

  Does every painter leave a fingerprint?

  5 months

  10 teams

  32 scripts

  Knowledge
• 

  Shelter Animal Outcomes

  Help improve outcomes for shelter animals

  58 days

  749 teams

  573 scripts

  Knowledge

• 

  Health Insurance Marketplace

  Explore health and dental plans data in the US Health Insurance Marketplace

  214 scripts

  3140downloads
• 

  World of Warcraft Avatar History

  Track the players of this popular online game

  28 scripts

  184 downloads
• 

  Annual Nominal Fish Catches

  Explore the impact of overfishing in the Northeast Atlantic region.

  2 scripts

  63 downloads

• 

  Digit Recognizer

  Classify handwritten digits using the famous MNIST data

  7 months

  1344 teams

  4439 scripts

  Knowledge
• 

  Titanic: Machine Learning from Disaster

  Predict survival on the Titanic using Excel, Python, R & Random Forests

  7 months

  4278 teams

  5585 scripts

  Knowledge
• 

  Facial Keypoints Detection

  Detect the location of keypoints on face images

  7 months

  118 teams

  Knowledge
• 

  First Steps With Julia

  Use Julia to identify characters from Google Street View images

  7 months

  53 teams

  Knowledge

______________________________

R Language Commands - Input and Display

# is for comments
R ignores everything that appears after the hash (#).
The assignment operator is symbol for less than followed by hash. (The blogger is creating a problem by giving strange interpretation of the symbol.)
#read files with labels in first row

read.table(filename,header=TRUE)           #read a tab or space delimited file
read.table(filename,header=TRUE,sep=',')   #read csv files

<- a="" c="" create="" data="" elements="" nbsp="" p="" specified="" vector="" with=""><- 1="" a="" c="" create="" data="" elements="" nbsp="" p="" to10="" vector="" with=""><- 10="" p=""><- a="" c="" create="" deviates="" item="" n="" nbsp="" normal="" of="" p="" random="" rnorm="" vector=""><- a="" added="" c="" create="" distribution="" each="" has="" item="" n="" nbsp="" p="" random="" runif="" that="" to="" uniform="" vector=""><- binomial="" create="" from="" n="" nbsp="" of="" p="" prob="" probability="" rbinom="" samples="" size="" the="" with=""><- and="" c="" combine="" into="" length="" nbsp="" of="" one="" p="" vector="" vectors="" x="" y=""><- 2="" a="" and="" cbind="" combine="" into="" matrix="" n="" nbsp="" p="" x="" y="">

mat[4,2]                                   #display the 4th row and the 2nd column
mat[3,]                                    #display the 3rd row
mat[,2]                                    #display the 2nd column


subset(dataset,logical)                    #those objects meeting a logical criterion
subset(data.df,select=variables,logical)   #get those objects from a data frame that meet a criterion
data.df[data.df=logical]                   #yet another way to get a subset
x[order(x$B),]                             #sort a dataframe by the order of the elements in B
x[rev(order(x$B)),]                        #sort the dataframe in reverse order

browse.workspace                           #a Mac menu command that creates a window with information about all variables in the workspace

Details

Input and display
#read files with labels in first row
read.table(filename,header=TRUE)           #read a tab or space delimited file
read.table(filename,header=TRUE,sep=',')   #read csv files

x <- a="" c="" create="" data="" elements="" nbsp="" p="" specified="" vector="" with="">To construct a vector
 > c(1,2,3,4,5)
[1] 1 2 3 4 5

y <- 1="" a="" c="" create="" data="" elements="" nbsp="" p="" to10="" vector="" with="">
> x <- 1:5="" nbsp="" p="">> x
[1] 1 2 3 4 5

assign the values 1:5 to a vector named x:
> x <- 1:5="" p="">> x
[1] 1 2 3 4 5

n <- 10="" p="">x1 <- a="" c="" create="" deviates="" item="" n="" nbsp="" normal="" of="" p="" random="" rnorm="" vector="">y1 <- a="" added="" c="" create="" distribution="" each="" has="" item="" n="" nbsp="" p="" random="" runif="" that="" to="" uniform="" vector="">z <- binomial="" create="" from="" n="" nbsp="" of="" p="" prob="" probability="" rbinom="" samples="" size="" the="" with="">vect <- and="" c="" combine="" into="" length="" nbsp="" of="" one="" p="" vector="" vectors="" x="" y="">

mat <- 2="" a="" and="" cbind="" combine="" into="" matrix="" n="" nbsp="" p="" x="" y="">mat[4,2]                                   #display the 4th row and the 2nd column
mat[3,]                                    #display the 3rd row
mat[,2]                                    #display the 2nd column


subset(dataset,logical)                    #those objects meeting a logical criterion
subset(data.df,select=variables,logical)   #get those objects from a data frame that meet a criterion
data.df[data.df=logical]                   #yet another way to get a subset

x[order(x$B),]                             #sort a dataframe by the order of the elements in B
x[rev(order(x$B)),]                        #sort the dataframe in reverse order

browse.workspace                           #a Mac menu command that creates a window with information about all variables in the workspace


Rules of Names of Variables, Vectors and Matrices in R

Names must start with a letter or a dot. If you start a name with a dot, the
second character can’t be a digit.

Names should contain only letters, numbers, underscore characters (_),
and dots (.). Although you can force R to accept other characters in names, you
shouldn’t, because these characters often have a special meaning in R.
You can’t use the following special keywords as names:
• break
• else
• FALSE
• for
• function
• if
• Inf
• NA
• NaN
• next
• repeat
• return
• TRUE
• while