Großartige Algorithmen, maschinelles Lernen und keine lineare Algebra


Ich unterrichte einen Kurs über fortgeschrittene Algorithmen und möchte einige Themen im Zusammenhang mit maschinellem Lernen behandeln, die für meine Studenten von Interesse sind. Infolgedessen würde ich gerne die Meinung der Leute zu den derzeit interessantesten / größten algorithmischen Ergebnissen beim maschinellen Lernen hören. Die potenziell knifflige Einschränkung besteht darin, dass die Schüler keine besonderen Vorkenntnisse in linearer Algebra oder den anderen Hauptthemen des maschinellen Lernens haben.

Dies soll sie wirklich für das Thema begeistern und sie wissen lassen, dass ML ein potenziell aufregendes Forschungsgebiet für Algorithmenexperten ist.

BEARBEITEN: Dies ist ein Grundstudiengang für das letzte Jahr (da es in Großbritannien hauptsächlich keine Grundstudiengänge gibt). Sie haben mindestens einen Grundkurs für Algorithmen im Voraus absolviert und es ist davon auszugehen, dass sie sich für den fortgeschrittenen Folgekurs entschieden haben. Der aktuelle Lehrplan des Fortgeschrittenenkurses umfasst Themen wie Perfect Hashing, Bloom-Filter, Van-Emde-Boas-Bäume, lineares Prog., Ca. Algorithmen für NP-schwierige Probleme usw. Ich habe nicht die Absicht, mehr als eine Vorlesung ausschließlich mit ML zu verbringen, aber wenn etwas für einen Algorithmenkurs und einen ML-Kurs wirklich relevant ist, könnte es natürlich auch enthalten sein.

Bitte klären Sie zwei Dinge: 1) Handelt es sich um ein Grundstudium oder ein Grundstudium? Welche verwandten Kurse (falls vorhanden) haben sie bestanden? 2) Wie viel Zeit möchten Sie ML widmen?
MS Dousti

hmmm Ich denke, lineare Algebra ist ein Muss und ein wichtiger Grundkurs, zumindest auf der Seite des maschinellen Lernens. und ich denke, das lineare Modell ist eine sehr gute Einführung in die Algorithmen des maschinellen Lernens. Sie können sich andere grundlegende Algorithmen wie K-Nearest Neighbor oder logistische Regressionsalgorithmen vorstellen.

Vielleicht haben einige Ideen aus , wie Hal Daume lehrt Machine Learning -
Yaroslav Bulatov

Dear Raphael, Avrim Blum typically concludes his senior-level algorithms class with machine learning and a few related topics; a recent iteration is at the following link , and you can get more info from his webpage. Having both TA'd and taken this class, I can say that it (and its concluding material) are very warmly received by the students.




You can cover boosting. It's very clever, easy to implement, is widely used in practice, and doesn't require much prerequisite knowledge to understand.

I've presented some parts of the survey by Arora et al. ( in the grad theory class a few years back. People seemed to like it and I think you need almost no background to understand this material.


If you just want to whet their appetite in a single lecture, it might be most exciting to present a powerful application. For example, support vector machines, and other machine learning algorithms, are used in chemoinformatics for drug discovery.

The learning problem essentially is: given a behavior we want a chemical to exhibit, devise a structure that exhibits that behavior by deducing it from a database of known structures that exhibit similar (or dissimilar) behaviors. The learning problem has an extra wrinkle: the new drug needs to be "distant" in global structure from previously known drugs, in order to found a patent estate.

One source is Clustering Methods and Their Uses in Computational Chemistry .

Thanks for the reference. I was thinking of perhaps teaching SVMs as an application of convex optimisation. That would relate the algorithms part with the ML part nicely.

how do you cover SVMs without linear algebra?
Lev Reyzin

I was hoping to teach them the minimum prerequisites for it in my course. Maybe that was too optimistic :-)

Are there still important examples where support vector machines are the best choice? I notice that on kaggle competitions, for example, they are never the main part of a winning entry. At least not any that I have seen recently. (I stand to be corrected of course.)


K-Means and KNN are very powerful and do not require any Linear Algebra except the computation of distances of points.

K-Means in particular is a very potent algorithm. It is incredibly effective despite not having proven bounds on its objective function performance, to such a spooky extent that it's almost like the Simplex algorithm's effective polynomial complexity (despite real exponential complexity). Its online version is also useful in large-scale data applications.
Elliot JJ


The second part of "Neural Networks and Machine Learning" by Christopher Bishop (at MSR) is on algorithms in ML. Bishop's textbooks are commonly used for graduate (and later undergraduate) textbooks and are extremely well-written.


This algorithm uses graph minimum cuts to classify large amount of unlabeled samples using only small amount of labelled samples.

Its undergrad friendly. I have explained this to a few randomly chosen undergrads and they understood it.

Ref: Blum, A., & Chawla, S. (2001). Learning from labeled and unlabeled data using graph mincuts.

Self promotion Visualization of the algorithm on youtube.


You can cover some algorithms which are classic or with good intuition.

For example, C4.5 and CART, which are classic decision tree algorithms.

You also can cover some ensemble methods(e.g., AdaBoost(Boosting), Bagging), which have a very good performance in real world applications.

Furthermore, deep learning is also a good topic, because it is very hot.


Native bayes and Bayesian network, decision tree algorithms are quite easy to visualize than starting with a neutral network or svm


Genetic programming is really cool. It uses inspiration from biology, and can be applied to a vast number of problems (for example, n-queens problem, and TSP).

It does not require deep mathematical skills.

EDIT: It only requires a way to estimate how good a potential solution is. It can be used for example to guess the rule behind a series of numbers, finding minima/maxima to multi-variate problems, and search huge parameter spaces. It is suited when you are not interested in the optimal solution, but when a good enough solution will do. I believe this has been used to find good strategies for games (build orders in Starcraft 2, and optimal play in Mario).

Is there any important problem for which it is the best method? I mean it certainly isn't for TSP for example.
Durch die Nutzung unserer Website bestätigen Sie, dass Sie unsere Cookie-Richtlinie und Datenschutzrichtlinie gelesen und verstanden haben.
Licensed under cc by-sa 3.0 with attribution required.