Visualizzazione post con etichetta devel. Mostra tutti i post
Visualizzazione post con etichetta devel. Mostra tutti i post

sabato 23 luglio 2011

Ignore a word in a regular expression

Today a friend of mine asked me a tricky regexp related question: he wanted to match against a set of strings like:

WORD_FOO
WORD_BAR
WORD_FOOBAR
WORD_QUUX
WORD_FAR

He did want to match and store any string starting with WORD_ and followed by a valid word, unless it was FOO. So it should match all the aforementioned lines but WORD_FOO.

Tricky.

He was using something like /^(WORD_[^\s]+)$/, so I suggested using [^\s|VERSION] (I did wake up only a few moments before), but of course that doesn't work, since it would exclude all strings containing the characters V, E, R, S, I, O, N.

It took me some digging, but finally I found this answer on StackOverflow that documents the use of negative look-arounds.

Using these constructs I managed to get this regex: WORD_((?!FOO\W)\S+) that satisfies the requirements (you can check it on Rubular).

How does it work?


(?!FOO\W) checks the next characters of the string. If they DON'T (!) contain the word FOO followed by a non-word (whitespace, etc) character (\W), then the matching will be made against \S+ (one or more non whitespace characters). So you'll get the second part of the word in your \1, $1, etc.

If you want to ignore all sub-strings starting with FOO, you can get rid of that \W.

martedì 11 gennaio 2011

Error compiling ATI Streams SDK on Fedora

I'm working with some OpenCL related things, so I need to install ATI Streams SDK on this machine.

I downloaded the software from ATI Streams website, then extracted it and exported the necessary env vars:

$ tar zxvf /path/to/ati-stream-sdk-v2.3-lnx64.tgz
$ cd ati-stream-*/
$ export ATISTREAMSDKROOT=$(PWD)
$ export ATISTREAMSDKSAMPLESROOT=${ATISTREAMSDKROOT}/samples

At this point I had to patch several makefiles to compile the SDK on my Fedora 13, so here's the patch. Save it as ati-streams.patch, then make sure you are in the ati-streams-sdk folder and launch:

$ patch -p1 < /path/to/ati-streams.patch
patching file samples/opencl/cl/app/BoxFilterGL/Makefile
patching file samples/opencl/cl/app/FluidSimulation2D/Makefile
patching file samples/opencl/cl/app/Mandelbrot/Makefile
patching file samples/opencl/cl/app/NBody/Makefile
patching file samples/opencl/cl/app/SimpleGL/Makefile
patching file samples/opencl/cl/app/Template/Makefile
patching file samples/opencl/cl/app/URNGNoiseGL/Makefile

Then you can launch make to compile the SDK.

mercoledì 29 dicembre 2010

Auto-learning playlist generator

Auto-rating algorithms for music players are quite primitive: if you listen to the song, the song gains a +1 rating, if you skip it, the rating stands still.

That's a quite basic method to give meaning to your choice to skip that particular song.

But... Maybe the song wasn't fitting the playlist mood or your mood in that moment.

What about those lovely 14-minutes-long songs? Skipping after a few seconds or after 10 minutes (maybe during that boring drums solo) should have different meanings.

What about these times when rand() doesn't love you and you skip 20 songs in a row? The player should be clever enough to present you with one of your favourites after a few clicks, so you can stop pressing Next and get back to listening.

How to deal with afk users? It happens: people get away from their computer and doesn't bother stopping their music player. Or maybe they are listening, but too lazy to get to computer to switch song. A "full play" should not reward much and a fast skip (skipping during the first 10-15 seconds) should penalize the song more than a full play.

Base play

Let's see one example: all songs starts with an internal rating of 10.

(A..Z) = 10
Base penalty

Our user starts playing from song A, but he don't like it now (why? back to that in a few moments). So he press Next after a few seconds, song A gets penalized.

A -= 0.5

Meta-data penalty

What about Artist, Album, Genre and Publishing Year? All those should be negatively affected, maybe toning down the rating a bit:

A.artist -= 0.2
A.album -= 0.2
A.genre -= 0.2
A.year -= 0.2

Timestamping rating associations

That's quite unfair: maybe song A doesn't fit with our current mood. Maybe we like that song (or artist, album, genre, year) in a different moment, so we should create an association between the timestamp and the bad rating, so we'll better write it as:

hour-of-the-day[A] -= 0.5
day-of-the-year[A] -= 0.5

It's the same for artist, album, etc:

hour-of-the-day[A.artist] -= 0.2
day-of-the-year[A.artist] -= 0.2
# [...]
hour-of-the-day[A.year] -= 0.2
day-of-the-year[A.year] -= 0.2

In the mood

Using these choices we could try to "learn" about the user mood with time. Every time the player starts it creates a new "mood profile" and eventually switches to an already existant mood profile if the user acts according to it.

So we can rewrite that as:

mood-profile01[ hour-of-the-day[A] ] -= 0.5
mood-profile01[ day-of-the-year[A] ] -= 0.5
# [...]
mood-profile01[ hour-of-the-day[A.year] ] -= 0.2
mood-profile01[ day-of-the-year[A.year] ] -= 0.2

Maybe we should consider merging similar profiles after a lot of learning.

Dealing with ossessive-compulsive skipping

Back to listening. Our user Next action has brought up another song chosen by his fellow rand() function, the song G. But it seems he doesn't like that also at the moment, so he press Next again. We should penalize song G, too, but maybe the user is only a bit lazy and we tone down the penalty, since it's the second fast-skip in a row:

mood-profile01[ hour-of-the-day[G] ] -= 0.4
mood-profile01[ day-of-the-year[G] ] -= 0.4
# [...]
mood-profile01[ hour-of-the-day[G.year] ] -= 0.1
mood-profile01[ day-of-the-year[G.year] ] -= 0.1

Our rand() function brings the user to listening to song M. He dislikes that too, so we tone down the penalty again:

mood-profile01[ hour-of-the-day[G] ] -= 0.25
mood-profile01[ day-of-the-year[G] ] -= 0.25
# [...]
mood-profile01[ hour-of-the-day[G.year] ] -= 0.03
mood-profile01[ day-of-the-year[G.year] ] -= 0.03

And so on.

Context matters

Now, what about the context? Our fellow user listen flawlessly to the 4 next songs: song V, F, I, P, each of which gains a little award:

mood-profile01[ hour-of-the-day[V] ] += 0.1
mood-profile01[ day-of-the-year[V] ] += 0.1
# [...]
mood-profile01[ hour-of-the-day[V.year] ] -= 0.01
mood-profile01[ day-of-the-year[V.year] ] -= 0.01

Then the user stumbles on song X. Maybe that song doesn't fit the current playlist (with time that should never occur), so we penalize it:

mood-profile01[ hour-of-the-day[V] ] -= 0.5
mood-profile01[ day-of-the-year[V] ] -= 0.5
# [...]
mood-profile01[ hour-of-the-day[V.year] ] -= 0.2
mood-profile01[ day-of-the-year[V.year] ] -= 0.2

And then we penalize the playlist association (maybe we can use less songs, like the last 3 or so):

mood-profile01[ playlist(V, F, I, P, X) ] -= 0.5
mood-profile01[ playlist(V, F, I, P, X) ] -= 0.5

Dealing with urgency

If you're skipping a song in the first few (10-15) seconds of playing, we're giving a 0.5 points penalty to that song (in that particular moment). But what about a skip that occurs when you've listened up to 95% of the song?

Penalty should scale so it's 0.5 for a skip during the first seconds and then tone down to 0.4 for the first quarter of the song, 0.25 for the first half and 0.1 if the skip occurs past half song.

Getting tags into consideration

At the moment I'm not aware of players supporting tags for songs, but that would be a great feature (maybe checking the associations on last.fm or any other similar site would be a plus). Of course with such a feature in place, profiling would be even easier. We could take into account also the tag list, like:

mood-profile01[ hour-of-the-day[V.tags] ] -= 0.2
mood-profile01[ day-of-the-year[V.tags] ] -= 0.2

lunedì 20 dicembre 2010

Albero di Natale in 1k di Javascript

Un demo abbastanza impressionante: un albero di Natale in 3D in 1k di Javascript.

Visto qui.

lunedì 6 dicembre 2010

Creare un repo git condiviso

Sulla mia box attuale ho due dischi in raid 1 su cui si trova la /home. Per sviluppare ho una vm QEMU/KVM con sopra Ubuntu 10.04 Server, il cui disco però non risiede sul suddetto raid.

Dato che voglio avere una certa ridondanza per i dati, volevo fare in modo di avere un repository git centralizzato sulla macchina host (nella home, quindi sul raid) da cui clonare nella vm (ed eventualmente sul portatile) per sviluppare e poi pushare di nuovo sul repo sul raid.

Inizialmente ho semplicemente creato un repository sulla macchina host, l'ho clonato sulla vm, ma al momento di effettuare il push usciva questo simpatico errore:

vm ➤ git push
Counting objects: 36, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (23/23), done.
Writing objects: 100% (25/25), 84.45 KiB, done.
Total 25 (delta 6), reused 0 (delta 0)
remote: error: refusing to update checked out branch: refs/heads/master
remote: error: By default, updating the current branch in a non-bare repository
remote: error: is denied, because it will make the index and work tree inconsistent
remote: error: with what you pushed, and will require 'git reset --hard' to match
remote: error: the work tree to HEAD.
remote: error: 
remote: error: You can set 'receive.denyCurrentBranch' configuration variable to
remote: error: 'ignore' or 'warn' in the remote repository to allow pushing into
remote: error: its current branch; however, this is not recommended unless you
remote: error: arranged to update its work tree to match what you pushed in some
remote: error: other way.
remote: error: 
remote: error: To squelch this message and still keep the default behaviour, set
remote: error: 'receive.denyCurrentBranch' configuration variable to 'refuse'.
To host:c0ding/ruby/project
! [remote rejected] master -> master (branch is currently checked out)
error: failed to push some refs to 'host:c0ding/ruby/project'

Il problema era che essendo loggato anche sulla macchina host ed avendo un repository "aperto", git si rifiutava di farmi mischiare le due cose. Inizialmente, per risolvere, mi sono limitato a creare un nuovo branch sulla macchina host:

host$ git checkout -b temp-repo

La soluzione più corretta, però, l'ho trovata su stack overflow (e sul sito linkato, pro git): era necessario creare un "bare repository", ovvero un repository che contiene solo i dati GIT e nessun altro file.

Quindi ho fatto:

host$ rm -rf ~/c0ding/ruby/project

vm ➤ git clone --bare project project.git  # creazione bare repo
vm ➤ scp -r project.git host:c0ding/ruby/  # copia del repo sull'host
vm ➤ rm -rf project project.git            # rimozione vecchi repo sulla vm
vm ➤ git clone host:c0ding/ruby/project.git # clone nuovo repo sulla vm
vm ➤ cd project
vm ➤ git log

commit xxx
Author: Gilberto "Velenux" Ficara
Date:   Mon Nov 1 15:40:20 2010 +0100

Added Cucumber, JQuery support gems. Created DB.

commit xxx
Author: Gilberto "Velenux" Ficara
Date:   Mon Nov 1 14:42:03 2010 +0100

Initial commit

vm ➤