Rodrigo Pérez (bilson)

HABTM search by multiple tags in CakePHP

bilson — Fri, 03 Dec 2010 18:47:50 +0000

The Problem

A client came to me with an interesting request. They wanted the ability to search a songs database based on tags that defined that song. A similar problem would be faced with a real estate portal trying to search properties that have specific amenities, or posts with certain tags, etc.. etc…

The concept is fairly simple, but one that can be a bit tricky to do with a HABTM relationship in CakePHP. This isn’t just a tricky problem in CakePHP, its a tricky problem on a database level.

Now searching for all songs based on a single tag is easy enough — set the join, set the condition, everyone eat cake. However, searching based on multiple tags is a little more involved.

The setup and scope

Consider the following song list and their related tags (ie … SongName (Tag1,Tag2,Tag3….)):

* Song1 (Drum Intro, Guitar Solo, No Vocal)
* Song2 (Drum Intro)
* Song3 (Guitar Solo, No Vocal)
* Song4 (Drum Intro, No Vocal)

Now, I want an easy way to get all songs based on an array of tags (ie … Search(Tag1,Tag2….) = Result Set):

* Search(Drum Intro, No Vocal) = Song1 and Song4
* Search(Guitar Solo) = Song1, Song3
* Search(Drum Intro, Guitar Solo) = no results
* Search(Drum Intro, Guitar Solo, No Vocal) = Song1

Google is our friend

While approaching this problem I stumbled across a great post by teknoid http://teknoid.wordpress.com/2008/08/06/habtm-and-join-trickery-with-cakephp/ and although it was close to what I wanted, it had a very large limitation — Teknoid’s solution would only return a result set of songs if the tags are an exact match — no more, no less. I needed something more flexible. I needed a way to search for a set of songs that had at least the specific set of tags, but could possibly have more.

The Solution

So, with the help of the CakePHP’s custom find methods I hit was off to work!

      //Song Model
      var $hasAndBelongsToMany = array('Tag');
 
      var $_findMethods = array('similar' => true);
 
      function _findSimilar($state, $query, $results = array()){
        if($state == 'before' && isset($query['tags'])){
          $query['joins'] = array(
            array(
              'table' => 'songs_tags',
              'alias' => 'SongsTag',
              'type' => 'inner',
              'conditions' => array('SongsTag.song_id = Song.id')
            ),
            array(
              'table' => 'tags',
              'alias' => 'Tag',
              'type' => 'inner',
              'conditions' => array(
                'Tag.id = SongsTag.tag_id'
              )
            )
          );
          if(!is_array($query['tags'])){
            $query['tags'] = array($query['tags']); //make it an array.
          }
 
          $query['conditions']['Tag.name'] = $query['tags'];
 
          //set limit
          $query['limit'] = isset($query['limit']) ? $query['limit'] : 5;
 
          return $query;
        }
        elseif($state == 'after' && isset($query['tags'])){
          $retval = array();
          foreach($results as $key => $result){
            $tags = Set::extract('/Tag/name', $result);
            if(count($query['tags']) == count(array_intersect($tags, $query['tags']))){
              $retval[] = $results[$key];
            }
          }
          $results = $retval;
        }
        return $results;
      }

Breakdown … prepare query

We’re actually doing a few things here. There are two states to a custom find — before and after. Lets break it down to bite size pieces.

      function _findSimilar($state, $query, $results = array()){
          if($state == 'before' && isset($query['tags'])){

In the before state we prepare the incoming query. We only want to manipulate the query if we have a ‘tags’ key in our options array, otherwise just pass it onto find(‘all’).

      $query['joins'] = array(
        array(
          'table' => 'songs_tags',
          'alias' => 'SongsTag',
          'type' => 'inner',
          'conditions' => array('SongsTag.song_id = Song.id')
        ),
        array(
          'table' => 'tags',
          'alias' => 'Tag',
          'type' => 'inner',
          'conditions' => array(
            'Tag.id = SongsTag.tag_id'
          )
        )
      );
      if(!is_array($query['tags'])){
        $query['tags'] = array($query['tags']);
      }

We have to manually set our joins so we can use our friendly CakePHP Tag.name condition syntax. We also make sure we’re dealing with an array of tags (this allows us to pass in a single string if wanted).

      $query['conditions']['Tag.name'] = $query['tags'];
      //set limit
      $query['limit'] = isset($query['limit']) ? $query['limit'] : 5;
 
      return $query;

We set the conditions with an IN operator, set the limit if we don’t have one (5 in this case) and return the modified query. This query is rather greedy, as such, its best to limit it to maintain performance.

The end result in the before state is we’re running a search on songs that have ANY of the tags we’ve included in our tags array. Later we’ll take a look at the results and make sure we only return results that include all or more of the tags we specified.

Breakdown … filter results

The second part of the custom find is the after state.

      elseif($state == 'after' && isset($query['tags'])){

So here again we make sure we’re preforming a tag search, by checking if the tags key is set.

          $retval = array();
          foreach($results as $key => $result){
            $tags = Set::extract('/Tag/name', $result);
            if(count($query['tags']) == count(array_intersect($tags, $query['tags']))){
              $retval[] = $results[$key];
            }
          }
        $results = $retval;
      }

It gets a little tricky here, we go through the result set that the query returned to us and extract all the names using CakePHP’s built in Set class. $tags ends up being a one dimensional array just like our $options['tags']. We then count the $options['tags'] and make sure its equal to the array_intersect of $tags, and $options['tags']. Array_intersect is great here because it will take two arrays, compare then and give us back what is matching. The result is only songs that have at least the specified search tags (possibly more) will be given as a valid result in the returned set.

Example Usage

Example usage of this custom similar find would be:

      $result = $this->Song->find('similar', array(
        'tags' => array('Drum Intro', 'No Vocal')
      )); //Song1 and Song4
 
      $result = $this->Song->find('similar', array(
        'tags' => 'Guitar Solo'
      )); //Song1 and Song3
 
      $result = $this->Song->find('similar', array(
        'tags' => array('Drum Intro', 'Guitar Solo')
      )); //Empty

Extra credit

A tip to paginate these results is to overwrite _findAll and _findCount, checking for ‘tags’, and doing the same thing. By doing so, all you’ll have to do is pass in a ‘tags’ option key on an ‘all’ or a ‘count’ to get the behavior you want. This end result is all you need for pagination to work.

I hope you’ve enjoyed this tutorial.

Artículo original WebTechNick

Modelizing HABTM join tables in CakePHP 1.2: with and auto-with models

bilson — Tue, 30 Nov 2010 17:44:56 +0000

One of CakePHP 1.2 coolest features are known as HABTM with models, which are particularly useful when you have a hasAndBelongsToMany binding between two models that may contain extra information (i.e: table fields) attached to the binding. They are also extremely useful when you need to make some model operations to the join table and feel too lazy to modelize it yourself, or when you have modelized the join table and need to tell CakePHP to use your own model as the join model.

Let’s start with a practical example, the classic Article -> hasAndBelongsToMany -> Tag binding. Set up the articles table in any way you like, and so the tags table. Now create the join table as follows:

CREATE TABLE `articles_tags`(
	`id` INT NOT NULL AUTO_INCREMENT,
	`article_id` INT NOT NULL,
	`tag_id` INT NOT NULL,
	PRIMARY KEY(`id`)
);

Yeah, I know, so much for extra fields: just the id. However feel free to add any extra fields yourself and the example here will still be valid. In fact, if there are more than two fields in the join table then CakePHP will assume that you are normally interested on those extra fields.

Assume you have set up some articles, some tags, and have joined articles with some tags. Let’s do a classic findAll on the Article model:

$records = $this->Article->findAll();
debug($records);

Bearing in mind the difference between the data you’ve set up, and the one we have on this example, the result may be something like this:

Ok now you may be asking: what is up with that ArticlesTag array that is showing up there? Well, you may have also guessed it: that’s the information coming from the HABTM join table. Let’s take a closer look at the SQL statements generated by CakePHP that yielded these results:

SELECT `Article`.`id`, `Article`.`title`, `Article`.`body`, `Article`.`created`, `Article`.`modified` FROM `articles` AS `Article` WHERE 1 = 1;
 
SELECT `Tag`.`id`, `Tag`.`name`, `Tag`.`created`, `Tag`.`modified`, `ArticlesTag`.`id`, `ArticlesTag`.`article_id`, `ArticlesTag`.`tag_id` FROM `tags` AS `Tag` JOIN `articles_tags` AS `ArticlesTag` ON (`ArticlesTag`.`article_id` IN (1, 5) AND `ArticlesTag`.`tag_id` = `Tag`.`id`)

The first query does the normal Article search, this is where your findAll conditions would be inserted (so don’t think yet that you can attach conditions based on your join model). The next query finds all the tags linked through the join table to the set of Article IDs obtained during the first query. Yes, CakePHP 1.1 users: this query has also been optimized (CakePHP 1.1 runs one finder query per resulting row).

What if you want to only get those articles that are linked to a specific tag? You need then to query the join table. This is where auto-with tables can also help you out, since the join table has been auto-modelized for you (hence the auto part of the name). Try this:

$joinRecords = $this->Article->ArticlesTag->findAll(array('ArticlesTag.tag_id' => 3));
debug($joinRecords);

We now get:

Array
(
    [0] => Array
        (
            [ArticlesTag] => Array
                (
                    [id] => 2
                    [article_id] => 1
                    [tag_id] => 3
                )
        )
    [1] => Array
        (
            [ArticlesTag] => Array
                (
                    [id] => 4
                    [article_id] => 5
                    [tag_id] => 3
                )
        )
)

So you can either play around and get the Article IDs using Set extract:

$ids = Set::extract($joinRecords, '{n}.ArticlesTag.article_id');
debug($ids);

After which $ids look like:

Array
(
    [0] => 1
    [1] => 5
)

Or use bindModel to join the Article and Tag models to the join table, and then running the query:

$this->Article->ArticlesTag->bindModel(array('belongsTo' => array('Article', 'Tag')));
$joinRecords = $this->Article->ArticlesTag->findAll(array('ArticlesTag.tag_id' => 3));
debug($joinRecords);

The records now look like:

Array
(
    [0] => Array
        (
            [ArticlesTag] => Array
                (
                    [id] => 2
                    [article_id] => 1
                    [tag_id] => 3
                )
            [Article] => Array
                (
                    [id] => 1
                    [title] => My First Article
                    [body] => Hi there!
                    [created] => 2007-03-19 15:01:16
                    [modified] => 2007-03-19 15:31:06
                )
            [Tag] => Array
                (
                    [id] => 3
                    [name] => tag3
                    [created] => 2007-09-19 03:04:33
                    [modified] => 2007-09-19 03:04:33
                )
        )
    [1] => Array
        (
            [ArticlesTag] => Array
                (
                    [id] => 4
                    [article_id] => 5
                    [tag_id] => 3
                )
            [Article] => Array
                (
                    [id] => 5
                    [title] => My Second Article
                    [body] => Hi there, new article!
                    [created] => 2007-03-19 15:06:02
                    [modified] => 2007-03-19 15:06:02
                )
            [Tag] => Array
                (
                    [id] => 3
                    [name] => tag3
                    [created] => 2007-09-19 03:04:33
                    [modified] => 2007-09-19 03:04:33
                )
        )
)

And if you need more information associated with each Article you can of course play with the $recursive level of the findAll query.

What if for some reason you have already modelized your join table? Let’s say you created a model called ArticlesTag (located on app/models/articles_tag.php) that looks like this:

class ArticlesTag extends AppModel {
	var $name = 'ArticlesTag';
	var $belongsTo = array('Article', 'Tag');
}

How can we now tell CakePHP that the join table should be modelized using our model? Let’s go back to the Article model definition and make a small change where we define the with model:

class Article extends AppModel {
	var $name = 'Article';
	var $hasAndBelongsToMany = array('Tag' => array('with' => 'ArticlesTag'));
}

Remember how we had to bind the Article and Tag model to the auto-with model set up with CakePHP? Looking at our own version of the join model that shouldn’t be necessary anymore, so let’s try:

$joinRecords = $this->Article->ArticlesTag->findAll(array('ArticlesTag.tag_id' => 3));
debug($joinRecords);

You can now see that the yielded results are the same as those obtained when the auto-with model was previously binded.

Anyway, that’s a short introduction to CakePHP 1.2 with models. As usual with CakePHP, what seems to be a small addition gives us a new set of exciting possibilities.

Artículo original de Mariano Iglesias

sudo: sorry, you must have a tty to run sudo

bilson — Sat, 09 Oct 2010 05:05:51 +0000

Tengo un script corriendo vía cron que requiere un comando privilegiado para funcionar. Para permitir el comando, utilizo sudo, agregando el comando y usuario en el archivo /etc/sudoers.

Cuando el script se ejecuta, devuelve un error:

sudo: sorry, you must have a tty to run sudo

Solución: Comentar la línea Default requiretty en /etc/sudoers

#Default requiretty

Contar IP y cantidad de accesos en logs de apache

bilson — Tue, 17 Aug 2010 20:24:46 +0000

No recuerdo de dónde lo saqué pero acá va:

awk '{!a[$1]++}END{for(i in a) if ( a[i] >10 ) print a[i],i }' ./access-log

Se demoró una fracción de segundo en procesar un log de 500.000 líneas!

Instalar Gnome y Xorg en Centos con yum

bilson — Fri, 13 Aug 2010 00:51:58 +0000

Si instalaste un centos sin entorno gráfico y ahora lo necesitas, la solución es:

yum groupinstall "X Window System" "GNOME Desktop Environment"

Yum es sensible a mayúsculas y minúsculas!

No hay lugar como localhost

bilson — Sat, 10 Jul 2010 13:00:47 +0000

Túnel SSH

bilson — Fri, 09 Jul 2010 23:17:49 +0000

(Reviviendo post añejos)

La situación es la siguiente:

Estoy en mi casa, necesito conectarme a la máquina 1.2.3.4, por distintos protocolos (FTP, HTTP, SSH, SFTP), pero la máquina 1.2.3.4 tiene acceso restringido por IP, y sólo la IP 1.2.3.5 tiene acceso. Pero yo puedo entrar por ssh a 1.2.3.5.

Solución: Hacer un túnel SSH.

Cómo ?

Fácil:

ssh -L 10022:1.2.3.4:22 -f -N usuario@1.2.3.5

De esta forma, yo hago ssh a localhost en el puerto 10022 y la conexión es redireccionada a 1.2.3.4, y ya estoy por ssh en la máquina que tenía restringido el acceso

Repetir el proceso para redireccionar más puertos.

Ejemplos:

ssh -L 1025:1.2.3.4:25 -f -N usuario@1.2.3.5
ssh -L 1080:1.2.3.4:80 -f -N usuario@1.2.3.5
ssh -L 10021:1.2.3.4:21 -f -N usuario@1.2.3.5
ssh -L 10110:1.2.3.4:110 -f -N usuario@1.2.3.5

Para puertos menores a 1025 se debe tener privilegios de root.

mod_rewrite Cheat Sheet

bilson — Mon, 28 Dec 2009 19:53:32 +0000

Navegando por ahí… encontré este buen resumen de lo que podemos hacer con mod_rewrite… para tener siempre a mano:

http://www.cheat-sheets.org/saved-copy/mod_rewrite_cheat_sheet.pdf

Configurar Sony Ericsson MD300 de EntelPCS en Ubuntu Intrepid Ibex (Conexión 3g)

bilson — Sun, 01 Feb 2009 20:22:33 +0000

La información que aquí publico está tomada básicamente de tres partes (ubuntu-cl.org, laudecioliveira.org y ubuntu-es.org). Ha sido probado con EntelPCS en Chile, y funciona sin problemas

Lo primero es ayudar a udev a identificar el módem como tal, para eso necesitamos el id del dispositivo y de vendedor. Esta información la obtenemos así:

sudo lsusb -v|grep -i vendor
 
  idVendor           0x0fce Sony Ericsson Mobile Communications AB

sudo lsusb -v|grep -i product
 
  idProduct          0xd0cf 
  iProduct                2 Sony Ericsson MD300

Ahora creamos el archivo /etc/udev/rules.d/50-md300modem.rules con nuestro editor favorito:

ACTION!="add", GOTO="3G_End"
BUS=="usb", SYSFS{idProduct}=="d0cf", SYSFS{idVendor}=="0fce", PROGRAM="/bin/sh -c 'echo 3 > /sys/%p/device/bConfigurationValue'"
LABEL="3G_END"

Ahora desconectamos el módem y descargamos los módulos del kernel, lo volvemos a cargar con los parámetros que necesitamos

sudo modprobe -r usbserial
sudo rmmod usb_storage
sudo modprobe usbserial vendor=0x0fce product=0xd0cf

Creamos el archivo /etc/modprobe.d/md300 con nuestro editor de texto favorito. Este archivo tendrá un alias para que el módulo se cargue correctamente luego de reiniciar el pc.

#Sony Ericsson MD300
alias md300 usbserial
options md300 vendor=0x0fce product=0xd0cf

Reiniciamos udev:

sudo /etc/init.d/udev restart

Ahora debemos configurar wvdial, editamos el archivo /etc/wvdial.conf

[Dialer Defaults]
Modem = /dev/ttyACM0
ISDN = off
Modem Type = USB Modem
Baud = 460800
Init = ATZ
Init2 = AT+CFUN=1
Init3 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Init4 = 
Init5 = ATZ+cgdcont=1,"IP","imovil.entelpcs.cl"
Init6 = 
Init7 = 
Init8 = 
Init9 = 
Phone = *99#
Phone1 = 
Phone2 = 
Phone3 = 
Phone4 = 
Dial Prefix = 
Dial Attempts = 1
Dial Command = ATM1L3DT
Ask Password = off
Password = entelpcs
Username = entelpcs
Auto Reconnect = off
Abort on Busy = off
Carrier Check = off
Check Def Route = on
Abort on No Dialtone = on
Stupid Mode = on
Idle Seconds = 0
Auto DNS = on

La línea Init2 = AT+CFUN=1 es para activar la función radio del módem. Sin esta línea, pppd se cierra antes de conectar.

Enchufamos nuevamente el módem, y ejecutamos wvdial para conectarnos a internet:

sudo wvdial

Obtendremos como resultado algo como esto:

--> WvDial: Internet dialer version 1.60
--> Cannot get information for serial port.
--> Initializing modem.
--> Sending: ATZ
ATZ
OK
--> Sending: AT+CFUN=1
AT+CFUN=1
OK
--> Sending: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
OK
--> Sending: ATZ+cgdcont=1,"IP","imovil.entelpcs.cl"
ATZ+cgdcont=1,"IP","imovil.entelpcs.cl"
OK
--> Modem initialized.
--> Sending: ATM1L3DT*99#
--> Waiting for carrier.
ATM1L3DT*99#
~[7f]}#@!}!}!} }9}#}%B#}%}(}"}'}"}"}&} } } } }%}&=;+zZC~
CONNECT
--> Carrier detected.  Starting PPP immediately.
--> Starting pppd at Sun Feb  1 20:21:26 2009
--> Pid of pppd: 8630
--> Using interface ppp0
--> local  IP address 186.9.56.27
--> remote IP address 10.64.64.64
--> primary   DNS address 164.77.252.249
--> secondary DNS address 164.77.252.215

Actualización: Si tienen distribuciones basadas en RPM, pueden bajar este “driver” de betavine (desarrollado por EntelPCS), que básicamente es lo mismo que hice yo a mano, y con algunas pequeñas modificaciones básicamente en el archivo wvdial.conf. Desde ubuntu pueden pasarlo a .deb con alien.

Si prefieren, también pueden usar gnome-ppp para discar. Eso es todo… disfruten

Get VOIP

bilson — Thu, 09 Oct 2008 08:28:35 +0000

Get VOIP

Originally uploaded by mringlein

Notable !!

No será mucho digo yo ?