Darrin's Tech Ramblings

… go fly a kite

  • Home
  • About

Zend Form — validate an array of data

Posted by kitepwr on July 22, 2015
Posted in: Zend. Leave a comment

Updated Zend SubForm usage

Darrin's Tech Ramblings

Using Zend Framework v 1.12, we can validate an array of data using a subForm.
In this example, we want to validate up to 10 transaction_ids passed in an array.
The validation for each will used Zend_Validate_Db_RecordExists to make sure the transaction_id exists in the database and is for the currently logged in seller_id.

View original post

Zend Form — validate an array of data

Posted by kitepwr on May 29, 2015
Posted in: PHP/Zend, Zend. Tagged: Form, PHP, Zend. 1 Comment

Using Zend Framework v 1.12, we can validate an array of data using a subForm.
In this example, we want to validate up to 10 transaction_ids passed in an array.
The validation for each will used Zend_Validate_Db_RecordExists to make sure the transaction_id exists in the database and is for the currently logged in seller_id.

    $form = new Zend_Form();

    // add validations for the 'simple', non Array elements
    $form->addElement( ................. ); 
    $form->addElement( ................. ); 
    $form->addElement( ................. ); 

    // add validations for the 'transaction_id' input field
    // Input: transaction_id[0] = 'value'
    //        transaction_id[1] = 'value'
    //               ...
    //        transaction_id[9] = 'value'
    $this->createTransactionIdSubForm($form, 10);
 
    /**
     * createTransactionIdSubForm creates a form element that handles an array of
     * transaction_ids.  Each will be validated to exist in the database and be
     * valid for our current seller_id.
     *
     * @param  Zend_Form $form
     * @param  integer $nbr_of_ids defines the number of subForm element to build
     */
    public function createTransactionIdSubForm($form, $nbr_of_ids)
    {
        $subForm = new Zend_Form_SubForm();

        // Create a validator for each array element of the transaction_id array
        for($i=0; $i < $nbr_of_ids; $i++) {
            $subForm->addElement('text', (string) $i, array(
                'required'   => false,
                'filters'    => array('Digits'),
                'validators' => array(
                    array('Db_RecordExists', true, array(
                        'transaction',
                        'transaction_id',
                        '(select seller_id 
                          from activity 
                          where activity_id = transaction.activity_id
                         ) = '.$this->getSellerId()
                        ))
            )));
        }
        $form->addSubForm($subForm, 'transaction_id');
    }

Add color to your ssh sessions – Mac OSX

Posted by kitepwr on November 7, 2014
Posted in: Zend. Leave a comment

NOTE: This solution only works on a Mac!  It requires AppleScript and the Terminal application.

I am constantly ssh’ing into different machines and then forgetting which window is which machine.  
My solution is to use different color settings for different environments.  For example, when I ssh into a production machine I want the Mac Terminal Profile to change to “Red Sands”.  And, when I close ssh I want it to revert back to the Profile that was in use originally.  This is not a new problem and there are 100’s of similar looking solutions out there.  However all the solutions I could find suffer from the same flaw — when a ssh session running in a background Window+Tab times out or gets disconnected, that window is not ‘reset’. In most cases the currently active window is reset, which is not at all what we want.

There are 2 files.  Put both in /usr/local/bin and do ‘chmod +x’ on them.  The code assumes that your current ssh program is in /usr/bin/ssh.  If it’s somewhere else, change the 2nd to last line in the ssh file.

In the following file, modify the case statement to include any servers you have. For each server provide a Terminal Profile Name to use.

File: /usr/local/bin/ssh

#!/bin/sh
## 
## Author: Darrin Skinner -- skinner@skinner.com
##
## This changes the terminal color depending on the ssh target environment
## and it changes it back to the original color when ssh exits
##
## In the case statement below, provide a search pattern 
## and a Terminal Profile Name to use for each server.
##

HOSTNAME=$@

case $HOSTNAME  in
    dev.*)      COLOR="Homebrew" ;; ## All development servers
    qa.*)       COLOR="Man Page" ;; ## All QA servers
    train*)     COLOR="Novel"    ;; ## Training server
    prod.*)     COLOR="Red Sands";; ## All Production servers
    *)          COLOR="Pro"      ;; ## Everything else
esac

ORIG_COLOR=`set_term_color "$COLOR"`
/usr/bin/ssh $@
ORIG_COLOR=`set_term_color "$ORIG_COLOR"`

File: /usr/local/bin/set_term_color

#!/bin/bash
## 
## Author: Darrin Skinner -- skinner@skinner.com
##
## This script will change the Mac Terminal setting (i.e. the 
## Terminal colors) to the requisted Profile.
## The old/original Profile name is returned.
##
## It will search through all the Terminal windows+tabs until it 
## finds the one the script is running from and execute the 
## new settings in that window+tab.
##
## Usage:
##     set_term_color Novel
##     set_term_color "Red Sands"
##
## The argument to pass is:
##     The Terminal "color theme" Profile desired 
##         -> Ocean, Novel, Homebrew, etc
##

mySettingName=$1
myTTY=`tty`

osascript <<EOD
    set mySettingName to "$mySettingName"
    set myTTY         to "$myTTY"
    tell application "Terminal"
        set allWindows to number of windows
        repeat with thisWindow from 1 to allWindows
            set allTabs to number of tabs of window thisWindow
            repeat with thisTab from 1 to allTabs
                if tty of tab thisTab of window thisWindow is myTTY then
                    set thisSettings to name of current settings of tab thisTab of window thisWindow
                    set current settings of tab thisTab of window thisWindow to settings set mySettingName
                    return thisSettings
                end if
            end repeat
        end repeat
    end tell
EOD

Using Zend Cache to save the results of a function

Posted by kitepwr on May 6, 2014
Posted in: PHP/Zend, Zend, Zend. Leave a comment

The Zend_Cache is a great way to save (i.e. cache) the results of a function. Using ->call() it will automatically handle building a unique key for each cache based on the arguments passed to the function. So, all you have to do is ->call() the function and use the results as you normally would.

The following example caches the results of calling the getData function in the CustomStats class. The cache automatically expires after 120 seconds which results in the getData function being run only once every other minute.

$class_name    = 'CustomStats';
$function_name = 'getData';

try {
    $cache = Zend_Cache::factory('Function', 'File', array(
        'caching'                 => true,
        'lifetime'                => 120,   // 120 seconds
        'cache_id_prefix'         => $function_name,
        'automatic_serialization' => true
    ), array(
        'cache_dir'               => APPLICATION_PATH."/cache/",
        'hashed_directory_level'  => 2,
        'hashed_directory_perm'   => '0755',
        'cache_file_perm'         => '0644'
    ));

    if ($class_name) {
        $results = $cache->call(array($class_name,$function_name), 
                                array($param1,$param2,$etc));
    } else {
        $results = $cache->call($function_name, 
                                array($param1,$param2,$etc));
    }

} catch(Zend_Cache_Exception $e) {

    // Don't blowup if Zend_Cache fails ... 
    // ... just put a message in the log and continue.
    trigger_error($e->getMessage(), E_USER_NOTICE);

    $results = false;
}

print $results;

Amazon AWS CloudSearch Overview

Posted by kitepwr on January 30, 2014
Posted in: AWS, CloudSearch. Tagged: Amazon, AWS, CloudSearch. Leave a comment

AWS CloudSearch

I’ve been working with Amazon’s AWS CloudSearch product for about a year now and it continues to be a challenge in minimalism.  The product seemingly only has a few features. The trick is to figure out how to do more than just  few things with it….

This article is a quick introduction to CloudSearch including some of the limitations and workarounds possible.  My background is very Relational Database heavy which actually made it harder for me to grok CloudSearch.  Hopefully, this article will help others get up to speed quickly.

What CloudSearch is not

  • It is not a relational DB — it has no tables, no joins, no data ‘structure’.

What CloudSearch is

  • A flat domain or repository (i.e. a single “table”) of documents (i.e. “records”)
  • Each document is made up of many fields (i.e. columns)
  • The fields are automatically indexed in a MASSIVE way so queries are really fast

CloudSearch Schema

  • Field Types — there are only 3 data types in a document
    • text — data that can be searched for partial matches
    • literal — data that can be search for complete matching only
    • uint — unsigned integer, can be filtered on, searched, and returned as a facet
  • Field Attributes
    • search — can the field be searched?
    • facet   — should faceted results be generated for this field?
    • result  — should the contents of this field be returned in the results?
  • Ranking… is the ability to sort.  
    • Rankings can be ‘defined’ to Amazon for pre-indexing
    • or they can be provided on-the-fly
    • Example of a complex ‘test’ ranking built on-the-fly (in PHP)
      • $loc           = 1.5;
      • $type         = 1.0;
      • $activity  = 0.7;
      • $title         = 1.0;
      • $reviewQuality = 1.0;
      • $ratingPoints  = 0.9;
      • $this->addExpression(“test”, “(((((rating_avg*{$reviewQuality} / 10 ) + (rating_points*{$ratingPoints})  + cs.text_relevance({weights:{title:{$title}, geo:{$loc}, type:{$type},  default_weight:0.5})))/2 + ((cs.text_relevance({weights:{title:{$title}, geo:{$loc}, type:{$type}, default_weight:0.5}))/2)) * 2000)”);

What are Facets

  • Facets are literal or uint fields used to return ‘group’ counts based on the field values.
  • Text fields can not be used for facets.

Two ways to search

  • General Query: use q=<search terms> in the URL and it will search all the ‘searchable’ fields looking for <search terms> somewhere in those fields.
    • If the field is a literal it must match exactly.
  • Boolean Query: use bq=<field_name>:<search term> in the URL and it will search that particular field.
    • This is how facet drill down filtering is generally accomplished
    • Example: bq=(or title:’star’ (not title:’wars’)
    • If the field is a literal it must match exactly.

Loading data

  • To load data, send a json encoded associative array of arrays.
  • CloudSearch will always completely replace (never update) pre-existing documents.  This means we can not update single fields within a document.  All fields must be provided each time.
  • By default, the name of the associated key in the array decides which field in the index is updated.
    • There is, however, a way to tell CloudSearch to look for a fields data under a different key name.

Limitations and Gotcha’s and how to get around them

  • Only unsigned integer numbers are supported.
    • This means a ticket_price of $10.75 must be stored as 1075.
    • You will need to post-process the results and convert it back to 10.75.
  • Max of 100 values in a field
    • This means we can’t put a years worth of dates in a date field.
    • The solution is to have a date field for each month of the year (or each quarter) and post process the results into a combined array of values.
  • When Filtering on a facet field if the  field type is ‘text’ then partial matches are acceptable.  For example if the facet = “New York City” and the search is for “York New” it would match.  This great for “searching” but usually NOT what we want when filtering!
    • To drill into New York City, we need exact matching, so the field type must be literal.  But for General Search’s we want partial matching so we need another field of type  ‘text’.  The solution is to have 2 fields (one text; the other literal) and to assign them the same source field.
  • When obsoleting/renaming fields, there is always a race condition during the release process.  The old field is needed for a small amount of time and the new field is also needed at the same time.
    • The solution is to create a new field and tell it to get it’s data from two different sources: (1) the old fields data source and (2) the new fields data source.  And also modify the old field and have it get it’s data from these two sources as well.  For example:
      • old_field   => old_field + new_field
      • new_field => old_field + new_field
    • The above provides forward and backward compatibility for code that queries as well as updates the Index.
    • The obsolete field can be removed after the new field is poplated and all the old code references are no longer used.
  • Deleted documents are sometimes returned if no search criteria is provided.  For example, you can do “searches” with “no search criteria” in order to get high-level facet counts.  Having deleted records included in these counts is a problem:
    • The solution is to set a default_value of 0 to a field that would never normally have a zero value.   An internal document_id value works well for this.  Then, whenever you do an ‘empty’ search, include a filter of document_id != 0.

Amazon AWS CloudSearch Hierarchies

Posted by kitepwr on January 29, 2014
Posted in: AWS, CloudSearch. Tagged: Amazon, AWS, CloudSearch. Leave a comment

AWS CloudSearch

I’ve been working with Amazon’s AWS CloudSearch product for about a year now and it continues to be a challenge in minimalism.  The product seemingly only has a few features. The trick is to figure out how to do more than just  few things with it….

Handling Hierarchical data:

We have a 3-tier hierarchy of Geography data for each Activity stored in the search domain. Unfortunately, there is no native support for anything like hierarchical data in AWS-CS.  So, we need to create our own custom solution.  There are two parts to the solution: (a) structuring the data in AWS and (b) post-processing the search results into a nested (hierarchical) array.

Given an Activity that takes place in 2 different Geo’s as follows:

  • Phoenix
    -> Scottsdale
    -> Downtown Scottsdale
  • Phoenix
    -> Mesa
    -> Apache Wells

Create 3 literal facet fields with the following content.NOTE:  AWS uses |‘s as delimiters between faceted fields and we chose to use >‘s between hierarchy levels of the geographies.

  • geo_level_1 = Phoenix|Phoenix
  • geo_level_2 = Phoenix>Scottsdale|Phoenix>Mesa
  • geo_level_3 = Phoenix>Scottsdale>Downtown Scottsdale|Phoenix>Mesa>Apache Wells

In the search results from AWS, the above 3 facet fields will be represented as facet data “something like” this:

  • geo_level_1:
    • value=”Phoenix”    count=12
  • geo_level_2:
    • value=”Phoenix>Scottsdale”  count=4
    • value=”Phoenix>Mesa” count=5
  • geo_level_3
    • value=”Phoenix>Scottsdale>Downtown Scottsdale” count=3
    • value=”Phoenix>Mesa>Apache Wells” count=2

Since the level n data contains the level n-1 parent values in it, it’s a simple matter to convert this “in code” to a proper multi-dimensional array like this…

  • geo:
    • [0]
      • value=”Phoenix”
      • count=12
      • primary_key=”Phoenix”
      • [0]
        • value=”Scottsdale”
        • count=4
        • primary_key=”Phoenix>Scottsdale”
        • [0]
          • value=”Downtown Scottsdale”
          • count=3
          • primary_key=”Phoenix>Scottsdale>Downtown Scottsdale”
      • [1]
        • value=”Mesa”
        • count=5
        • primary_key=”Phoenix>Mesa”
        • [0]
          • value=”Apache Wells”
          • count=2
          • primary_key=”Phoenix>Mesa>Apache Wells”

The above array makes presenting this information on a web page in a visually intuitive way super simple.  Notice that we’ve change the ‘value’ field to match just the last node/level of the geo name since this is what want to display to the end users.  And, we’ve introduced a new field called primary_key that can be used as an ‘exact match’ filter value for drilling to that specific geo.

When filtering on the geography, the calling program will pass the ‘primary_key’ to the search function. The search function looks at the number of >‘s and determines if the filter is for a geo_level_1, 2, or 3 field. It then constructs a match express like (and geo_level_3:’Phoenix>Mesa>Apache Wells’). Since the geo fields are defined as ‘literal’ we don’t need to worry about CloudSearch inadvertently doing any partial matching. We always pass a 100% exactly matching value for these fields.

This approach provides a complete, end-to-end solution for dealing with displaying and filtering hierarchical data in AWS CloudSearch.

Searching and Filtering on the same faceted data:

In the above scenario, we created a way to handle displaying and filtering hierarchical facet data.  But what if we want to search that data too?  The are two blockers to that idea (1) facet fields can not ‘also’ be searchable and (2) literal fields require exact matching to be returned.  The solution is to create another field that is (a) searchable and (b) defined as text (assuming you want partial matching to return results!).  The other “trick” is to define the new field to use the existing field as it’s source.  In this way, no additional work is required to maintain the new fields’ data.   If you know that all your geo’s are 3 levels deep, then you can just use geo_level_3 as a source since it has the super set of names in it already.  Otherwise, fields for additional levels will also need to be created.  Alternatively, you can specify all 3 fields (geo_level_1, geo_level_2, and geo_level_3) as source fields for the new searchable field.  The values for each of the source fields will be concatenated into the target field.

If you do need to include multiple geo level fields into your search field, be aware that this will cause values like ‘Phoenix’ to appear more than once in the data.  As a result when someone searches for “gondola rides in Phoenix”, the text_relevance score will be more weighted toward the geo_search field than other fields (i.e. wherever ‘gondola’ and ‘rides’ was found).  This can be handled using relative field weighting in you rank expression… but that’s a topic for another day 🙂   If you can’t wait, here’s Amazon’s documentation about that

… http://docs.aws.amazon.com/cloudsearch/latest/developerguide/fwranking.html

Zend Framework Forms

Posted by kitepwr on January 28, 2014
Posted in: PHP/Zend, Zend. Tagged: Form, PHP, Zend. Leave a comment

Zend Forms … what can you do with it?

The general structure of a Zend_Form class is :

class my_form extends Zend_Form
{
    protected $_originalData; //holds the data used to populate the form.

    /**
     * setOriginalData is automatically used by the
     * constructor when invoked with the argument...
     *           (array('originalData' => $array_of_data));
     */
    public function setOriginalData(array $original_data)
    {
        $this->_originalData = $original_data;
        return $this;
    }

    public function init()
    {
        if (empty($this->_originalData)) {
            throw new Zend_Form_Exception("originalData must be set during construction.");
        }

        $this->setName('MapLocationForm')
             ->setAttrib('enctype', 'multipart/form-data');

        // initialize form
        $this->setMethod('post');

        // --------------- csrf handling --------------------------
        $this->addElement(new Zend_Form_Element_Hash('csrf',
                                                     array('salt'    => $this->getName(),
                                                           'timeout' => 1800)));

        $this->getElement('csrf')
             ->getValidator('Identical')
             ->setMessage('There was a problem with your form submission, please try again.');

        // ---------------------------------------------------------
        $street = new Zend_Form_Element_Text('map_street');
        $street ->setOptions(array('size' => '40', 'maxlength' => '60'))
                ->setRequired(false)
                ->addFilter('StringTrim')
                ->setLabel("Street address");

        $city = new Zend_Form_Element_Text('map_city');
        $city ->setOptions(array('size' => '40', 'maxlength' => '40'))
              ->setRequired(false)
              ->addFilter('StringTrim')
              ->setLabel("City");

        $submit = new Zend_Form_Element_Submit('submit');
        $submit ->setOptions(array('ignore' => true))
                ->setAttrib('class','bp') // add a class of 'bp' to the input element
                ->setLabel('Save');

        $cancel = new Zend_Form_Element_Button('cancel');
        $cancel ->setOptions(array('ignore' => true))
	        // add a class of 'bs' to the button element
                ->setAttrib('class','bs')
                // suppress the ability to 'tab' to the cancel button
                ->setAttrib('tabindex','-1')
                ->setLabel('Cancel and exit to other.php')
                ->setAttrib('onclick', "window.location.href='/other.php';return false");

        // setup the form elements
        $this->addElement($street)
             ->addElement($city)
             ->addElement($submit)
             ->addElement($cancel);

        // add data to the form...
        $this->populate($this->_originalData);
    }

}

The point of this blog is to highlight the things you can do with a form element all in one place (i.e. here).

->setLabel("Element Label")

->setRequired(true|false)

->setAtrtrib('class','<class_name>') // associate a class name with the element
->setAttrib('tabindex','-1') // suppress tabbing to this element

->setOptions(array('size'      => '20,'
                   'maxlength' => '40') // for text elements
->setOptions(array('rows'      => '2',
                   'cols'      => '30',
                   'maxlength' => 200,
                   'style'     => 'width:100%')) // for textarea elements

->addDecorator('HtmlTag', array('tag'=>'div',
                                'class'=>'input-wrapper'))

->addValidator('NotEmpty',
               true,
               array('messages'=> "Please enter a value, it's reqired."))
->addValidator('regex',
               true,
               array(NUMBER_6DECIMAL,
                     'messages'=> "Please enter a number with up to 6 decimals."))
->addValidator('between',
               true,
               array('min' => -90,
                     'max' => 90,
                     'messages'=> Please enter a Latitude between -90 and 90."));
->addValidator('Int',
               true,
               array('messages' => "zip code must be all numeric."))
->addValidator('StringLength',
               true,
               array(3, 5, 'messages'=>"value must be 3 to 5 characters long."));
->addValidator('StringLength',
               true,
               array(5, 5, 'messages'=>"zip code must be 5 digits."));

NOTE: NUMBER_6DECIMAL = /^-?[0-9]*(\.[0-9]{1,6})?$/

Posts navigation

  • Recent Posts

    • Zend Form — validate an array of data
    • Zend Form — validate an array of data
    • Add color to your ssh sessions – Mac OSX
    • Using Zend Cache to save the results of a function
    • Amazon AWS CloudSearch Overview
  • Recent Comments

    kitepwr on Zend Form — validate an…
  • Archives

    • July 2015
    • May 2015
    • November 2014
    • May 2014
    • January 2014
  • Categories

    • AWS
    • CloudSearch
    • PHP/Zend
    • Zend
    • Zend
  • Meta

    • Register
    • Log in
    • Entries feed
    • Comments feed
    • WordPress.com
Blog at WordPress.com.
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy
  • Follow Following
    • Darrin's Tech Ramblings
    • Already have a WordPress.com account? Log in now.
    • Darrin's Tech Ramblings
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar