Multi values field development drupal

Field api is a very useful feature in drupal in terms of extension of entity, reuse of code and modularization of software development.

To me the Field is the way to establish the references between two data block, the entity and the Field, as in drupal Field can have columns, I have yet to see how to establish references between entities without any use of none core modules, using Field can allow us to achieve similar results with just built in drupal core modules.

We all are familiar with master and detail table relationship in database, one to one or one to many reference between tables, you can easily extend that kind of concept into drupal , Entity is the master table, while Field is the child table. How about one to one or one to many relationship mapping?
Well in Field API, there is an interesting property $field[cardinality] which can be set to 1, more than one and unlimited, this means a Field, is designed to have multiple values for us to achieve one to many reference.
We are one the right track, even though there have been some significant hurdles to overcome to really implement a multi values Field, but I am determined in trying to find a satisfactory solution as in most of our projects, one to many relationship is one of those most ordinary data structures we need to use.

In the core of drupal, unfortunately there is not much example we can draw on, only mutli values Field example is taxonomy reference, as it is using a comma delimited string as the form of multi values, it is hardly useful for us to learn the standard widget authoring of a multi values Field. With the basic features like add a new item, remove item, changing weights of items etc.

The drupal has implemented add a new item, and weighting management, but not remove button, if you are looking for a solution for remove button, you have come to right place.
I do have principles when I do drupal development, I will try my best to reuse any existing implementations that are provided by drupal, for parts that do not suit, there are hooks for us to use, and do everything as its intended way, there might be short cuts, but following the guidelines of drupal design is very critic for the quality and reuse of our code.
Some back ground about my example project, all I want to build is a contact field, as it is very possible there might be more than one contact, so it is a multi values field, it has a few columns like phone, fax, mobile, email etc, to make it slightly more complicated, it has a contact type, when contact type is person , then first name and sur name columns apply, but when contact type is a company, then company name applies. This is of course to be implemented in ajax way.
Here is the screen short of the end result

MultiValuesField

First to make a field multiple values:
Set cardinality as unlimited(-1), you could set it to be more than 1, but we are talking unlimited instances here.
There are two ways to set cardinality, one is from admin UI, the other is through code

  $field = array(
        'field_name' => $fieldname,
        'type' => ‘xxxx_contactfield',
        'entity_types' => array('user'),
        'cardinality' => FIELD_CARDINALITY_UNLIMITED,
      );

    field_create_field($field);

Second set right value to multiple values behaviour in hook_field_widget_info

   'behaviors' => array(
        'multiple values' => FIELD_BEHAVIOR_DEFALT,
      ),

It can also be FIELD_BEHAVIOR_CUSTOM, so what is right value? This is very tricky, when it is set to default, you will be able to see ‘add new item’ and widget can be repeated. While if it is set to custom it means, your widget will be rendered once, no ‘add new item’, your widget is responsible for repeated rendering of controls , as I said I would like to use drupal’s implementation as much as possible, so I choose the former, set it to default, the widget we are authoring is only for one instance.
Hook_field_widget_form

function xxxx_contactfield_field_widget_form(
    &$form, &$form_state, $field, $instance, $langcode,
    $items, $delta, $element)
{

  $contact = array();

  if (!empty($items[$delta])) {
    // Else use the saved value for the field.
    $contact = $items[$delta];
  }

  // get field_state
  $parents = $form['#parents'];
  $field_name= $field['field_name'];
  // use field_form_get_state to get current field state
  $field_state = field_form_get_state($parents, $field_name, $langcode, $form_state);

  // control blank item
  // only render one when items_coount 0 which is first initial rendering
  // if ($field_state['items_count']> 0 && $delta ==$field_state['items_count'])
  if ($delta ==$field_state['items_count'])
  {
    if ($field_state['items_count']==0 &&
        !(isset($form_state['process_input'])&&$form_state['process_input']))
    {
      // this is the first rendering of field(must exclude remove scenario), no item
      // we must gaurantee at least one item is displayed
      // set $field_state['items_count']==1 in order to work with add more button
      // in this case, minimu of $field_state['items_count'] is one

      $field_state['items_count']++;
      field_form_set_state($parents, $field_name, $langcode, $form_state, $field_state);

    }
    else if($field_state['items_count']==0){
      // this is because of last item being removed
      // display a message
      // add desction
      // this serves as fake item, from form otherwise, no update will be performed on field update
      $element['noitemsdescription'] = array(
          '#type'=> 'markup',
          '#markup'=>'</pre>
<div>All contacts have been removed, save to confirm</div>
<pre>
',
      );

      return $element;
    }
    else
    {
      $element=array();

      return $element; // not rendering anything
    }
  }

  //always recover items from form_state

  if (isset($form_state['values']))
  {
    $key_exists = NULL;

    $deltaParent = array_merge($parents,array($field_name, $langcode, $delta));

    $value = drupal_array_get_nested_value($form_state['values'],
        $deltaParent, $key_exists);

    if ($key_exists) {
      // replace contact type with new value
      $contact = $value;

    }
  }

  // Merge in default values to provide a value for every expected array key.

  $contact += _xxxx_contactfield_default_value();

  // always having a tree structure for UI
  $element['#tree'] = true;

  // add the type
  $element['contacttype'] = array(
      '#type' => 'select',
      '#title' => t('Type'),
      '#weight' => 10,
      '#options' => array(
          10=>t('Personal'),
          20=>t('Company')),
      '#default_value' => $contact['contacttype'],
      '#ajax' => array(
          'callback' => 'xxxx_contact_type_callback',
          // 'wrapper' is the HTML id of the page element that will be replaced.
          'wrapper' => 'names_block_div'.$delta,

          'effect' => 'fade',

          'method' => 'replaceWith',
      ),
  );

  // The names block.
  $element['names_block'] = array(
      '#type' => 'xxxx_container',
      '#attributes' => array('class' => array('names-block')),
      '#id' => 'names_block_div'.$delta, // this will be controlled by ajax
      '#weight' => 20,
  );

  // then render names_block depending on which contact type is
  switch ($contact['contacttype'])
  {
    case 10:
      // it is a person
      $element['names_block']['first_name'] = array(
      '#type' => 'textfield',
      '#title' => t('First name'),
      '#size' => 50,
      '#weight' => 24,
      '#default_value' =>$contact['first_name'],
      '#attributes' => array('class' => array('contact-first-name')),
      '#required' => TRUE,
      );

      $element['names_block']['last_name'] = array(
          '#type' => 'textfield',
          '#title' => t('Last name'),
          '#size' => 50,
          '#weight' => 26,
          '#default_value' =>$contact['last_name'],
          '#attributes' => array('class' => array('contact-last-name')),
          '#required' => TRUE,
      );

      break;
    case 20:
      // it is a company
      $element['names_block']['organisation_name'] = array(
      '#type' => 'textfield',
      '#title' => t('Company'),
      '#size' => 50,
      '#weight' => 22,
      '#default_value' =>$contact['organisation_name'],
      '#attributes' => array('class' => array('contact-organisation-name')),
      '#required' => TRUE,
      );
      break;

    default:
      break;

  }

  $element['mobile'] = array(
      '#type' => 'textfield',
      '#title' => t('Mobile'),
      '#size' => 50,
      '#weight' => 30,
      '#default_value' =>$contact['mobile'],
      '#attributes' => array('class' => array('contact-mobile')),
  );

  $element['workphone'] = array(
      '#type' => 'textfield',
      '#title' => t('Work'),
      '#size' => 50,
      '#weight' => 40,
      '#default_value' =>$contact['workphone'],
      '#attributes' => array('class' => array('contact-workphone')),
  );

  $element['ahphone'] = array(
      '#type' => 'textfield',
      '#title' => t('After hours'),
      '#size' => 50,
      '#weight' => 50,
      '#default_value' =>$contact['ahphone'],
      '#attributes' => array('class' => array('contact-ahphone')),
  );

  $element['email'] = array(
      '#type' => 'textfield',
      '#title' => t('Email'),
      '#size' => 50,
      '#weight' => 60,
      '#default_value' =>$contact['email'],
      '#attributes' => array('class' => array('contact-email')),
  );

  $element['bestway'] = array(
      '#type' => 'textfield',
      '#title' => t('Notes'),
      '#size' => 50,
      '#weight' => 70,
      '#default_value' =>$contact['bestway'],
      '#attributes' => array('class' => array('contact-bestway')),
  );

  // not to render remove button when the rendered is the default blank
  if ($field_state['items_count']>1 ||
      ($field_state['items_count']==1 &&
          !xxxx_contactfield_field_is_empty($contact,$field)))
  {
    $buttonparents = array_merge($form['#parents'],array( $field_name, $langcode, $delta,'remove'));

    // this drupal style naming in field
    $name = array_shift($buttonparents);

    $name.= implode('_', $buttonparents);
    // $name .= '[' . implode('][', $buttonparents) . ']'; // in this way button will not be recognized

    $limitvalidationerrors = array();

    // only limit validation errors to items on other row
    for($i=0;$i<$field_state['items_count'];$i++)     {       if ($i!= $delta)       {         $limitvalidationerrors[] = array_merge($parents, array($field_name, $langcode, $i));       }     }           $element['remove'] = array(         '#type' => 'submit',
        '#value' => t('Remove'),
        '#name' => $name,
        '#weight' => 80,
        // '#limit_validation_errors' => array(array_merge($parents, array($field_name, $langcode, $delta))),
        '#limit_validation_errors' =>$limitvalidationerrors,
        '#submit' => array('xxxx_contactfield_remove_submit'),
    );
  }
  return $element;
}

Well it is a lot to take, let me have some explanation
Interface of hook_field_widget_form

The interface must look exactly same, including the references based parameters &$form and &$form_state

$items and $delta

Where $items are holding values from DB of this Field, $delta is the sequence number of instance it is rendering.

$field_state

Understanding $field_state[items_count] is very important for multi values field development, it starts from count of $items, then it is controlled by ‘add new item’ click, and by our ‘remove’ button. Another big issue is default implementation of instances of Field is always attaching a blank widget at the end, this to me is not very desirable, especially when you have already got items, a blank widget is attached, and ‘add new item’, I prefer no blank item automatically created, it will be triggered by ‘add new item’, but the initial rendering of Field the blank item is rendered to let user know what this field is about. So it is at least rendered once, but not rendered when there is always instances in Field
$delta is between 0 and $field_state[items_count], so you can see from code the last item has been returned with empty array.
And for the first time rendering of the blank field, it is rendered, and for consistence $field_state[items_count] it is set to 1.
Retrieving of values of instances
The principle here is if there is value in $items, load it first, due to this function is called as a result of handling of ajax, ‘add new item’ and remove button clicks, we need to restore value from $form_state if there are values in form_state[values]
Contacttype ajax

The ajax call back is

function xxxx_contact_type_callback($form, $form_state) {
  // The form has already been submitted and updated. We can return the replaced
  // item as it is.
  $names_block = array();

  $n =sizeof($form_state['triggering_element']['#parents']);

  $parents = $form_state['triggering_element']['#parents'];

  $parents[$n-1] = 'names_block'; // replace contacttype with names_block

  $key_exists = NULL;
  $value = drupal_array_get_nested_value($form, $parents, $key_exists);
  if ($key_exists) {

    $names_block = $value;

  }

  return  $names_block;

}

Basically it returns a new name_block, be aware that I embedded name related columns under [name_block] which is a customized type, the parents of these columns need to bypass this [name_block] layer.
Then remove button:
First it must have a name, otherwise it will be all using #name of ‘op’ which will not tell you which remove button has been clicked
Second the name needs to be unique, and can not use naming conventions as Field columns like [fieldname][languagecode][delta], because doing this will bury button value in $form_state['input'][fieldname][und][0] rather than discoverable $form_state['input'][$element['#name']] for _form_button_was_clicked(),

Remove button is not rendered on first blank item, and it has complex #limit_validation_errors, what it does is bypass errors on the same item which is going to be removed, but other items still need validation.
And the submit handler for remove button

function xxxx_contactfield_remove_submit($form, &$form_state)
{
// the way the removed button is designed with #limit_validation_errors
// so all form values except the removed item are available in form_state
// but what we do here is to move the next one forward,
// for the last one we do not have any form_state values except button value

  $button = $form_state['triggering_element'];

  $length =  sizeof($button['#parents']);

  $field_name= $button['#parents'][$length-4];

  $langcode = $button['#parents'][$length -3];

  $delta =  $button['#parents'][$length-2];

  // get field_state
  $parents = $form['#parents'];

  // use field_form_get_state to get current field state
  $field_state = field_form_get_state($parents, $field_name, $langcode, $form_state);

  // delete row in form_state
  if ($delta <$field_state['items_count']-1)
  {
    for ($i = $delta;$i<$field_state['items_count']-1;$i++)
    {
      $itemParent = array_merge(array_slice($button['#parents'],0, $length-2), array($i));

      // get the next item
      $nextParent = array_merge(array_slice($button['#parents'],0, $length-2), array($i+1));

      $key_exists = NULL;

      $value = drupal_array_get_nested_value($form_state['values'],
          $nextParent, $key_exists);
      if ($key_exists) {

        $nextItem = $value;

        // replace this item with next item
        drupal_array_set_nested_value($form_state['values'],  $itemParent, $nextItem);
        // need to replace $form_state['input'] and all, otherwise values will be read from input by form_builder()
        drupal_array_set_nested_value($form_state['input'],  $itemParent, $nextItem);

      }
    }

  }

  //no need delete the last item as items_count decreasing will cause it

  // decrease items_count
  $field_state['items_count']--;
  field_form_set_state($parents, $field_name, $langcode, $form_state, $field_state);

  $form_state['rebuild'] = TRUE;
}

What it does is to move the items one place forward, and leave the last item, as items count is decreased by one, the moving must be performed on form_state[input] as well, as form rebuild will look to use form_state[input] collections.
Ok, mission accomplished… just joking, the task is not completed without any intro on ‘add new item’ system button, and the change of $field_state[items_count] to not render one extra blank item has some implication, especially when has a look of implementation of ‘add new item’.

There are two parts of ‘add new item’, one is the submit handler
field_add_more_submit() basically what it does is to increase field_state[items_count ] by one. Another part is the callback field_add_more_js()

$delta = $element['#max_delta'];
  $element[$delta]['#prefix'] = '</pre>
<div class="ajax-new-content">' . (isset($element[$delta]['#prefix']) ? $element[$delta]['#prefix'] : '');
 $element[$delta]['#suffix'] = (isset($element[$delta]['#suffix']) ? $element[$delta]['#suffix'] : '') . '</div>
<pre>
';

So it uses #max_delta to add a wrapper div around it, as we fiddled with items we displayed, so #max_delta is one greater than real number of instances we have rendered, the symptom is we will see an empty item with nothing in it, as setting of $element[$delta]['#prefix'] is unconditional, so if $element[$delta] does not exist which is our case, it creates one.
To modify value of #max_delta is not easy, as it is not accessible from hook_field_widget_form(), it is added by field_multiple_value_form(), I will have to use Field Attach API hooks to get the access, the thing is Field Attach api hooks are hooks that are triggered on any module and any entity, they do not appear performance efficient to me. When choosing Field Attach API, there is only hook_field_attach_form() we can use for this purpose as we have to use form elements.

/**
 * Implements hook_field_attach_form().
 */
function xxxx_contactfield_field_attach_form($entity_type, $entity, &$form, &$form_state, $langcode) {
  // as only displayed field_state items_count -1 , so #max_delta needs to be correct
  // otherwise the add new item button will have trouble
  $entity_info = entity_get_info($entity_type);
  $bundle_name = empty($entity_info['entity keys']['bundle']) ? $entity_type : $entity->{$entity_info['entity keys']['bundle']};
  foreach (field_info_instances($entity_type, $bundle_name) as $field_instance) {
    // to get the field
    $field= field_info_field($field_instance['field_name']);

    if ($field['type'] == 'xxxx_contactfield') {

      // we do not have $langcode, so have to get field a level up

      // to get #max_delta
      $parents = array_merge($form['#parents'], array($field['field_name']));

      $key_exists = NULL;
      $value = drupal_array_get_nested_value($form, $parents, $key_exists);
      if ($key_exists) {

        foreach(element_children($value) as $key)

         if(isset($value[$key]['#max_delta']))
         {
            if ($value[$key]['#max_delta']>0)
            {
               $value[$key]['#max_delta']-- ;

               $newparents = array_merge($parents,array($key));
               drupal_array_set_nested_value($form, $newparents, $value[$key]);

            }

         }

      }
      break;
    }
  }

}

As you can see there is a lot of work done just to tell if our interested field is present. This function is called for every entity, the system does not filter these kind of hooks based on field usage, as a result, $field is not in the interface.

Reordering items by dragging
This is implemented by drupal, and works very well, the table view of items had a _weight field added to every item, and when saving, the items will be sorted by _weight field, no extra code is needed.
That is pretty much all I have to do to have a functioning multi values field.
The understanding of all the tricks and traps of making multiple values field widget is very important that we can use the same understanding to create other one to many references, like multi upload of files, etc, the potential is unlimited.