The Perils of Foreach, or “I’ve got yer references dangling right here!”

Here’s another one of those pitfalls in PHP that might be classified as a neophyte mistake, but nevertheless it bit me in the ass a while ago. I was perplexed at first why this one test I had written kept failing, and finally it boiled down to the way I was iterating through an array multiple times in the same block of code. As it turned out, I had made the mistake of leaving a reference to a variable hanging around after I was done with it. Later, in the same (unfortunately lengthy) block of code, I reused the variable name containing the reference for something else (assigned it a value), and it gave me some, shall we say, unexpected results.

Here, it’s easier to demonstrate with a short example. My bug was buried deep in some more convoluted code, this is just a simplified version of the same sort of thing:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

<?php
 
$fruits = array(
	'apple',
	'banana',
	'cherry'
);
 
echo "foreach with assign by reference:\n";
foreach($fruits as &$fruit) {
	echo $fruit . "\n";
}
 
echo "\nforeach with assign by value:\n";
foreach($fruits as $fruit) {
	echo $fruit . "\n";
}
 
?>

The output for this looks something like this:

So yeah, you’ve got your fruits in alphabetical order, and the second time you go through the list you end up with two bananas and no cherries. The hell?

Problem Number A: Dangling Reference

Let me just start by saying the example above is a very simplified example of the core problem. I’m sure some of you are wondering why I might bother writing code that uses references like this, but rather than dive into that right now, just know that I’m not insane and I had my reasons. Accepting the code as it stands, the first foreach loop is iterating through the $fruits array and assigning individual elements by reference. It works well enough – if you only ran that part of the program, you would never see a bug. However, there is a problem with it, and it is this: the $fruit reference remains even after the foreach loop is over. The variable $fruit points to a location that contains the value ‘cherry’. Also, because of the way references work, the third element of the $fruits array also points to the value ‘cherry’. Check it out:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

<?php
 
$fruits = array(
	'apple',
	'banana',
	'cherry'
);
 
echo "foreach with assign by reference:\n";
foreach($fruits as &$fruit) {
	echo $fruit . "\n";
}
var_dump($fruits);
$fruit = 'date';
var_dump($fruits);
 
?>

As you can see, the variable $fruit contains a reference, even after we’re done with it in the foreach loop. It points (or ‘refers’, if you like) back to a memory location that happens(just after the end of the loop) to contain the string ‘cherry’. Look closer, though – there’s something else going on that’s equally improtant. Notice the value of $fruits[2]. It, too, is a reference! That’s just the way references work – the moment you create a reference to a value, the original variable that contained the value also becomes a reference to that value. It’s one location in memory with two pointers; assign a new value to either and the other will point to the changed value as well.

Now, just to completely beat this dead horse, after the foreach loop, $fruit contains a reference, because that’s the last thing we assigned to it. PHP doesn’t care that the loop is over, it has no way of knowing whether or not you intend to use that reference value later on. When you assign the value ‘date’ to the variable $fruit, you’re putting the value ‘date’ into the location $fruit was referencing. The $fruits array is modified, ‘cherry’ becomes ‘date’.

Going back to the original example, it should now be (hopefully) apparent what’s going on – it’s really the same exact thing. Once the foreach by reference loop is over, $fruit contains a reference to a memory location that contains the string ‘cherry’. At the same time, $fruits[2] contains a reference to the same space. When we hit the second foreach loop, we’re iterating through the $fruits array and assiging that value to the location to which $fruit points, which (again) affects $fruits[2]. The 1st iteration, ‘apple’ is assigned. On the 2nd, it’s ‘banana’. At this point, $fruits[2] and $fruit both point to a location containing the string ‘banana’. On the last iteration, $fruits is assigned the value of whatever value is pointed to by $fruits[2] which was just changed to ‘banana’. Here’s a modified version of the original example with some different output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

<?php
 
$fruits = array(
	'apple',
	'banana',
	'cherry'
);
 
echo "foreach with assign by reference:\n";
foreach($fruits as &$fruit) {
	echo $fruit . "\n";
}
 
echo "\nforeach with assign by value:\n";
$count = 0;
foreach($fruits as $fruit) {
	echo "assigning value of \$fruits[$count] to \$fruit " .
	     "and also \$fruits[2]: " . $fruits[$count] ."\n";
	$count++;
}
echo "after 2nd foreach loop:\n";
var_dump($fruits);
?>

Finally, a Point

If you’ve managed to get this far, congratulations! The upshot of all this dancing around with references is this little nugget o’ advice:

Always unset a reference variable after you’re done with it, especially following a foreach loop!!!

If you don’t believe me, check out the php documentation. Doing so will destroy the reference, and prevent the kind of bug that can be quite the PITA to pin down. Here, then, is the example one last time, this time “fixed” with an unset call:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

<?php
 
$fruits = array(
	'apple',
	'banana',
	'cherry'
);
 
echo "foreach with assign by reference:\n";
foreach($fruits as &$fruit) {
	// etc.
}
unset($fruit);
 
foreach($fruits as $fruit) {
	// etc.
}
echo "after 2nd foreach loop:\n";
var_dump($fruits);
?>

And the blessed, correct output:

As a programmer, I have read through more fat books than I care to think about. That doesn’t mean I’ve retained it – quite the opposite. I’ve probably forgotten at least ten facts for any one I remember. That said, there are times when I’m working on something that I have that “Oh yeahhhh… a-Dur! *self thwap*” moment when something I should know pops up and says hello. Recently, I had a moment like that when I was (foolishly) trying to check a float value for equality in PHP.

Allow me to back the truck up for a moment. In case you don’t know, float values are numbers with a decimal component, e.g. 1.234 . In PHP, you end up with a float any time you explicitly create a decimal, like so:

You can end up with a float value many other ways, but you get the idea. The trouble with floats is this: the value of a float is always an estimate!. That might not seem like a surprise to you when considering the decimal representation of, say, 2/3. However, it can be annoying if you (as I did) stop thinking of floats as what they are for a moment in the context of what you’re doing.

Before I babble any further, here’s a stripped down code example, illustrating the basic problem I encountered:

1
2
3
4
5
6
7
8
9
10
11
12

<?php
 
$x = .6 + .3 + .1;
var_dump($x); // value of $x shows as "float(1)"
 
if($x == 1) {
    echo "$x = 1. All is right in the world\n";
} else {
    echo "$x != 1? The hell you say?!?\n";
}
 
?>

Computers don’t store numbers using base 10. That is, a computer doesn’t think of .6 as six tenths. It tries to store that fraction as best it can using what essentially boils down to ones and zeroes. In that conversion, rounding occurs. It is similar to when we have to represent a fraction like 2/3 as a decimal. Without “cheating” and putting a line over the repeating digit, the best we can do is something like 0.66666667, where the final 7 is the point at which we got tired of writing sixes, gave up, and wrote a 7. We round up, figure that’s close enough, and move on with our lives. The computer does the same thing, but some times it’s not as obvious.

In the above script, when the value of $x is dumped to the screen, it is shown as “float(1)”. That really only adds to the confusion, because it’s clearly not equal to 1. So what is $x? The var_dump function is actually rounding off the value. However, in PHP, you can output the value of a float with forced precision using the printf function, using some formatting directives. Here’s an example: printf("%.30f", $x); The string “%.30f” is what’s called a *conversion specifier* that tells the computer to represent the argument (the $x) as a float with a precision of 30 decimal places. See the documentation for more information. When we add that to the example script and run it, things become a little easier to understand:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

<?php
 
$x = .6 + .3 + .1;
echo "var_dump of \$x: \n";
var_dump($x); // value of $x shows as "float(1)"
echo "\n more precise version of \$x:";
printf("%.30f\n\n", $x); // value of $x shows as 0.999999999999999822364316059975
 
if($x == 1) {
    echo "$x = 1. All is right in the world\n";
} else {
    echo "$x != 1? The hell you say?!?\n";
}
 
?>

Here’s the final revised code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

<?php
 
$x = .6 + .3 + .1;
echo "var_dump of \$x: \n";
var_dump($x); // value of $x shows as "float(1)"
echo "\n more precise version of \$x:\n";
printf("%.30f\n\n", $x); // value of $x shows as 0.999999999999999822364316059975
 
if(round($x, 5) == 1) {
    echo "$x = 1. All is right in the world\n";
} else {
    echo "$x != 1? The hell you say?!?\n";
}
 
?>

That does it for the basic bit, and ends the “beginner” part of this post. For the truly masochistic: Here’s a description of the scenario:

I was working on something that involved percentage weights for a list of values. Basically, each value had a weight reflecting its relevance that was factored in when all of the values were added. The sum of the weights was 100%, and were 60%, 30%, and 10% for the 3 values. In essence, here’s how it played out:

So far so good. However, I also needed to account for the possibility that all three values might not be present. For the requirements of the problem (and I’m keeping the example much more simplified than the actual code involved), I needed to ’scale’ the adjusted_value by the total weights of the values that were present. For example, if only the first two values were present, the total weight would be 60% + 30% = 90%, so I would have to divide my adjusted_value by the total weight, or:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

<?php
$weights = array(.6, .3, .1);
$values = array(75, 62, 80);
 
$count = 0;
$total_weight = 0;
$adjusted_value = 0;
foreach ($values as $v) {
  if (isset($v)) {
    $adjusted_value += $v * $weights[$count];
    $total_weight += $weights[$count];
  }
  $count++;
}
if ($total_weight == 0) { // covers case to avoid divide by zero
  $adjusted_value = 0;
} elseif($total_weight != 1) { // error: this *always* gets fired!
  $adjusted_value /= $total_weight;
} else {
  // code that never gets fired, even at 100%
}
?>

Changing the comparison to round the float value being compared gives me the result I expected: } elseif(round($total_weight, 5) != 1) { Once again, all is right in the world.