PHP – Tokenize string – David Kittell

This PHP function will assist in splitting a string based on characters.

Example 1:

PHP

<?php
$string = "This is\tan example\nstring";
/* Use tab and newline as tokenizing characters as well  */
$tok = strtok($string, " \n\t");

while ($tok !== false) {
    echo "Word={$tok}\n";
    $tok = strtok(" \n\t");
}
?>

<?php
$string = "This is\tan example\nstring";
/* Use tab and newline as tokenizing characters as well  */
$tok = strtok($string, " \n\t");

while ($tok !== false) {
    echo "Word={$tok}\n";
    $tok = strtok(" \n\t");
}
?>

Result: Word=This Word=is Word=an Word=example Word=string

Example 2:

PHP

<?php
$first_token  = strtok('This/is/an/example/string', '/');
$second_token = strtok('/');
var_dump($first_token, $second_token);
?>

<?php
$first_token  = strtok('This/is/an/example/string', '/');
$second_token = strtok('/');
var_dump($first_token, $second_token);
?>

Result: string(4) "This" string(2) "is"

Example 3:

PHP

<?php
$string = ";This;;is;;an;;example;;string;";

$parts = [];
$tok = strtok($string, ";");
while ($tok !== false) {
    $parts[] = $tok;
    $tok = strtok(";");
}
echo json_encode($parts),"\n";

$parts = explode(";", $string);
echo json_encode($parts),"\n";

<?php
$string = ";This;;is;;an;;example;;string;";

$parts = [];
$tok = strtok($string, ";");
while ($tok !== false) {
    $parts[] = $tok;
    $tok = strtok(";");
}
echo json_encode($parts),"\n";

$parts = explode(";", $string);
echo json_encode($parts),"\n";

Result: ["This","is","an","example","string"] ["","This","","is","","an","","example","","string",""]

Source: http://www.php.net/manual/en/function.strtok.php

Originally Posted on July 8, 2014
Last Updated on February 2, 2026

All information on this site is shared with the intention to help. Before any source code or program is ran on a production (non-development) system it is suggested you test it and fully understand what it is doing not just what it appears it is doing. I accept no responsibility for any damage you may do with this code.

Related