{"issue":{"state":"open","created_at":"2011-11-29T19:42:47Z","updated_at":"2011-11-29T19:42:50Z","user":{"gravatar_id":"9994df352157ae1b778b792744bdbf05","id":1073273,"url":"https://api.github.com/users/aschempp","avatar_url":"https://secure.gravatar.com/avatar/9994df352157ae1b778b792744bdbf05?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png","login":"aschempp"},"comments":3,"body":"Today I came to a very strange problem with the search index. system/libraries/Search.php on about Line 105 has two while-loops to remove unwanted javascript and non-indexing stuff. However, in some special cases, the while loops never ends. In my particular case, it was the first loop never stopping. After searching for more than two hours, I found the reason and a solution to the problem.\n\n```\n// Strip JavaScript (thanks to Pieter Dreef)\nwhile (preg_match('/<script[^>]*>/i', $strContent, $arrOuter, PREG_OFFSET_CAPTURE))\n{\n\tif (\n\n![](preg_match('/<\\/script>/i', $strContent, $arrInner, PREG_OFFSET_CAPTURE, (strlen($arrOuter[0][0]) + $arrOuter[0][1])))\n\t{\n\t\tbreak;\n\t}\n\n\t$strContent = substr($strContent, 0, $arrOuter[0][1]) . substr($strContent, (strlen($arrInner[0][0]) + $arrInner[0][1]));\n}\n\n// Strip non-indexable areas (thanks to Pieter Dreef)\nwhile (preg_match('/<)-- indexer::stop -->/', $strContent, $arrOuter, PREG_OFFSET_CAPTURE))\n{\n\tif (\n\n![](preg_match('/<)-- indexer::continue -->/', $strContent, $arrInner, PREG_OFFSET_CAPTURE, (strlen($arrOuter[0][0]) + $arrOuter[0][1])))\n\t{\n\t\tbreak;\n\t}\n\n\t$strContent = substr($strContent, 0, $arrOuter[0][1]) . substr($strContent, (strlen($arrInner[0][0]) + $arrInner[0][1]));\n}\n```\n\nThe problem is, PREG_OFFSET_CAPTURE is not binary safe, and therefore returning wrong positions. In my case, there were so much unicode characters, the <script> tag never got deleted cause of the offset. My solution to replace those 2 routines could even be simpler and faster, but at least it works:\n\n```\n$matches = preg_split('/(<\\/?script[^>]*>)|(<!-- indexer::(stop|continue) -->)/i', $strContent);\n$strContent = '';\n\nfor($_rit=0, $total=count($matches); $_rit<$total; $_rit=$_rit+2)\n{\n\t$strContent .= $matches[$_rit];\n}\n```\n\nBasically, split the content with <script> or --indexer-- tags and only keep every second part. If will break the content if a closing tag is missing, but that's not a problem we need to solve...\n\nWhat I did not understand, as far as I know, substr() is not binary safe either. However, I remember it should become binary safe in PHP6, and there's possibly already a compile option to do so, which could cause this problem.\n\n[Download the attachments](http://www.contao.org/tl_files/tickets/2520.zip \"Download the attachments of this ticket\")\nRelated issues: #546, #2949\n\n--- _Originally created on September 9th, 2010, at 08:59pm (ID 2520)_","assignee":null,"id":2391232,"number":2520,"pull_request":{"patch_url":null,"html_url":null,"diff_url":null},"labels":[{"name":"Defect","color":"bd2c00","url":"https://api.github.com/repos/contao/import/labels/Defect"},{"name":"framework","color":"e6e6e6","url":"https://api.github.com/repos/contao/import/labels/framework"},{"name":"Invalid","color":"ffdd00","url":"https://api.github.com/repos/contao/import/labels/Invalid"}],"title":"Endless loop in search index","closed_at":null,"html_url":"https://github.com/contao/import/issues/2520","url":"https://api.github.com/repos/contao/import/issues/2520","milestone":null},"action":"created","comment":{"created_at":"2011-11-29T19:42:50Z","updated_at":"2011-11-29T19:42:50Z","user":{"gravatar_id":"9994df352157ae1b778b792744bdbf05","id":1073273,"url":"https://api.github.com/users/aschempp","avatar_url":"https://secure.gravatar.com/avatar/9994df352157ae1b778b792744bdbf05?d=https://a248.e.akamai.net/assets.github.com%2Fimages%2Fgravatars%2Fgravatar-140.png","login":"aschempp"},"body":"I see. Probably not all servers allow to ini_set() a higher backtrack_limit, right?\nThe issue should be easily reproducible when you have a lot of umlauts in your text. In my case, the mootools scripts were successfully replaces (because they appear before any umlaut), but the number of umlauts (or the byte count of them) was higher than the string length of the moo_mediabox template.\n\n--- _Originally created on September 13th, 2010, at 02:46pm_","id":2944080,"url":"https://api.github.com/repos/contao/import/issues/comments/2944080"}}