unpack(C*,data) Eats Memory [php exhausting memory on unpack function] -


i want read binary file byte[] in php , suggested here unpack'ed fread's output . have like:

$file=fopen($filename,'r'); fseek($file, $offset);  //file 500mb take 10mb @ time $tmp = fread($file,$len); //so far , $tmp includes 10mb of data var_dump(strlen($tmp));    //int(10485760) 10mb var_dump(memory_get_usage(true)); //int(11272192) 11mb $data = unpack('c*',$tmp); 

this throws

php fatal error:  allowed memory size of 536870912 bytes exhausted (tried allocate 32 bytes) in [myfile.php] on line [unpack line] 

as error suggests memory limit set @ 512mb , according memory_get_usage 11/512mb used , unpacking 10mb string . @ should need 30mb (10mb $tmp , 10mb $data , 10mb internal variables). why explode , can't unpack $tmp 512mb ram ?

so question is , doing wrong here or bug? , there other way array of bytes (0 255) read binary files in php or should switch language this?

additional notes : code works 117kb file.

php -v php 5.5.3-1ubuntu2.2 (cli) (built: feb 28 2014 20:06:05)  copyright (c) 1997-2013 php group zend engine v2.5.0, copyright (c) 1998-2013 zend technologies     zend opcache v7.0.3-dev, copyright (c) 1999-2013, zend technologies 

in php variables stored internally zvals. each element in array take more memory expect. due php being weakly typed language , therefore requiring ability swap type of variable internally. there overhead of gc , fact array in php hash table.

you can find in-depth details here:

http://nikic.github.io/2011/12/12/how-big-are-php-arrays-really-hint-big.html

however, create array of 10485760 elements require approx. 760mb on 32bit , 1440mb on 64bit.

your best option not unpack string , instead when require element in array access position in string.

for example library use uses concept:

https://github.com/reiner-dolp/php-memory-efficient-arrays


Comments

Popular posts from this blog

PHPMotion implementation - URL based videos (Hosted on separate location) -

javascript - Using Windows Media Player as video fallback for video tag -

c# - Unity IoC Lifetime per HttpRequest for UserStore -