Not able to upload file >700MB file even after changing values in php.ini file
|
Praba
Hello,
Just started to work on Fossology tool last few days. I am trying to upload linux5.4.30 component which is around 1.7G and not able to upload the file even after changing “post_max_size” & “upload_max_filesize”. I was able to upload and scan other components less than 700MB.
I have Fossology installed using docker on R640 Dell server. FOSSology version 3.8.0-86-gf0c84e229 (code revision f0c84e)
Below are the details. Please provide details if any modifications are needed.
root@abbf7f77a767:/fossology# cat /etc/php/7.0/apache2/php.ini|grep memory_limit memory_limit = 1002M
root@abbf7f77a767:/fossology# cat /etc/php/7.0/apache2/php.ini|grep post_max_size post_max_size = 4001M
root@abbf7f77a767:/fossology# cat /etc/php/7.0/apache2/php.ini|grep upload_max_filesize upload_max_filesize = 3000M
Also made some tunable and config changes to few files. /etc/sysctl.conf kernel.shmmax=67328311296 kernel.shmall=16437576
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep max_connections max_connections = 100 # (change requires restart)
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep shared_buffers #shared_buffers = 128MB # min 128kB shared_buffers = 32GB # min 128kB #wal_buffers = -1 # min 32kB, -1 sets based on shared_buffers
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep effective_cache_size #effective_cache_size = 4GB effective_cache_size = 10GB
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep work_mem #work_mem = 4MB # min 64kB work_mem = 2GB # min 64kB #maintenance_work_mem = 64MB # min 1MB maintenance_work_mem = 6GB # min 1MB #autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep maintenance_work_mem #maintenance_work_mem = 64MB # min 1MB maintenance_work_mem = 6GB # min 1MB #autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep fsync #fsync = on # flush data to disk for crash safety fsync = on # flush data to disk for crash safety #wal_sync_method = fsync # the default is the first option # fsync # fsync_writethrough #data_sync_retry = off # retry or panic on failure to fsync
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep full_page_writes #full_page_writes = on # recover from partial page writes full_page_writes = off # recover from partial page writes
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep autovacuum #autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem #autovacuum = on # Enable autovacuum subprocess? 'on' autovacuum = on # Enable autovacuum subprocess? 'on' #log_autovacuum_min_duration = -1 # -1 disables, 0 logs all actions and ..
Thanks!
|
|
|
|
Michael C. Jaeger
hello,
toggle quoted messageShow quoted text
ja a quick question: if the uploa dis 1.7Gb and the max mem limit is 1.0 GB then this would be the problem. Pls. try setting the mem limit only few GBs above post max size. (something like 2.2 GB, 2.1 GB and and 2.0 GB. And you re right postgresql needs different setting for uploads of 1.7Gb size. I am not sure about work meme of 2GB 8could be also Ok with 256MB), maybe you re not going to use 6GB of maintenance mem. but a good approach is use a tool at first to get you reaosnable settings for the machine youre using: https://pgtune.leopard.in.ua/#/ was a good help for me in the past for tuning postgresql. Please note that you re using this link at your own risk. Kind regards, Michael
On 21. Jul 2020, at 01:12, Praba <@dpraba> wrote:
|
|
|