Docker Container Restart Problems
flp@...
Hey,
I try to run fossology as docker compose cluster. Unfortunately, it always crashes when I need to restart the fossology container (3.7.0, 3.8.1 or 3.9.0-rc2). I end up with the following error in 3.8.1 und 3.9.0: *** Running postinstall for common actions***
*** Creating user and group ***
NOTE: group 'fossy' already exists, good.
NOTE: user 'fossy' already exists, good.
*** Making sure needed dirs exist with right ownership/permissions ***
*** clearing file cache ***
NOTE: Repository already exists at /srv/fossology/repository
NOTE: Running the PostgreSQL vacuum and analyze command can result in a large database performance improvement.
We suggest that you either configure postgres to run its autovacuum and autoanalyze daemons, or maintagent -D in a cron job, or run Admin > Maintenance on a regular basis.
Admin > Dashboard will show you the last time vacuum and analyze have been run.
NOTE: using external DB on host foss-db
*** update the database and license_ref table ***
PHP Warning: syntax error, unexpected ')' in /usr/local/etc/fossology/Db.conf on line 4
in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 96
PHP Warning: Invalid argument supplied for foreach() in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 101
PHP Notice: Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
Old release was 3.0.1
*** Cleaning tables for new constraints ***
Table author or agent does not exists, not cleaning!
Table author or pfile does not exists, not cleaning!
Table bucket_container or bucket_def does not exists, not cleaning!
Table bucket_file or bucket_def does not exists, not cleaning!
Table bucket_file or pfile does not exists, not cleaning!
Table copyright or agent does not exists, not cleaning!
Table copyright_decision or pfile does not exists, not cleaning!
Table ecc or agent does not exists, not cleaning!
Table ecc or pfile does not exists, not cleaning!
Table ecc_decision or pfile does not exists, not cleaning!
Table highlight_keyword or pfile does not exists, not cleaning!
Table keyword or agent does not exists, not cleaning!
Table keyword or pfile does not exists, not cleaning!
Table keyword_decision or pfile does not exists, not cleaning!
Table pkg_deb_req or pkg_deb does not exists, not cleaning!
Table pkg_rpm_req or pkg_rpm does not exists, not cleaning!
Table report_cache or upload does not exists, not cleaning!
Table report_info or upload does not exists, not cleaning!
Table reportgen or upload does not exists, not cleaning!
Table upload or pfile does not exists, not cleaning!
Table upload_clearing_license or upload does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table report_info does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table group_user_member does not exists, not cleaning!
Removed 0 rows from tables with new constraints
PHP Notice: Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
PHP Warning: pg_query(): Query failed: ERROR: column "rf_copyleft" of relation "license_ref" already exists in /usr/local/share/fossology/lib/php/Db/Driver/Postgres.php on line 79
[2020-10-18 21:38:19] /usr/local/share/fossology/lib/php/libschema.php.CRITICAL: ERROR: column "rf_copyleft" of relation "license_ref" already exists [] []
PHP Fatal error: Uncaught Fossology\Lib\Exception: error executing: ALTER TABLE "license_ref" ADD COLUMN "rf_copyleft" bit(1)
ERROR: column "rf_copyleft" of relation "license_ref" already exists in /usr/local/share/fossology/lib/php/Db/DbManager.php:144
Stack trace:
#0 /usr/local/share/fossology/lib/php/Db/DbManager.php(203): Fossology\Lib\Db\DbManager->checkResult(false, 'ALTER TABLE "li...')
#1 /usr/local/share/fossology/lib/php/libschema.php(299): Fossology\Lib\Db\DbManager->queryOnce('ALTER TABLE "li...')
#2 /usr/local/share/fossology/lib/php/libschema.php(161): fo_libschema->applyTables()
#3 /usr/local/lib/fossology/fossinit.php(196): fo_libschema->applySchema('/usr/local/shar...', false, 'fossology', Array)
#4 {main}
thrown in /usr/local/share/fossology/lib/php/Db/DbManager.php on line 144
With 3.7.0 I get the following: *** Running postinstall for common actions***
*** Creating user and group ***
NOTE: group 'fossy' already exists, good.
NOTE: user 'fossy' already exists, good.
*** Making sure needed dirs exist with right ownership/permissions ***
*** clearing file cache ***
NOTE: Repository already exists at /srv/fossology/repository
NOTE: Running the PostgreSQL vacuum and analyze command can result in a large database performance improvement.
We suggest that you either configure postgres to run its autovacuum and autoanalyze daemons, or maintagent -D in a cron job, or run Admin > Maintenance on a regular basis.
Admin > Dashboard will show you the last time vacuum and analyze have been run.
NOTE: using external DB on host foss-db
*** update the database and license_ref table ***
PHP Warning: syntax error, unexpected ')' in /usr/local/etc/fossology/Db.conf on line 4
in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 96
PHP Warning: Invalid argument supplied for foreach() in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 101
PHP Notice: Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
Old release was 0
*** Cleaning tables for new constraints ***
Table author or agent does not exists, not cleaning!
Table author or pfile does not exists, not cleaning!
Table bucket_container or bucket_def does not exists, not cleaning!
Table bucket_file or bucket_def does not exists, not cleaning!
Table bucket_file or pfile does not exists, not cleaning!
Table copyright or agent does not exists, not cleaning!
Table copyright_decision or pfile does not exists, not cleaning!
Table ecc or agent does not exists, not cleaning!
Table ecc or pfile does not exists, not cleaning!
Table ecc_decision or pfile does not exists, not cleaning!
Table highlight_keyword or pfile does not exists, not cleaning!
Table keyword or agent does not exists, not cleaning!
Table keyword or pfile does not exists, not cleaning!
Table keyword_decision or pfile does not exists, not cleaning!
Table pkg_deb_req or pkg_deb does not exists, not cleaning!
Table pkg_rpm_req or pkg_rpm does not exists, not cleaning!
Table report_cache or upload does not exists, not cleaning!
Table report_info or upload does not exists, not cleaning!
Table reportgen or upload does not exists, not cleaning!
Table upload or pfile does not exists, not cleaning!
Table upload_clearing_license or upload does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table report_info does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table group_user_member does not exists, not cleaning!
Removed 0 rows from tables with new constraints
PHP Notice: Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
PHP Warning: pg_query(): Query failed: ERROR: column "option_value" of relation "sysconfig" already exists in /usr/local/share/fossology/lib/php/Db/Driver/Postgres.php on line 79
[2020-10-22 21:49:47] /usr/local/share/fossology/lib/php/libschema.php.CRITICAL: ERROR: column "option_value" of relation "sysconfig" already exists [] []
PHP Fatal error: Uncaught Fossology\Lib\Exception: error executing: ALTER TABLE "sysconfig" ADD COLUMN "option_value" character varying(40) DEFAULT NULL
ERROR: column "option_value" of relation "sysconfig" already exists in /usr/local/share/fossology/lib/php/Db/DbManager.php:144
Stack trace:
#0 /usr/local/share/fossology/lib/php/Db/DbManager.php(203): Fossology\Lib\Db\DbManager->checkResult(false, 'ALTER TABLE "sy...')
#1 /usr/local/share/fossology/lib/php/libschema.php(299): Fossology\Lib\Db\DbManager->queryOnce('ALTER TABLE "sy...')
#2 /usr/local/share/fossology/lib/php/libschema.php(161): fo_libschema->applyTables()
#3 /usr/local/lib/fossology/fossinit.php(191): fo_libschema->applySchema('/usr/local/shar...', false, 'fossology', Array)
#4 {main} thrown in /usr/local/share/fossology/lib/php/Db/DbManager.php on line 144 I'm using fossology with the following compose cluster: version: '3.8' services: foss: image: fossology/fossology:3.9.0-rc2 restart: unless-stopped environment: FOSSOLOGY_DB_HOST: FOSSOLOGY_DB_NAME: FOSSOLOGY_DB_USER: FOSSOLOGY_DB_PASSWORD: volumes: - ./foss-repo:/srv/fossology/repository/ networks: - foss depends_on: - foss-db foss-db: image: postgres:11 restart: unless-stopped environment: POSTGRES_DB: POSTGRES_USER: POSTGRES_PASSWORD: POSTGRES_INITDB_ARGS: '-E SQL_ASCII' volumes: - ./postgres-data:/var/lib/postgresql/data networks: - foss networks: foss: external: name: foss I have no particular experience with Php but it looks like the startup script is always executed within the docker container so I might need to create my own one that does not execute the database script at startup?
|
|
End of Google Summer of Code - 2020
Shaheem Azmal M MD
Hello Everyone,
We would like to congratulate Darshan, Kaushlendra and Ayush for their contributions to FOSSology community. We are very impressed by the overall quality of their work and organizational skills, keep up the good work and stay in touch with community.
Details of features that were developed in this GSOC-2020.
Project : FOSSology Student Name : Darshan Kansagara Feature Title : Dashboard (Python, influxDB, Grafana) Wiki : https://github.com/darshank15/GSoC_2020_FOSSOlogy/wiki
Project : Atarashi Student Name : Ayush (HastagAB) Feature Title : Accelerating Atarashi Wiki : https://github.com/hastagAB/GSoC-2020
Project : Atarashi Student Name : Kaushlendra Pratap Singh Feature Title : Refurbishing Atarashi Wiki : https://github.com/Kaushl2208/GSoC-2020
We would also like to thank mentors(Michael, Shaheem, Anupam, Gaurav, Nicolas, Aman and Sandeep) for helping students achieve their targets.
Best Regards Shaheem Azmal M MD
|
|
Re: Not able to upload file >700MB file even after changing values in php.ini file
Michael C. Jaeger
hello,
toggle quoted messageShow quoted text
ja a quick question: if the uploa dis 1.7Gb and the max mem limit is 1.0 GB then this would be the problem. Pls. try setting the mem limit only few GBs above post max size. (something like 2.2 GB, 2.1 GB and and 2.0 GB. And you re right postgresql needs different setting for uploads of 1.7Gb size. I am not sure about work meme of 2GB 8could be also Ok with 256MB), maybe you re not going to use 6GB of maintenance mem. but a good approach is use a tool at first to get you reaosnable settings for the machine youre using: https://pgtune.leopard.in.ua/#/ was a good help for me in the past for tuning postgresql. Please note that you re using this link at your own risk. Kind regards, Michael
On 21. Jul 2020, at 01:12, Praba <dmuthumani@marvell.com> wrote:
|
|
Not able to upload file >700MB file even after changing values in php.ini file
Praba
Hello,
Just started to work on Fossology tool last few days. I am trying to upload linux5.4.30 component which is around 1.7G and not able to upload the file even after changing “post_max_size” & “upload_max_filesize”. I was able to upload and scan other components less than 700MB.
I have Fossology installed using docker on R640 Dell server. FOSSology version 3.8.0-86-gf0c84e229 (code revision f0c84e)
Below are the details. Please provide details if any modifications are needed.
root@abbf7f77a767:/fossology# cat /etc/php/7.0/apache2/php.ini|grep memory_limit memory_limit = 1002M
root@abbf7f77a767:/fossology# cat /etc/php/7.0/apache2/php.ini|grep post_max_size post_max_size = 4001M
root@abbf7f77a767:/fossology# cat /etc/php/7.0/apache2/php.ini|grep upload_max_filesize upload_max_filesize = 3000M
Also made some tunable and config changes to few files. /etc/sysctl.conf kernel.shmmax=67328311296 kernel.shmall=16437576
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep max_connections max_connections = 100 # (change requires restart)
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep shared_buffers #shared_buffers = 128MB # min 128kB shared_buffers = 32GB # min 128kB #wal_buffers = -1 # min 32kB, -1 sets based on shared_buffers
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep effective_cache_size #effective_cache_size = 4GB effective_cache_size = 10GB
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep work_mem #work_mem = 4MB # min 64kB work_mem = 2GB # min 64kB #maintenance_work_mem = 64MB # min 1MB maintenance_work_mem = 6GB # min 1MB #autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep maintenance_work_mem #maintenance_work_mem = 64MB # min 1MB maintenance_work_mem = 6GB # min 1MB #autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep fsync #fsync = on # flush data to disk for crash safety fsync = on # flush data to disk for crash safety #wal_sync_method = fsync # the default is the first option # fsync # fsync_writethrough #data_sync_retry = off # retry or panic on failure to fsync
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep full_page_writes #full_page_writes = on # recover from partial page writes full_page_writes = off # recover from partial page writes
root@abbf7f77a767:/fossology# cat /etc/postgresql//9.6/main/postgresql.conf |grep autovacuum #autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem #autovacuum = on # Enable autovacuum subprocess? 'on' autovacuum = on # Enable autovacuum subprocess? 'on' #log_autovacuum_min_duration = -1 # -1 disables, 0 logs all actions and ..
Thanks!
|
|
MIT License
Ashkar Dev
Hi, This question is about OpenSource License I am on developing a project and used MIT Licensed codes of others and I collected the Licenses but now I'm not sure which code is with which license and also the code was may be modified is it ok to add all licenses to a place on a web page? instead of making it comment? in every page footer, make a link to the page of licenses? and adding the link to that page at the top of CSS files so it matches the MIT License condition that says substantial portions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. and I want to create a page with licenses and references to the original code like this: this project may include these licenses : 1) --- with reference 2) --- with reference ......
|
|
Fossology license manager
surendra bhargava
Hi, I have come across that fossology can be used as a license manager. How is fossology helpful as a license manager.It is only a compliance tool that scans and reports the detected licenses, is it? Thanks. Surendra
|
|
fossology license manager
surendra bhargava
I have come across that fossology can be used as a license manager.
How is fossology helpful as a license manager.It is only a compliance tool that scans and reports the detected licenses, is it?
|
|
FOSSology 3.7.0 release
Shaheem Azmal M MD
Hello everyone,
We have released FOSSology 3.7.0 yesterday. As for the release 3.7.0 of FOSSology, most notable additions since are:
ContributorsCredits go to the following persons for this release since 3.7.0-RC1:
Please find the release and binary packages for Debian and Ubuntu based systems on release page.
|
|
Fossology REST Clients for .NET
Graf, Thomas
Dear all,
I like to inform you, that I have added a .NET implementation of the FOSSology REST API to the FOSSology GitHub project: https://github.com/fossology/FOSSology.REST.dotnet
The implementation covers most of the REST commands. The unit tests will run on FOSSology in a docker container. Our goal is to add one further step to the automation of software license compliance.
Do not hesitate to drop me an email in case of questions or suggestions.
Best regards,
|
|
Google Summer of Code final report, Integration of Software Heritage in FOSSology
Sandip Bhuyan
Hello Everyone, I am Sandip Kumar Bhuyan was a Google Summer of Code 2019 intern for fossology organization. Thank you for selecting my proposal for this year GSoC. I think I have reached the expectation. I was working on integrating software heritage in fossology. I have completed the GSoC 2019 successfully. You can find my report in my blogpost Blog Post. I hope everyone will like it. Cheers Sandip
|
|
FOSSology 3.6.0 release
Shaheem Azmal M MD
Hello everyone, After two release candidates, making fixes for migration tests, unified report and load issues with tree-view, FOSSology is stable enough for a new release. The main features of the 3.6.0 release can be found under [RC1](https://github.com/fossology/fossology/releases/tag/3.6.0-rc1). Particular corrections after RC1 can be found under [RC2](https://github.com/fossology/fossology/releases/tag/3.6.0-rc2).
Few interesting features in this release are:
Credits to 3.6.0
From the git commit history, we have following contributors since 3.5.0:
> @andi8086 <andreas.reichel@...>, > @ag4ums <anupam.ghosh@...>, > @hastagAB <classicayush@...>, > @chienphamvu <chienphamvu@...>, > @ChristopheRequillart <christophe.requillart@...>, > @GMishx <mishra.gaurav@...>, > @maxhbr <maximilian.huber@...>, > @mcjaeger <michael.c.jaeger@...>, > @NicolasToussaint <nicolas1.toussaint@...>, > @PeterDaveHello <hsu@...>, > @rlintu <raino.lintulampi@...>, > @sandipbhuyan <sandipbhuyan@...>, > @shaheemazmalmmd <shaheem.azmal@...>, > @tiegz <tieg@...>, > @vivekaindia <vvksindia@...>
|
|
GSoC 19 - Spasht Agent Documentation
vivek kumar
Please Check Spasht Documentation @
or Regards Vivek ![]()
|
|
Need to remove Debian packaging meta info from master branch
Hello all,
During our effort to publish FOSSology as a Debian package, we got few suggestions from the Debian community. One of those suggestion is to remove the Debian packaging information (debian folder) from the master branch and put it into another branch like chore/debian/jessie.
This is done so to avoid conflicts as Debian maintainers will be editing this packaging information in the FOSSology mirror (hosted at Debian Sala). And any change in upstream can result in conflicts.
As this change will alter the packaging steps required by many of FOSSology users, we need your feedback.
I have opened an issue on GitHub for the same: https://github.com/fossology/fossology/issues/1341
Kindly respond either on this thread or on the GitHub issue if you have any concerns regarding the same. With best regards,
|
|
Release of 3.5.0
Shaheem Azmal M MD
Hello all, Particular corrections after RC1 can be found under RC2. Mainly 3.5.0 adds more documentation, infrastructure improvements and support for brand new FOSSology REST API. A brief introduction about the REST API can be found at: https://www.fossology.org/get-started/basic-rest-api-calls/ Moreover, new functionality has improved JSON output for nomos and restructured license detection for nomos. Last but not the least, FOSSology now have capabilities to ignore files specific to version control systems from the scanning improving scan times. CreditsFrom the git commit history, we have following contributors since 3.4.0:
Please find the release and binary packages for Debian and Ubuntu based systems here https://github.com/fossology/fossology/releases/tag/3.5.0 Thanks & Regards Shaheem Azmal M MD
|
|
Re: Is there any plans to use ALL features of SPDX 2.1 specification in future releases?
Michael C. Jaeger
Hello,
toggle quoted messageShow quoted text
currently, I am not aware of the FOSSology project planning to support "ALL features of SPDX 2.1 specification in future releases", for a number of reasons, just a few examples: * Spec 2.1 supports identification of code snippets (See section 5), currently fossology does not support it and I am not aware of plans by someone to contribute it * Spec 2.1 support for example besides the copyright statement also a file contributor, which could be taken, maybe, from some SCM information. I am not aware of plans here neither * ... and so forth. I think FOSSology will support only parts of the SPDX 2.1 spec as they are covered by the application functionality. I am not sure of that answer covers your question? The fact that you have used bold types letters in your e-mail provides a slight impression that you expect something in particular from the FOSSology project? Maybe the following two issues provide also helpful information to you? https://github.com/fossology/fossology/issues/1309 https://github.com/spdx/spdx-spec/issues/112 Please do not hesitate to clarify what you intended to say about our release notes or our idea of SPDX 2.1 document generation. Kind regards, Michael
On 12. Mar 2019, at 13:30, thuy.tran.xh@renesas.com wrote:
|
|
Is there any plans to use ALL features of SPDX 2.1 specification in future releases?
thuy.tran.xh@...
Hi all, Key features in FOSSology 3.1 are: I have tried the fossology installation in the ways of docker as well as from source. Regards,
|
|
Re: VS: [FOSSology] Help a newbie
marc.mcgarry@...
On Fri, Oct 12, 2018 at 08:05 AM, Martin von Willebrand wrote:
|
|
VS: [FOSSology] Help a newbie
Martin von Willebrand
Fossology agent doing the unpacking will unpack zip-packages and jar-packages, even if they are nested.
You likely want to run source code files through Fossology, though, because many build systems remove texts that would be interesting for license analysis purposes. In java, you should look at uploading .java-files and not .class-files.
Best Martin
Martin von Willebrand, Attorney-at-law, Partner HH Partners shines in international rankings. See details at hhpartners.fi. Privileged and confidential information may be contained in this message. If you are not addressee indicated in this message (or responsible for delivery of the message to such person), you may not copy or deliver this message to anyone. In such case, kindly notify us by reply e-mail and delete this message immediately. Thank you.
Lähettäjä: main@... [mailto:main@...]
Puolesta marc.mcgarry@...
I am trying to run software packages/components through fossology. If i zip the package folder (with the jar file inside), will this successfully test all of the classes inside?
|
|
Re: Help a newbie
Michael C. Jaeger
Hello,
oh interesting, I have not seen your point from your answer, glad that our e-mails crossed.
Kind regards, Michael
From: main@... [mailto:main@...]
On Behalf Of Nicolas Toussaint
Sent: Freitag, 12. Oktober 2018 17:03 To: main@... Subject: Re: [FOSSology] Help a newbie
Hi Marc, On 12/10/2018 16:56, marc.mcgarry@... wrote:
-- Nicolas Toussaint OAB - Orange Applications for Business - Lyon _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
|
|
Re: Help a newbie
Michael C. Jaeger
Hello,
your mail raises a few questions, I try to bring a few considerations.
1. „classes“ refers to *.class files? if yes, then this is maybe not the right thing for FOSSology, but maybe for some binary analysis tool (- next generation) or so. class files are usually results of a compiler so they omit source code comments which is where licensing statements are usually in.
2. FOSSology should be able to unpack *.jar files, so no need to zip it, except you would like to upload a set of jar files at once. If fossology cannot look into a jar file, it is a bug (and should be thus put in the issues tracker).
3. what is your point with “test” what are look for exactly?
Kind regards, Michael
From: main@... [mailto:main@...]
On Behalf Of marc.mcgarry@...
Sent: Freitag, 12. Oktober 2018 16:57 To: main@... Subject: [FOSSology] Help a newbie
I am trying to run software packages/components through fossology. If i zip the package folder (with the jar file inside), will this successfully test all of the classes inside?
|
|
21 - 40 of 59 |