Date   

FOSSology 3.11.0 Release Candidate 1

Shaheem Azmal M MD
 

Hello everyone,

We have released FOSSology 3.11.0-rc1 today.

This release adds important corrections to 3.10.0
 
The release  3.11.0-rc1 introduces following major changes since 3.10.0:
 
  • Add bulk undo for deactivated copyrights.
  • Configurable irrelevant file scan for monkbulk.
  • Add job to remove expired tokens from database.
  • Add a simple search to get folder.
  • Unit test cases for REST API.
  • Reuse edited copyright.
  • Add scroll to NOTICE file modal

Credits to contributors for 3.11.0-rc1 :
 
  • Aman Dwivedi <aman.dwivedi5@...>
  • Anupam Ghosh <anupam.ghosh@...>
  • Anwar Hashmi @HashmiAS
  • bighnesh0404 <saibighneshprusty@...>
  • Gaurav Mishra <mishra.gaurav@...>
  • OmarAbdelSamea <1700903@...>
  • R3da <hash.rkh@...>
  • Rolf Eike Beer <eb@...>
  • Shaheem Azmal M MD <shaheem.azmal@...>
  • shivamgoyal7 <goyalshivam661@...>
  • Shruti3004 <mail2shruti.ag@...>
  • Wonjae Park <wonjae.park@...>
  • Ying-Chun Liu (PaulLiu) <paulliu@...>
 
For complete list of commits, please check 3.11.0-rc1.

We would like to thank you all for your support for this release.

Best Regards
FOSSology Team
 


FOSSology 3.10.0 release

Shaheem Azmal M MD
 

Hello everyone,

We have released FOSSology 3.10.0 today.

This release adds important corrections to 3.10.0-rc2
 
The release 3.10.0 introduces following major changes since 3.9.0:
 
* Change copyright handling add new table `copyright_event`.
* Drop support for PHP5 and update dependencies for PHP7
* Update password hashing algorithm from SHA1 to more secure bcrypt.
* Advance search and replace for copyrights.
* Ability to enforce password policies (under Admin > Customize).
* Feature to import license acknowledgement from NOTICE file.
* Read XML in chunks to support large files for ReportImport.
* Add license search based on short name in REST.
* Ability to search file from hash values in REST API.
* Do not add decisions if the events have no change.
* Migrate github pages deployment to GHA.

NOTE: This release also adds a migration script which migrates copyright data to new table copyright_event.
      Migration processes is mandatory because without migration, old copyright activation/deactivation may not work.
      Also, it approximately takes 15 min for 1M records.
 
#### Credits to contributors for 3.10.0
 
From the GIT commit history, we have following contributors since [3.9.0](https://github.com/fossology/fossology/releases/tag/3.9.0):
 
Alan Hohn <Alan.M.Hohn@...>
Aman Dwivedi <aman.dwivedi5@...>
Andreas J. Reichel <andreas.reichel@...>
Anupam Ghosh <anupam.ghosh@...>
Avinal Kumar <avinal.xlvii@...>
Bartłomiej Dróżdż <bartlomiej.drozdz@...>
Darshan <kansagara.darshan97@...>
David Lechner <david@...>
Dineshkumar Devarajan (RBEI/BSF6) <Devarajan.Dineshkumar@...>
Gaurav Mishra <mishra.gaurav@...>
Helio Chissini de Castro <helio@...>
Mikko Murto <mikko.murto@...>
Michael C.Jaeger <michael.c.jaeger@...>
Pawan Kumar Meena <Pawank1804@...>
Piotr Pszczola <piotr.pszczola@...>
rlintu <raino.lintulampi@...>
Sahil <sjha200000@...>
Shaheem Azmal M MD <shaheem.azmal@...>
Shruti3004 <mail2shruti.ag@...>
Toussaint Nicolas <nicolas1.toussaint@...>
YashJipkate <yashjipkate@...>
 
For complete list of commits, please check 3.10.0-rc1 and 3.10.0-rc2.

We would like to thank you all for your support for this release.

Best Regards
FOSSology Team


Re: EXTERNAL: Re: [FOSSology] Unpacking .msi Files - Fossology Version 3.8.1

D Frye
 

-----Original Message-----
From: main@lists.fossology.org <main@lists.fossology.org> On Behalf Of Michael C. Jaeger
Sent: Thursday, December 31, 2020 6:09 AM
To: main@lists.fossology.org
Subject: EXTERNAL: Re: [FOSSology] Unpacking .msi Files - Fossology Version 3.8.1

Hi,

please help us a little bit with an example file - an URL to it would be great. I do not have a an *.msi file handy right now and I am not sure if it will be better if you could share the file you re having problems with?

If you feel that is too much of information revealing to the public, you could send the URL to donwload the file(s) in question directly to my e-mail.

Kind regards and if you do not respond earlier, have good start into 2021!

- Michael

On 30. Dec 2020, at 16:36, D Frye <deborah.l.frye@lmco.com> wrote:

Everyone was so helpful with the issue of unpacking .deb files in the newer version of Fossology. It looks like there’s also an issue with unpacking .msi files. Has anyone identified a fix for these?

Please let me know.

Thanks – Deb Frye

From: main@lists.fossology.org <main@lists.fossology.org> On Behalf Of
Frye, Deborah L (US)
Sent: Monday, November 9, 2020 9:19 AM
To: main@lists.fossology.org
Subject: EXTERNAL: Re: [FOSSology] Unpacking Debian Files - Fossology
Version 3.8.1

Thanks for the quick find and fix.

Yes, we did change the base system on which Fossology is running. Our Fossology 3.8.1 instance is deployed to a separate server running Ubuntu 18.04 LTS.

Based on your diagnosis, as a workaround we tried changing /etc/mime.types to point 'deb' files back to the previous MIME type of "application/x-debian-package" but still saw the same behavior. Next we will try a custom compiled version that applies your patch.

Thank you for your prompt response!

Deb

From: main@lists.fossology.org <main@lists.fossology.org> On Behalf Of
Gaurav Mishra
Sent: Monday, November 9, 2020 4:24 AM
To: main@lists.fossology.org
Subject: EXTERNAL: Re: [FOSSology] Unpacking Debian Files - Fossology
Version 3.8.1

Hello Frye,

From what I understood with inconsistency in the unpacking, the deb files were not extracted at all and I tried it on my machine and it didn't work.

It seems like the mimetype of .deb archives was changed (only official document I could find is this https://www.iana.org/assignments/media-types/application/vnd.debian.binary-package).
I have created a PR with the new mimetype which you can find here:
https://github.com/fossology/fossology/pull/1831

Also, I was wondering did you changed the base system the fossology is running, as there was no change in the code base related to it?

With best regards,
Gaurav Mishra
From: main@lists.fossology.org <main@lists.fossology.org> on behalf of
D Frye <deborah.l.frye@lmco.com>
Sent: 05 November 2020 22:55
To: main@lists.fossology.org <main@lists.fossology.org>
Subject: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

Recently upgraded Fossology to Version 3.8.1. Previous version unpacked and scanned Debian files within FOSS packages; however, seems like new version is not consistently unpacking them. Has anyone else run into this?

Any help would be greatly appreciated.
Deb Frye

Lockheed Martin Corporation
Rotary and Mission Systems (RMS)
Software License Compliance / F-35 3PC Lead P.O. Box 4840, EP7-C182,
MD8 Syracuse, NY 13221-4840
(315) 456-2485
deborah.l.frye@lmco.com


Re: Unpacking .msi Files - Fossology Version 3.8.1

Michael C. Jaeger
 

Hi,

please help us a little bit with an example file - an URL to it would be great. I do not have a an *.msi file handy right now and I am not sure if it will be better if you could share the file you re having problems with?

If you feel that is too much of information revealing to the public, you could send the URL to donwload the file(s) in question directly to my e-mail.

Kind regards and if you do not respond earlier, have good start into 2021!

- Michael

On 30. Dec 2020, at 16:36, D Frye <deborah.l.frye@lmco.com> wrote:

Everyone was so helpful with the issue of unpacking .deb files in the newer version of Fossology. It looks like there’s also an issue with unpacking .msi files. Has anyone identified a fix for these?

Please let me know.

Thanks – Deb Frye

From: main@lists.fossology.org <main@lists.fossology.org> On Behalf Of Frye, Deborah L (US)
Sent: Monday, November 9, 2020 9:19 AM
To: main@lists.fossology.org
Subject: EXTERNAL: Re: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

Thanks for the quick find and fix.

Yes, we did change the base system on which Fossology is running. Our Fossology 3.8.1 instance is deployed to a separate server running Ubuntu 18.04 LTS.

Based on your diagnosis, as a workaround we tried changing /etc/mime.types to point 'deb' files back to the previous MIME type of "application/x-debian-package" but still saw the same behavior. Next we will try a custom compiled version that applies your patch.

Thank you for your prompt response!

Deb

From: main@lists.fossology.org <main@lists.fossology.org> On Behalf Of Gaurav Mishra
Sent: Monday, November 9, 2020 4:24 AM
To: main@lists.fossology.org
Subject: EXTERNAL: Re: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

Hello Frye,

From what I understood with inconsistency in the unpacking, the deb files were not extracted at all and I tried it on my machine and it didn't work.

It seems like the mimetype of .deb archives was changed (only official document I could find is this https://www.iana.org/assignments/media-types/application/vnd.debian.binary-package).
I have created a PR with the new mimetype which you can find here: https://github.com/fossology/fossology/pull/1831

Also, I was wondering did you changed the base system the fossology is running, as there was no change in the code base related to it?

With best regards,
Gaurav Mishra
From: main@lists.fossology.org <main@lists.fossology.org> on behalf of D Frye <deborah.l.frye@lmco.com>
Sent: 05 November 2020 22:55
To: main@lists.fossology.org <main@lists.fossology.org>
Subject: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

Recently upgraded Fossology to Version 3.8.1. Previous version unpacked and scanned Debian files within FOSS packages; however, seems like new version is not consistently unpacking them. Has anyone else run into this?

Any help would be greatly appreciated.
Deb Frye

Lockheed Martin Corporation
Rotary and Mission Systems (RMS)
Software License Compliance / F-35 3PC Lead
P.O. Box 4840, EP7-C182, MD8
Syracuse, NY 13221-4840
(315) 456-2485
deborah.l.frye@lmco.com


FW: Unpacking .msi Files - Fossology Version 3.8.1

D Frye
 

Everyone was so helpful with the issue of unpacking .deb files in the newer version of Fossology.  It looks like there’s also an issue with unpacking .msi files.  Has anyone identified a fix for these?

 

Please let me know.

 

Thanks – Deb Frye

 

From: main@... <main@...> On Behalf Of Frye, Deborah L (US)
Sent: Monday, November 9, 2020 9:19 AM
To: main@...
Subject: EXTERNAL: Re: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

 

Thanks for the quick find and fix.

 

Yes, we did change the base system on which Fossology is running. Our Fossology 3.8.1 instance is deployed to a separate server running Ubuntu 18.04 LTS.

 

Based on your diagnosis, as a workaround we tried changing /etc/mime.types to point 'deb' files back to the previous MIME type of "application/x-debian-package" but still saw the same behavior. Next we will try a custom compiled version that applies your patch.

 

Thank you for your prompt response!

 

Deb

 

From: main@... <main@...> On Behalf Of Gaurav Mishra
Sent: Monday, November 9, 2020 4:24 AM
To: main@...
Subject: EXTERNAL: Re: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

 

Hello Frye,

 

From what I understood with inconsistency in the unpacking, the deb files were not extracted at all and I tried it on my machine and it didn't work.

 

It seems like the mimetype of .deb archives was changed (only official document I could find is this https://www.iana.org/assignments/media-types/application/vnd.debian.binary-package).

I have created a PR with the new mimetype which you can find here: https://github.com/fossology/fossology/pull/1831

 

Also, I was wondering did you changed the base system the fossology is running, as there was no change in the code base related to it?

 

With best regards,

Gaurav Mishra


From: main@... <main@...> on behalf of D Frye <deborah.l.frye@...>
Sent: 05 November 2020 22:55
To: main@... <main@...>
Subject: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

 

Recently upgraded Fossology to Version 3.8.1.  Previous version unpacked and scanned Debian files within FOSS packages; however, seems like new version is not consistently unpacking them.  Has anyone else run into this?

 

Any help would be greatly appreciated.

Deb Frye

Lockheed Martin Corporation

Rotary and Mission Systems (RMS)

Software License Compliance / F-35 3PC Lead
P.O. Box 4840, EP7-C182, MD8
Syracuse, NY 13221-4840
(315) 456-2485
deborah.l.frye@...

 


Gsoc 2021

Akshay Sharma
 

Hey everyone ! Hope u are doing well ! My self Akshay and want to contribute in open source community .As i dont want to be in trapped in learning loops i want to implement what i learn by contributing in open source ...Can anyone here guide me how i start my contribution .


Re: Docker Container Restart Problems

flp@...
 

Just a short update. Checking out the repository and building the containers from source works. I think it should also work with the dockerhub container but I guess I'm missing some essential information to get that work since it's quite annoying to regularily pull the current code.


Re: GSoC 2021

Gaurav Mishra
 

Hello Lasal,

Nice to have you here with us! Under FOSSology, we have 3 projects with which we participate in GSoC. Hopefully we will be selected as an organization this year around as well by Google.

I'll give you a brief overview of the projects so you can explore more about them.
  1. The major project is FOSSology itself. https://github.com/fossology/fossology
    It is an open source license complain toolkit which helps you find out different licenses which are present in a given source code. With the help of a web based UI, users can then curate the results and generate the reports required for compliance.
  2. As a part of FOSSology, a new scanner called Atarashi. https://github.com/fossology/atarashi
    Atarashi uses various text statistic techniques like tf-idf, dld, n-grams, etc. to identify license related text in a source file. It is designed to work stand-alone and with FOSSology.
  3. This year, a new project was introduced called Nirjas. https://github.com/fossology/Nirjas
    It is a python library and a stand-alone tool which can extract comments from a source code, differentiate between multiline and single line comments, do a sloc count, etc.
For all the three projects, you can find the issues labelled as "good first issue" to start with. If you need further help, please feel free to drop an email, open an issue or ping us on our slack channel:

With best regards,
Gaurav Mishra


From: main@... <main@...> on behalf of doctor.lasalflair@... <doctor.lasalflair@...>
Sent: 16 November 2020 20:43
To: main@... <main@...>
Subject: [FOSSology] GSoC 2021
 
Hello Everyone!!! I am Lasal Jayawardena and I am very happy to join the community.

I am planning to enter the GSoC 2021 and I want to start contributing.
Can anyone give some pointers as to how I can do that?


GSoC 2021

doctor.lasalflair@...
 

Hello Everyone!!! I am Lasal Jayawardena and I am very happy to join the community.

I am planning to enter the GSoC 2021 and I want to start contributing.
Can anyone give some pointers as to how I can do that?


Re: Unpacking Debian Files - Fossology Version 3.8.1

Jeremiah C. Foster
 

Hi Deb,

I haven't run into this problem before. If this is a bug it would be kind of a big regression since a Debian developer helped build FOSSology.

For clarity - you're talking about .deb files? That is to say the Debian formatted binary package? Because there are also Debian source packages which are in a different format. .debs are AR archives and source packages tend to be compressed tarballs.

Regards,

Jeremiah


Re: Unpacking Debian Files - Fossology Version 3.8.1

Gaurav Mishra
 

Hello Deb,

 

Yesterday we merged the PR with the fix. You can compile the latest master or pull the latest docker image from Docker hub with the patch.

 

With best regards,

Gaurav Mishra

 

From: main@... <main@...> On Behalf Of D Frye
Sent: 09 November 2020 19:49
To: main@...
Subject: Re: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

 

Thanks for the quick find and fix.

 

Yes, we did change the base system on which Fossology is running. Our Fossology 3.8.1 instance is deployed to a separate server running Ubuntu 18.04 LTS.

 

Based on your diagnosis, as a workaround we tried changing /etc/mime.types to point 'deb' files back to the previous MIME type of "application/x-debian-package" but still saw the same behavior. Next we will try a custom compiled version that applies your patch.

 

Thank you for your prompt response!

 

Deb

 

From: main@... <main@...> On Behalf Of Gaurav Mishra
Sent: Monday, November 9, 2020 4:24 AM
To: main@...
Subject: EXTERNAL: Re: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

 

Hello Frye,

 

From what I understood with inconsistency in the unpacking, the deb files were not extracted at all and I tried it on my machine and it didn't work.

 

It seems like the mimetype of .deb archives was changed (only official document I could find is this https://www.iana.org/assignments/media-types/application/vnd.debian.binary-package).

I have created a PR with the new mimetype which you can find here: https://github.com/fossology/fossology/pull/1831

 

Also, I was wondering did you changed the base system the fossology is running, as there was no change in the code base related to it?

 

With best regards,

Gaurav Mishra


From: main@... <main@...> on behalf of D Frye <deborah.l.frye@...>
Sent: 05 November 2020 22:55
To: main@... <main@...>
Subject: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

 

Recently upgraded Fossology to Version 3.8.1.  Previous version unpacked and scanned Debian files within FOSS packages; however, seems like new version is not consistently unpacking them.  Has anyone else run into this?

 

Any help would be greatly appreciated.

Deb Frye

Lockheed Martin Corporation

Rotary and Mission Systems (RMS)

Software License Compliance / F-35 3PC Lead
P.O. Box 4840, EP7-C182, MD8
Syracuse, NY 13221-4840
(315) 456-2485
deborah.l.frye@...

 


Re: Unpacking Debian Files - Fossology Version 3.8.1

D Frye
 

Thanks for the quick find and fix.

 

Yes, we did change the base system on which Fossology is running. Our Fossology 3.8.1 instance is deployed to a separate server running Ubuntu 18.04 LTS.

 

Based on your diagnosis, as a workaround we tried changing /etc/mime.types to point 'deb' files back to the previous MIME type of "application/x-debian-package" but still saw the same behavior. Next we will try a custom compiled version that applies your patch.

 

Thank you for your prompt response!

 

Deb

 

From: main@... <main@...> On Behalf Of Gaurav Mishra
Sent: Monday, November 9, 2020 4:24 AM
To: main@...
Subject: EXTERNAL: Re: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

 

Hello Frye,

 

From what I understood with inconsistency in the unpacking, the deb files were not extracted at all and I tried it on my machine and it didn't work.

 

It seems like the mimetype of .deb archives was changed (only official document I could find is this https://www.iana.org/assignments/media-types/application/vnd.debian.binary-package).

I have created a PR with the new mimetype which you can find here: https://github.com/fossology/fossology/pull/1831

 

Also, I was wondering did you changed the base system the fossology is running, as there was no change in the code base related to it?

 

With best regards,

Gaurav Mishra


From: main@... <main@...> on behalf of D Frye <deborah.l.frye@...>
Sent: 05 November 2020 22:55
To: main@... <main@...>
Subject: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1

 

Recently upgraded Fossology to Version 3.8.1.  Previous version unpacked and scanned Debian files within FOSS packages; however, seems like new version is not consistently unpacking them.  Has anyone else run into this?

 

Any help would be greatly appreciated.

Deb Frye

Lockheed Martin Corporation

Rotary and Mission Systems (RMS)

Software License Compliance / F-35 3PC Lead
P.O. Box 4840, EP7-C182, MD8
Syracuse, NY 13221-4840
(315) 456-2485
deborah.l.frye@...

 


Re: Unpacking Debian Files - Fossology Version 3.8.1

Gaurav Mishra
 

Hello Frye,

From what I understood with inconsistency in the unpacking, the deb files were not extracted at all and I tried it on my machine and it didn't work.

It seems like the mimetype of .deb archives was changed (only official document I could find is this https://www.iana.org/assignments/media-types/application/vnd.debian.binary-package).
I have created a PR with the new mimetype which you can find here: https://github.com/fossology/fossology/pull/1831

Also, I was wondering did you changed the base system the fossology is running, as there was no change in the code base related to it?

With best regards,
Gaurav Mishra


From: main@... <main@...> on behalf of D Frye <deborah.l.frye@...>
Sent: 05 November 2020 22:55
To: main@... <main@...>
Subject: [FOSSology] Unpacking Debian Files - Fossology Version 3.8.1
 

Recently upgraded Fossology to Version 3.8.1.  Previous version unpacked and scanned Debian files within FOSS packages; however, seems like new version is not consistently unpacking them.  Has anyone else run into this?

 

Any help would be greatly appreciated.

Deb Frye

Lockheed Martin Corporation

Rotary and Mission Systems (RMS)

Software License Compliance / F-35 3PC Lead
P.O. Box 4840, EP7-C182, MD8
Syracuse, NY 13221-4840
(315) 456-2485
deborah.l.frye@...

 


Unpacking Debian Files - Fossology Version 3.8.1

D Frye
 

Recently upgraded Fossology to Version 3.8.1.  Previous version unpacked and scanned Debian files within FOSS packages; however, seems like new version is not consistently unpacking them.  Has anyone else run into this?

 

Any help would be greatly appreciated.

Deb Frye

Lockheed Martin Corporation

Rotary and Mission Systems (RMS)

Software License Compliance / F-35 3PC Lead
P.O. Box 4840, EP7-C182, MD8
Syracuse, NY 13221-4840
(315) 456-2485
deborah.l.frye@...

 


Re: Docker Container Restart Problems

 

PHP Warning:  syntax error, unexpected ')' in
/usr/local/etc/fossology/Db.conf on line 4

Did you check the content of the file Db.conf ?

--

Nicolas Toussaint - Open Source Expert
OBS - Orange Business Services - Lyon, France
Tel: +33 608 763 559

On 28/10/2020 13:10, flp via lists.fossology.org wrote:
If I downgrade to 9.6 with my docker-compose file, I get the following error:

*** Running postinstall for common actions***
*** Creating user and group ***
NOTE: group 'fossy' already exists, good.
NOTE: user 'fossy' already exists, good.
*** Making sure needed dirs exist with right ownership/permissions ***
*** clearing file cache ***
NOTE: Repository already exists at /srv/fossology/repository
NOTE: Running the PostgreSQL vacuum and analyze command can result in a large database performance improvement.
      We suggest that you either configure postgres to run its autovacuum and autoanalyze daemons, or maintagent -D in a cron job, or run Admin > Maintenance on a regular basis.
      Admin > Dashboard will show you the last time vacuum and analyze have been run.
NOTE: using external DB on host foss-db
*** update the database and license_ref table ***
PHP Warning:  syntax error, unexpected ')' in /usr/local/etc/fossology/Db.conf on line 4
 in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 98
PHP Warning:  Invalid argument supplied for foreach() in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 103
PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
Old release was 3.0.1
*** Cleaning tables for new constraints ***
Table author or agent does not exists, not cleaning!
Table author or pfile does not exists, not cleaning!
Table bucket_container or bucket_def does not exists, not cleaning!
Table bucket_file or bucket_def does not exists, not cleaning!
Table bucket_file or pfile does not exists, not cleaning!
Table copyright or agent does not exists, not cleaning!
Table copyright_decision or pfile does not exists, not cleaning!
Table ecc or agent does not exists, not cleaning!
Table ecc or pfile does not exists, not cleaning!
Table ecc_decision or pfile does not exists, not cleaning!
Table highlight_keyword or pfile does not exists, not cleaning!
Table keyword or agent does not exists, not cleaning!
Table keyword or pfile does not exists, not cleaning!
Table keyword_decision or pfile does not exists, not cleaning!
Table pkg_deb_req or pkg_deb does not exists, not cleaning!
Table pkg_rpm_req or pkg_rpm does not exists, not cleaning!
Table report_cache or upload does not exists, not cleaning!
Table report_info or upload does not exists, not cleaning!
Table reportgen or upload does not exists, not cleaning!
Table upload or pfile does not exists, not cleaning!
Table upload_clearing_license or upload does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table report_info does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table group_user_member does not exists, not cleaning!
Removed 0 rows from tables with new constraints
PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
PHP Warning:  pg_query(): Query failed: ERROR:  column "group_fk" of relation "license_candidate" already exists in /usr/local/share/fossology/lib/php/Db/Driver/Postgres.php on line 79
[2020-10-28 12:00:24] /usr/local/share/fossology/lib/php/libschema.php.CRITICAL: ERROR: column "group_fk" of relation "license_candidate" already exists [] []
PHP Fatal error:  Uncaught Fossology\Lib\Exception: error executing: ALTER TABLE "license_candidate" ADD COLUMN "group_fk" int8

ERROR:  column "group_fk" of relation "license_candidate" already exists in /usr/local/share/fossology/lib/php/Db/DbManager.php:144
Stack trace:
#0 /usr/local/share/fossology/lib/php/Db/DbManager.php(203): Fossology\Lib\Db\DbManager->checkResult(false, 'ALTER TABLE "li...')
#1 /usr/local/share/fossology/lib/php/libschema.php(299): Fossology\Lib\Db\DbManager->queryOnce('ALTER TABLE "li...')
#2 /usr/local/share/fossology/lib/php/libschema.php(163): fo_libschema->applyTables(true)
#3 /usr/local/lib/fossology/fossinit.php(196): fo_libschema->applySchema('/usr/local/shar...', false, 'fossology', Array)
#4 {main}
  thrown in /usr/local/share/fossology/lib/php/Db/DbManager.php on line 144


Re: Docker Container Restart Problems

flp@...
 

If I downgrade to 9.6 with my docker-compose file, I get the following error:

*** Running postinstall for common actions***
*** Creating user and group ***
NOTE: group 'fossy' already exists, good.
NOTE: user 'fossy' already exists, good.
*** Making sure needed dirs exist with right ownership/permissions ***
*** clearing file cache ***
NOTE: Repository already exists at /srv/fossology/repository
NOTE: Running the PostgreSQL vacuum and analyze command can result in a large database performance improvement.
      We suggest that you either configure postgres to run its autovacuum and autoanalyze daemons, or maintagent -D in a cron job, or run Admin > Maintenance on a regular basis.
      Admin > Dashboard will show you the last time vacuum and analyze have been run.
NOTE: using external DB on host foss-db
*** update the database and license_ref table ***
PHP Warning:  syntax error, unexpected ')' in /usr/local/etc/fossology/Db.conf on line 4
 in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 98
PHP Warning:  Invalid argument supplied for foreach() in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 103
PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
Old release was 3.0.1
*** Cleaning tables for new constraints ***
Table author or agent does not exists, not cleaning!
Table author or pfile does not exists, not cleaning!
Table bucket_container or bucket_def does not exists, not cleaning!
Table bucket_file or bucket_def does not exists, not cleaning!
Table bucket_file or pfile does not exists, not cleaning!
Table copyright or agent does not exists, not cleaning!
Table copyright_decision or pfile does not exists, not cleaning!
Table ecc or agent does not exists, not cleaning!
Table ecc or pfile does not exists, not cleaning!
Table ecc_decision or pfile does not exists, not cleaning!
Table highlight_keyword or pfile does not exists, not cleaning!
Table keyword or agent does not exists, not cleaning!
Table keyword or pfile does not exists, not cleaning!
Table keyword_decision or pfile does not exists, not cleaning!
Table pkg_deb_req or pkg_deb does not exists, not cleaning!
Table pkg_rpm_req or pkg_rpm does not exists, not cleaning!
Table report_cache or upload does not exists, not cleaning!
Table report_info or upload does not exists, not cleaning!
Table reportgen or upload does not exists, not cleaning!
Table upload or pfile does not exists, not cleaning!
Table upload_clearing_license or upload does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table report_info does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table group_user_member does not exists, not cleaning!
Removed 0 rows from tables with new constraints
PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
PHP Warning:  pg_query(): Query failed: ERROR:  column "group_fk" of relation "license_candidate" already exists in /usr/local/share/fossology/lib/php/Db/Driver/Postgres.php on line 79
[2020-10-28 12:00:24] /usr/local/share/fossology/lib/php/libschema.php.CRITICAL: ERROR:  column "group_fk" of relation "license_candidate" already exists [] []
PHP Fatal error:  Uncaught Fossology\Lib\Exception: error executing: ALTER TABLE "license_candidate" ADD COLUMN "group_fk" int8

ERROR:  column "group_fk" of relation "license_candidate" already exists in /usr/local/share/fossology/lib/php/Db/DbManager.php:144
Stack trace:
#0 /usr/local/share/fossology/lib/php/Db/DbManager.php(203): Fossology\Lib\Db\DbManager->checkResult(false, 'ALTER TABLE "li...')
#1 /usr/local/share/fossology/lib/php/libschema.php(299): Fossology\Lib\Db\DbManager->queryOnce('ALTER TABLE "li...')
#2 /usr/local/share/fossology/lib/php/libschema.php(163): fo_libschema->applyTables(true)
#3 /usr/local/lib/fossology/fossinit.php(196): fo_libschema->applySchema('/usr/local/shar...', false, 'fossology', Array)
#4 {main}
  thrown in /usr/local/share/fossology/lib/php/Db/DbManager.php on line 144


Re: Docker Container Restart Problems

 

Hi flp,

I did a quick check on my server, reusing you Dockerfile:
- changing to docker-compose version to 3.3 (because I had to)
- setting POSTGRES_* variables to fossology / fossy / fossy

It looks like the DB container keeps on restarting with Postgres v11, but that does not happen when I downgrade to v9.6 (which is the version I use in production)

Hoping it  helps a little

Nico

--

Nicolas Toussaint - Open Source Expert
OBS - Orange Business Services - Lyon, France
Tel: +33 608 763 559

On 27/10/2020 11:48, Michael C. Jaeger wrote:

Hello flp,

I am not very familiar with docker, so I do not have a good insight on what you do with your custom docker file cited at the end. I checked out current master and executed docker-compose up and down and up and down with uploading packages in between - > no issues at all.

Would there be any context information (like an old image laying around which is not subject for update) for your setup? Or did you create the docker-compose setup from scratch just based on the attached docker-compose.yml ?

Kind regards,

  Michael

*From: *<main@lists.fossology.org> on behalf of "flp via lists.fossology.org" <flp=mailbox.org@lists.fossology.org>
*Reply to: *"main@lists.fossology.org" <main@lists.fossology.org>
*Date: *Saturday, 24. October 2020 at 06:16
*To: *"main@lists.fossology.org" <main@lists.fossology.org>
*Subject: *[FOSSology] Docker Container Restart Problems

Hey,

I try to run fossology as docker compose cluster. Unfortunately, it always crashes when I need to restart the fossology container (3.7.0, 3.8.1 or 3.9.0-rc2).

I end up with the following error in 3.8.1 und 3.9.0:

*** Running postinstall for common actions***

*** Creating user and group ***

NOTE: group 'fossy' already exists, good.

NOTE: user 'fossy' already exists, good.

*** Making sure needed dirs exist with right ownership/permissions ***

*** clearing file cache ***

NOTE: Repository already exists at /srv/fossology/repository

NOTE: Running the PostgreSQL vacuum and analyze command can result in a large database performance improvement.

    We suggest that you either configure postgres to run its autovacuum and autoanalyze daemons, or maintagent -D in a cron job, or run Admin > Maintenance on a regular basis.

    Admin > Dashboard will show you the last time vacuum and analyze have been run.

NOTE: using external DB on host foss-db

*** update the database and license_ref table ***

PHP Warning:  syntax error, unexpected ')' in /usr/local/etc/fossology/Db.conf on line 4

 in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 96

PHP Warning:  Invalid argument supplied for foreach() in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 101

PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520

Old release was 3.0.1

*** Cleaning tables for new constraints ***

Table author or agent does not exists, not cleaning!

Table author or pfile does not exists, not cleaning!

Table bucket_container or bucket_def does not exists, not cleaning!

Table bucket_file or bucket_def does not exists, not cleaning!

Table bucket_file or pfile does not exists, not cleaning!

Table copyright or agent does not exists, not cleaning!

Table copyright_decision or pfile does not exists, not cleaning!

Table ecc or agent does not exists, not cleaning!

Table ecc or pfile does not exists, not cleaning!

Table ecc_decision or pfile does not exists, not cleaning!

Table highlight_keyword or pfile does not exists, not cleaning!

Table keyword or agent does not exists, not cleaning!

Table keyword or pfile does not exists, not cleaning!

Table keyword_decision or pfile does not exists, not cleaning!

Table pkg_deb_req or pkg_deb does not exists, not cleaning!

Table pkg_rpm_req or pkg_rpm does not exists, not cleaning!

Table report_cache or upload does not exists, not cleaning!

Table report_info or upload does not exists, not cleaning!

Table reportgen or upload does not exists, not cleaning!

Table upload or pfile does not exists, not cleaning!

Table upload_clearing_license or upload does not exists, not cleaning!

Table obligation_ref does not exists, not cleaning!

Table report_info does not exists, not cleaning!

Table obligation_ref does not exists, not cleaning!

Table group_user_member does not exists, not cleaning!

Removed 0 rows from tables with new constraints

PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520

PHP Warning:  pg_query(): Query failed: ERROR:  column "rf_copyleft" of relation "license_ref" already exists in /usr/local/share/fossology/lib/php/Db/Driver/Postgres.php on line 79

[2020-10-18 21:38:19] /usr/local/share/fossology/lib/php/libschema.php.CRITICAL: ERROR:  column "rf_copyleft" of relation "license_ref" already exists [] []

PHP Fatal error:  Uncaught Fossology\Lib\Exception: error executing: ALTER TABLE "license_ref" ADD COLUMN "rf_copyleft" bit(1)

ERROR:  column "rf_copyleft" of relation "license_ref" already exists in /usr/local/share/fossology/lib/php/Db/DbManager.php:144

Stack trace:

#0 /usr/local/share/fossology/lib/php/Db/DbManager.php(203): Fossology\Lib\Db\DbManager->checkResult(false, 'ALTER TABLE "li...')

#1 /usr/local/share/fossology/lib/php/libschema.php(299): Fossology\Lib\Db\DbManager->queryOnce('ALTER TABLE "li...')

#2 /usr/local/share/fossology/lib/php/libschema.php(161): fo_libschema->applyTables()

#3 /usr/local/lib/fossology/fossinit.php(196): fo_libschema->applySchema('/usr/local/shar...', false, 'fossology', Array)

#4 {main}

thrown in /usr/local/share/fossology/lib/php/Db/DbManager.php on line 144


With 3.7.0 I get the following:

*** Running postinstall for common actions***

*** Creating user and group ***

NOTE: group 'fossy' already exists, good.

NOTE: user 'fossy' already exists, good.

*** Making sure needed dirs exist with right ownership/permissions ***

*** clearing file cache ***

NOTE: Repository already exists at /srv/fossology/repository

NOTE: Running the PostgreSQL vacuum and analyze command can result in a large database performance improvement.

    We suggest that you either configure postgres to run its autovacuum and autoanalyze daemons, or maintagent -D in a cron job, or run Admin > Maintenance on a regular basis.

    Admin > Dashboard will show you the last time vacuum and analyze have been run.

NOTE: using external DB on host foss-db

*** update the database and license_ref table ***

PHP Warning:  syntax error, unexpected ')' in /usr/local/etc/fossology/Db.conf on line 4

 in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 96

PHP Warning:  Invalid argument supplied for foreach() in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 101

PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520

Old release was 0

*** Cleaning tables for new constraints ***

Table author or agent does not exists, not cleaning!

Table author or pfile does not exists, not cleaning!

Table bucket_container or bucket_def does not exists, not cleaning!

Table bucket_file or bucket_def does not exists, not cleaning!

Table bucket_file or pfile does not exists, not cleaning!

Table copyright or agent does not exists, not cleaning!

Table copyright_decision or pfile does not exists, not cleaning!

Table ecc or agent does not exists, not cleaning!

Table ecc or pfile does not exists, not cleaning!

Table ecc_decision or pfile does not exists, not cleaning!

Table highlight_keyword or pfile does not exists, not cleaning!

Table keyword or agent does not exists, not cleaning!

Table keyword or pfile does not exists, not cleaning!

Table keyword_decision or pfile does not exists, not cleaning!

Table pkg_deb_req or pkg_deb does not exists, not cleaning!

Table pkg_rpm_req or pkg_rpm does not exists, not cleaning!

Table report_cache or upload does not exists, not cleaning!

Table report_info or upload does not exists, not cleaning!

Table reportgen or upload does not exists, not cleaning!

Table upload or pfile does not exists, not cleaning!

Table upload_clearing_license or upload does not exists, not cleaning!

Table obligation_ref does not exists, not cleaning!

Table report_info does not exists, not cleaning!

Table obligation_ref does not exists, not cleaning!

Table group_user_member does not exists, not cleaning!

Removed 0 rows from tables with new constraints

PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520

PHP Warning:  pg_query(): Query failed: ERROR:  column "option_value" of relation "sysconfig" already exists in /usr/local/share/fossology/lib/php/Db/Driver/Postgres.php on line 79

[2020-10-22 21:49:47] /usr/local/share/fossology/lib/php/libschema.php.CRITICAL: ERROR:  column "option_value" of relation "sysconfig" already exists [] []

PHP Fatal error:  Uncaught Fossology\Lib\Exception: error executing: ALTER TABLE "sysconfig" ADD COLUMN "option_value" character varying(40) DEFAULT NULL

ERROR:  column "option_value" of relation "sysconfig" already exists in /usr/local/share/fossology/lib/php/Db/DbManager.php:144

Stack trace:

#0 /usr/local/share/fossology/lib/php/Db/DbManager.php(203): Fossology\Lib\Db\DbManager->checkResult(false, 'ALTER TABLE "sy...')

#1 /usr/local/share/fossology/lib/php/libschema.php(299): Fossology\Lib\Db\DbManager->queryOnce('ALTER TABLE "sy...')

#2 /usr/local/share/fossology/lib/php/libschema.php(161): fo_libschema->applyTables()

#3 /usr/local/lib/fossology/fossinit.php(191): fo_libschema->applySchema('/usr/local/shar...', false, 'fossology', Array)

#4 {main} thrown in /usr/local/share/fossology/lib/php/Db/DbManager.php on line 144

I'm using fossology with the following compose cluster:

version: '3.8'
services:
  foss:
    image: fossology/fossology:3.9.0-rc2
    restart: unless-stopped
    environment:
      FOSSOLOGY_DB_HOST:
      FOSSOLOGY_DB_NAME:
      FOSSOLOGY_DB_USER:
      FOSSOLOGY_DB_PASSWORD:
    volumes:
      - ./foss-repo:/srv/fossology/repository/
    networks:
      - foss
    depends_on:
      - foss-db
  foss-db:
    image: postgres:11
    restart: unless-stopped
    environment:
      POSTGRES_DB:
      POSTGRES_USER:
      POSTGRES_PASSWORD:
      POSTGRES_INITDB_ARGS: '-E SQL_ASCII'
    volumes:
      - ./postgres-data:/var/lib/postgresql/data
    networks:
      - foss
networks:
  foss:
    external:
      name: foss

I have no particular experience with Php but it looks like the startup script is always executed within the docker container so I might need to create my own one that does not execute the database script at startup?



Re: Docker Container Restart Problems

Michael C. Jaeger
 

Hello flp,

 

I am not very familiar with docker, so I do not have a good insight on what you do with your custom docker file cited at the end. I checked out current master and executed docker-compose up and down and up and down with uploading packages in between - > no issues at all.

 

Would there be any context information (like an old image laying around which is not subject for update) for your setup? Or did you create the docker-compose setup from scratch just based on the attached docker-compose.yml ?

 

Kind regards,

  Michael

 

From: <main@...> on behalf of "flp via lists.fossology.org" <flp=mailbox.org@...>
Reply to: "main@..." <main@...>
Date: Saturday, 24. October 2020 at 06:16
To: "main@..." <main@...>
Subject: [FOSSology] Docker Container Restart Problems

 

Hey,

I try to run fossology as docker compose cluster. Unfortunately, it always crashes when I need to restart the fossology container (3.7.0, 3.8.1 or 3.9.0-rc2).

I end up with the following error in 3.8.1 und 3.9.0:

*** Running postinstall for common actions***

*** Creating user and group ***

NOTE: group 'fossy' already exists, good.

NOTE: user 'fossy' already exists, good.

*** Making sure needed dirs exist with right ownership/permissions ***

*** clearing file cache ***

NOTE: Repository already exists at /srv/fossology/repository

NOTE: Running the PostgreSQL vacuum and analyze command can result in a large database performance improvement.

      We suggest that you either configure postgres to run its autovacuum and autoanalyze daemons, or maintagent -D in a cron job, or run Admin > Maintenance on a regular basis.

      Admin > Dashboard will show you the last time vacuum and analyze have been run.

NOTE: using external DB on host foss-db

*** update the database and license_ref table ***

PHP Warning:  syntax error, unexpected ')' in /usr/local/etc/fossology/Db.conf on line 4

 in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 96

PHP Warning:  Invalid argument supplied for foreach() in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 101

PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520

Old release was 3.0.1

*** Cleaning tables for new constraints ***

Table author or agent does not exists, not cleaning!

Table author or pfile does not exists, not cleaning!

Table bucket_container or bucket_def does not exists, not cleaning!

Table bucket_file or bucket_def does not exists, not cleaning!

Table bucket_file or pfile does not exists, not cleaning!

Table copyright or agent does not exists, not cleaning!

Table copyright_decision or pfile does not exists, not cleaning!

Table ecc or agent does not exists, not cleaning!

Table ecc or pfile does not exists, not cleaning!

Table ecc_decision or pfile does not exists, not cleaning!

Table highlight_keyword or pfile does not exists, not cleaning!

Table keyword or agent does not exists, not cleaning!

Table keyword or pfile does not exists, not cleaning!

Table keyword_decision or pfile does not exists, not cleaning!

Table pkg_deb_req or pkg_deb does not exists, not cleaning!

Table pkg_rpm_req or pkg_rpm does not exists, not cleaning!

Table report_cache or upload does not exists, not cleaning!

Table report_info or upload does not exists, not cleaning!

Table reportgen or upload does not exists, not cleaning!

Table upload or pfile does not exists, not cleaning!

Table upload_clearing_license or upload does not exists, not cleaning!

Table obligation_ref does not exists, not cleaning!

Table report_info does not exists, not cleaning!

Table obligation_ref does not exists, not cleaning!

Table group_user_member does not exists, not cleaning!

Removed 0 rows from tables with new constraints

PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520

PHP Warning:  pg_query(): Query failed: ERROR:  column "rf_copyleft" of relation "license_ref" already exists in /usr/local/share/fossology/lib/php/Db/Driver/Postgres.php on line 79

[2020-10-18 21:38:19] /usr/local/share/fossology/lib/php/libschema.php.CRITICAL: ERROR:  column "rf_copyleft" of relation "license_ref" already exists [] []

PHP Fatal error:  Uncaught Fossology\Lib\Exception: error executing: ALTER TABLE "license_ref" ADD COLUMN "rf_copyleft" bit(1)

 

ERROR:  column "rf_copyleft" of relation "license_ref" already exists in /usr/local/share/fossology/lib/php/Db/DbManager.php:144

Stack trace:

#0 /usr/local/share/fossology/lib/php/Db/DbManager.php(203): Fossology\Lib\Db\DbManager->checkResult(false, 'ALTER TABLE "li...')

#1 /usr/local/share/fossology/lib/php/libschema.php(299): Fossology\Lib\Db\DbManager->queryOnce('ALTER TABLE "li...')

#2 /usr/local/share/fossology/lib/php/libschema.php(161): fo_libschema->applyTables()

#3 /usr/local/lib/fossology/fossinit.php(196): fo_libschema->applySchema('/usr/local/shar...', false, 'fossology', Array)

#4 {main}

  thrown in /usr/local/share/fossology/lib/php/Db/DbManager.php on line 144


With 3.7.0 I get the following:

*** Running postinstall for common actions***

*** Creating user and group ***

NOTE: group 'fossy' already exists, good.

NOTE: user 'fossy' already exists, good.

*** Making sure needed dirs exist with right ownership/permissions ***

*** clearing file cache ***

NOTE: Repository already exists at /srv/fossology/repository

NOTE: Running the PostgreSQL vacuum and analyze command can result in a large database performance improvement.

      We suggest that you either configure postgres to run its autovacuum and autoanalyze daemons, or maintagent -D in a cron job, or run Admin > Maintenance on a regular basis.

      Admin > Dashboard will show you the last time vacuum and analyze have been run.

NOTE: using external DB on host foss-db

*** update the database and license_ref table ***

PHP Warning:  syntax error, unexpected ')' in /usr/local/etc/fossology/Db.conf on line 4

 in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 96

PHP Warning:  Invalid argument supplied for foreach() in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 101

PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520

Old release was 0

*** Cleaning tables for new constraints ***

Table author or agent does not exists, not cleaning!

Table author or pfile does not exists, not cleaning!

Table bucket_container or bucket_def does not exists, not cleaning!

Table bucket_file or bucket_def does not exists, not cleaning!

Table bucket_file or pfile does not exists, not cleaning!

Table copyright or agent does not exists, not cleaning!

Table copyright_decision or pfile does not exists, not cleaning!

Table ecc or agent does not exists, not cleaning!

Table ecc or pfile does not exists, not cleaning!

Table ecc_decision or pfile does not exists, not cleaning!

Table highlight_keyword or pfile does not exists, not cleaning!

Table keyword or agent does not exists, not cleaning!

Table keyword or pfile does not exists, not cleaning!

Table keyword_decision or pfile does not exists, not cleaning!

Table pkg_deb_req or pkg_deb does not exists, not cleaning!

Table pkg_rpm_req or pkg_rpm does not exists, not cleaning!

Table report_cache or upload does not exists, not cleaning!

Table report_info or upload does not exists, not cleaning!

Table reportgen or upload does not exists, not cleaning!

Table upload or pfile does not exists, not cleaning!

Table upload_clearing_license or upload does not exists, not cleaning!

Table obligation_ref does not exists, not cleaning!

Table report_info does not exists, not cleaning!

Table obligation_ref does not exists, not cleaning!

Table group_user_member does not exists, not cleaning!

Removed 0 rows from tables with new constraints

PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520

PHP Warning:  pg_query(): Query failed: ERROR:  column "option_value" of relation "sysconfig" already exists in /usr/local/share/fossology/lib/php/Db/Driver/Postgres.php on line 79

[2020-10-22 21:49:47] /usr/local/share/fossology/lib/php/libschema.php.CRITICAL: ERROR:  column "option_value" of relation "sysconfig" already exists [] []

PHP Fatal error:  Uncaught Fossology\Lib\Exception: error executing: ALTER TABLE "sysconfig" ADD COLUMN "option_value" character varying(40) DEFAULT NULL

 

ERROR:  column "option_value" of relation "sysconfig" already exists in /usr/local/share/fossology/lib/php/Db/DbManager.php:144

Stack trace:

#0 /usr/local/share/fossology/lib/php/Db/DbManager.php(203): Fossology\Lib\Db\DbManager->checkResult(false, 'ALTER TABLE "sy...')

#1 /usr/local/share/fossology/lib/php/libschema.php(299): Fossology\Lib\Db\DbManager->queryOnce('ALTER TABLE "sy...')

#2 /usr/local/share/fossology/lib/php/libschema.php(161): fo_libschema->applyTables()

#3 /usr/local/lib/fossology/fossinit.php(191): fo_libschema->applySchema('/usr/local/shar...', false, 'fossology', Array)

          #4 {main} thrown in /usr/local/share/fossology/lib/php/Db/DbManager.php on line 144

I'm using fossology with the following compose cluster:

version: '3.8'
services:
  foss:
    image: fossology/fossology:3.9.0-rc2
    restart: unless-stopped
    environment:
      FOSSOLOGY_DB_HOST:
      FOSSOLOGY_DB_NAME:
      FOSSOLOGY_DB_USER:
      FOSSOLOGY_DB_PASSWORD:
    volumes:
      - ./foss-repo:/srv/fossology/repository/
    networks:
      - foss
    depends_on:
      - foss-db
  foss-db:
    image: postgres:11
    restart: unless-stopped
    environment:
      POSTGRES_DB:
      POSTGRES_USER:
      POSTGRES_PASSWORD:
      POSTGRES_INITDB_ARGS: '-E SQL_ASCII'
    volumes:
      - ./postgres-data:/var/lib/postgresql/data
    networks:
      - foss
networks:
  foss:
    external:
      name: foss


I have no particular experience with Php but it looks like the startup script is always executed within the docker container so I might need to create my own one that does not execute the database script at startup?



Docker Container Restart Problems

flp@...
 

Hey,

I try to run fossology as docker compose cluster. Unfortunately, it always crashes when I need to restart the fossology container (3.7.0, 3.8.1 or 3.9.0-rc2).

I end up with the following error in 3.8.1 und 3.9.0:
*** Running postinstall for common actions***
*** Creating user and group ***
NOTE: group 'fossy' already exists, good.
NOTE: user 'fossy' already exists, good.
*** Making sure needed dirs exist with right ownership/permissions ***
*** clearing file cache ***
NOTE: Repository already exists at /srv/fossology/repository
NOTE: Running the PostgreSQL vacuum and analyze command can result in a large database performance improvement.
      We suggest that you either configure postgres to run its autovacuum and autoanalyze daemons, or maintagent -D in a cron job, or run Admin > Maintenance on a regular basis.
      Admin > Dashboard will show you the last time vacuum and analyze have been run.
NOTE: using external DB on host foss-db
*** update the database and license_ref table ***
PHP Warning:  syntax error, unexpected ')' in /usr/local/etc/fossology/Db.conf on line 4
 in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 96
PHP Warning:  Invalid argument supplied for foreach() in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 101
PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
Old release was 3.0.1
*** Cleaning tables for new constraints ***
Table author or agent does not exists, not cleaning!
Table author or pfile does not exists, not cleaning!
Table bucket_container or bucket_def does not exists, not cleaning!
Table bucket_file or bucket_def does not exists, not cleaning!
Table bucket_file or pfile does not exists, not cleaning!
Table copyright or agent does not exists, not cleaning!
Table copyright_decision or pfile does not exists, not cleaning!
Table ecc or agent does not exists, not cleaning!
Table ecc or pfile does not exists, not cleaning!
Table ecc_decision or pfile does not exists, not cleaning!
Table highlight_keyword or pfile does not exists, not cleaning!
Table keyword or agent does not exists, not cleaning!
Table keyword or pfile does not exists, not cleaning!
Table keyword_decision or pfile does not exists, not cleaning!
Table pkg_deb_req or pkg_deb does not exists, not cleaning!
Table pkg_rpm_req or pkg_rpm does not exists, not cleaning!
Table report_cache or upload does not exists, not cleaning!
Table report_info or upload does not exists, not cleaning!
Table reportgen or upload does not exists, not cleaning!
Table upload or pfile does not exists, not cleaning!
Table upload_clearing_license or upload does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table report_info does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table group_user_member does not exists, not cleaning!
Removed 0 rows from tables with new constraints
PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
PHP Warning:  pg_query(): Query failed: ERROR:  column "rf_copyleft" of relation "license_ref" already exists in /usr/local/share/fossology/lib/php/Db/Driver/Postgres.php on line 79
[2020-10-18 21:38:19] /usr/local/share/fossology/lib/php/libschema.php.CRITICAL: ERROR:  column "rf_copyleft" of relation "license_ref" already exists [] []
PHP Fatal error:  Uncaught Fossology\Lib\Exception: error executing: ALTER TABLE "license_ref" ADD COLUMN "rf_copyleft" bit(1)
 
ERROR:  column "rf_copyleft" of relation "license_ref" already exists in /usr/local/share/fossology/lib/php/Db/DbManager.php:144
Stack trace:
#0 /usr/local/share/fossology/lib/php/Db/DbManager.php(203): Fossology\Lib\Db\DbManager->checkResult(false, 'ALTER TABLE "li...')
#1 /usr/local/share/fossology/lib/php/libschema.php(299): Fossology\Lib\Db\DbManager->queryOnce('ALTER TABLE "li...')
#2 /usr/local/share/fossology/lib/php/libschema.php(161): fo_libschema->applyTables()
#3 /usr/local/lib/fossology/fossinit.php(196): fo_libschema->applySchema('/usr/local/shar...', false, 'fossology', Array)
#4 {main}
  thrown in /usr/local/share/fossology/lib/php/Db/DbManager.php on line 144

With 3.7.0 I get the following:
*** Running postinstall for common actions***
*** Creating user and group ***
NOTE: group 'fossy' already exists, good.
NOTE: user 'fossy' already exists, good.
*** Making sure needed dirs exist with right ownership/permissions ***
*** clearing file cache ***
NOTE: Repository already exists at /srv/fossology/repository
NOTE: Running the PostgreSQL vacuum and analyze command can result in a large database performance improvement.
      We suggest that you either configure postgres to run its autovacuum and autoanalyze daemons, or maintagent -D in a cron job, or run Admin > Maintenance on a regular basis.
      Admin > Dashboard will show you the last time vacuum and analyze have been run.
NOTE: using external DB on host foss-db
*** update the database and license_ref table ***
PHP Warning:  syntax error, unexpected ')' in /usr/local/etc/fossology/Db.conf on line 4
 in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 96
PHP Warning:  Invalid argument supplied for foreach() in /usr/local/share/fossology/lib/php/common-sysconfig.php on line 101
PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
Old release was 0
*** Cleaning tables for new constraints ***
Table author or agent does not exists, not cleaning!
Table author or pfile does not exists, not cleaning!
Table bucket_container or bucket_def does not exists, not cleaning!
Table bucket_file or bucket_def does not exists, not cleaning!
Table bucket_file or pfile does not exists, not cleaning!
Table copyright or agent does not exists, not cleaning!
Table copyright_decision or pfile does not exists, not cleaning!
Table ecc or agent does not exists, not cleaning!
Table ecc or pfile does not exists, not cleaning!
Table ecc_decision or pfile does not exists, not cleaning!
Table highlight_keyword or pfile does not exists, not cleaning!
Table keyword or agent does not exists, not cleaning!
Table keyword or pfile does not exists, not cleaning!
Table keyword_decision or pfile does not exists, not cleaning!
Table pkg_deb_req or pkg_deb does not exists, not cleaning!
Table pkg_rpm_req or pkg_rpm does not exists, not cleaning!
Table report_cache or upload does not exists, not cleaning!
Table report_info or upload does not exists, not cleaning!
Table reportgen or upload does not exists, not cleaning!
Table upload or pfile does not exists, not cleaning!
Table upload_clearing_license or upload does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table report_info does not exists, not cleaning!
Table obligation_ref does not exists, not cleaning!
Table group_user_member does not exists, not cleaning!
Removed 0 rows from tables with new constraints
PHP Notice:  Undefined index: user in /usr/local/share/fossology/lib/php/libschema.php on line 520
PHP Warning:  pg_query(): Query failed: ERROR:  column "option_value" of relation "sysconfig" already exists in /usr/local/share/fossology/lib/php/Db/Driver/Postgres.php on line 79
[2020-10-22 21:49:47] /usr/local/share/fossology/lib/php/libschema.php.CRITICAL: ERROR:  column "option_value" of relation "sysconfig" already exists [] []
PHP Fatal error:  Uncaught Fossology\Lib\Exception: error executing: ALTER TABLE "sysconfig" ADD COLUMN "option_value" character varying(40) DEFAULT NULL
 
ERROR:  column "option_value" of relation "sysconfig" already exists in /usr/local/share/fossology/lib/php/Db/DbManager.php:144
Stack trace:
#0 /usr/local/share/fossology/lib/php/Db/DbManager.php(203): Fossology\Lib\Db\DbManager->checkResult(false, 'ALTER TABLE "sy...')
#1 /usr/local/share/fossology/lib/php/libschema.php(299): Fossology\Lib\Db\DbManager->queryOnce('ALTER TABLE "sy...')
#2 /usr/local/share/fossology/lib/php/libschema.php(161): fo_libschema->applyTables()
#3 /usr/local/lib/fossology/fossinit.php(191): fo_libschema->applySchema('/usr/local/shar...', false, 'fossology', Array)
          #4 {main} thrown in /usr/local/share/fossology/lib/php/Db/DbManager.php on line 144

I'm using fossology with the following compose cluster:

version: '3.8'
services:
  foss:
    image: fossology/fossology:3.9.0-rc2
    restart: unless-stopped
    environment:
      FOSSOLOGY_DB_HOST:
      FOSSOLOGY_DB_NAME:
      FOSSOLOGY_DB_USER:
      FOSSOLOGY_DB_PASSWORD:
    volumes:
      - ./foss-repo:/srv/fossology/repository/
    networks:
      - foss
    depends_on:
      - foss-db
  foss-db:
    image: postgres:11
    restart: unless-stopped
    environment:
      POSTGRES_DB:
      POSTGRES_USER:
      POSTGRES_PASSWORD:
      POSTGRES_INITDB_ARGS: '-E SQL_ASCII'
    volumes:
      - ./postgres-data:/var/lib/postgresql/data
    networks:
      - foss
networks:
  foss:
    external:
      name: foss


I have no particular experience with Php but it looks like the startup script is always executed within the docker container so I might need to create my own one that does not execute the database script at startup?




End of Google Summer of Code - 2020

Shaheem Azmal M MD
 

Hello Everyone,

 

We would like to congratulate Darshan, Kaushlendra and Ayush for their contributions to FOSSology community. We are very impressed by the overall quality of their work and organizational skills, keep up the good work and stay in touch with community.

 

Details of features that were developed in this GSOC-2020.

 

Project : FOSSology

Student Name : Darshan Kansagara

Feature Title : Dashboard (Python, influxDB, Grafana)

Wiki : https://github.com/darshank15/GSoC_2020_FOSSOlogy/wiki

 

Project : Atarashi

Student Name : Ayush (HastagAB)

Feature Title : Accelerating Atarashi

Wiki : https://github.com/hastagAB/GSoC-2020

 

Project : Atarashi

Student Name : Kaushlendra Pratap Singh

Feature Title : Refurbishing Atarashi

Wiki : https://github.com/Kaushl2208/GSoC-2020

 

We would also like to thank mentors(Michael, Shaheem, Anupam, Gaurav, Nicolas, Aman and Sandeep) for helping students achieve their targets.

 

 

Best Regards

Shaheem Azmal M MD

 

 

 

1 - 20 of 57