8000 [BUG] New install, WebGUI shows "None found" - Maybe Manticore issue? · Issue #273 · jsuto/piler · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[BUG] New install, WebGUI shows "None found" - Maybe Manticore issue? #273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
fs2810 opened this issue Apr 19, 2025 · 10 comments
Open

[BUG] New install, WebGUI shows "None found" - Maybe Manticore issue? #273

fs2810 opened this issue Apr 19, 2025 · 10 comments
Assignees
@jsuto
Labels
stale

Comments

@fs2810
Copy link
fs2810 commented Apr 19, 2025

Describe the bug
After doing a clean install on a newly upgraded Ubuntu 22.04 Server and importing ~ 4600 mails (EML) using pilerimport, the WebGUI shows "None found" for any search, even though the Health check shows imported mails.

Image

First try was with Manticore 9 as per apt install from their Repo -> None found
make postinstall worked flawlessly though

Tried to install Manticore 6.3.8 but for the life of me was unable to do so :-(
Building joinsorter threw an error about conflicting headers.

Installed Manticore 7.4.6 (Building this worked without a problem!)
make postinstall also worked fine with this version

After installing every Manticore version mentioned above, I did a clean reinstall of piler
(Dropping DB, removing /var/piler, cleaning up /usr/local)

reindex -a does not change anything afterwards

Expected behavior
Should show imported mails in WebGUI

Piler version:
piler 1.4.7, Janos SUTO sj@acts.hu

Build Date: Fri Apr 18 21:12:19 CEST 2025
ldd version: ldd (Ubuntu GLIBC 2.35-0ubuntu3.9) 2.35
gcc version: gcc version 11.4.0 (Ubuntu 11.4.0-1ubuntu1~22.04)
OS: Linux mail.designwindmill.com 5.15.0-138-generic #148-Ubuntu SMP Fri Mar 14 19:05:48 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Configure command: ./configure --localstatedir=/var --with-database=mysql
MySQL client library version: 8.0.41
Extractors: /usr/bin/pdftotext /usr/bin/catdoc /usr/bin/catppt /usr/bin/xls2csv /usr/local/bin/ppthtml /usr/bin/unrtf /usr/bin/tnef libzip

Notes
Building Manticore from sources was done according to their instructions: https://manual.manticoresearch.com/Installation/Compiling_from_sources, Installing Manticore 9 was done according to https://manticoresearch.com/install/
@jsuto
Copy link
Owner
jsuto commented Apr 19, 2025 •
edited
Loading

OK, for starters check the manticore tables:

mysql -h0 -P9306
show tables;

Assuming you have real time index, you may see "piler1" index table. In that case check how many emails it has indexed:

show index piler1 status;

If you can see roughly 4600 documents, then create an auditor user (later you may remove it if not needed), login as auditor, as simply hit the red search button.

@fs2810
Copy link
Author
fs2810 commented Apr 19, 2025 •
edited
Loading

mysql> show tables;
+--------+------+
| Table | Type |
+--------+------+
| audit1 | rt |
| note1 | rt |
| piler1 | rt |
| tag1 | rt |
+--------+------+
4 rows in set (0.00 sec)

mysql> show index piler1 status;
+-----------------------------+---------------------------------------------------------------------------------------------------------+
| Variable_name 8000 | Value |
+-----------------------------+---------------------------------------------------------------------------------------------------------+
| table_type | rt |
| indexed_documents | 4651 |
| indexed_bytes | 12407185 |
| ram_bytes | 734025 |
| disk_bytes | 7530672 |
| disk_mapped | 698301 |
| disk_mapped_cached | 708608 |
| disk_mapped_doclists | 0 |
| disk_mapped_cached_doclists | 0 |
| disk_mapped_hitlists | 0 |
| disk_mapped_cached_hitlists | 0 |
| killed_documents | 0 |
| killed_rate | 0.00% |
| ram_chunk | 18545 |
| ram_chunk_segments_count | 1 |
| disk_chunks | 1 |
| mem_limit | 536870912 |
| mem_limit_rate | 95.00% |
| ram_bytes_retired | 0 |
| optimizing | 0 |
| locked | 0 |
| tid | 4651 |
| tid_saved | 4651 |
| query_time_1min | {"queries":0, "avg":"-", "min":"-", "max":"-", "pct95":"-", "pct99":"-"} |
| query_time_5min | {"queries":0, "avg":"-", "min":"-", "max":"-", "pct95":"-", "pct99":"-"} |
| query_time_15min | {"queries":0, "avg":"-", "min":"-", "max":"-", "pct95":"-", "pct99":"-"} |
| query_time_total | {"queries":11, "avg_sec":0.000, "min_sec":0.000, "max_sec":0.001, "pct95_sec":0.001, "pct99_sec":0.001} |
| found_rows_1min | {"queries":0, "avg":"-", "min":"-", "max":"-", "pct95":"-", "pct99":"-"} |
| found_rows_5min | {"queries":0, "avg":"-", "min":"-", "max":"-", "pct95":"-", "pct99":"-"} |
| found_rows_15min | {"queries":0, "avg":"-", "min":"-", "max":"-", "pct95":"-", "pct99":"-"} |
| found_rows_total | {"queries":11, "avg":0, "min":0, "max":0, "pct95":0, "pct99":0} |
+-----------------------------+---------------------------------------------------------------------------------------------------------+
31 rows in set (0.00 sec)

Logging in as an Auditor does seem work...
But why can't I see mails as an admin?

And apart from that, I still have an issue with the webGUI, it constantly switches the URL to the first Virtualhost on the server (First time after login and several times later like when changing the password of the built-in auditor user...
And no, config-site.php has the correct SITE_URL
Even the Health-Check has the wrong URL (mail.designwindmill.com, as seen in the screenshot above, instead of archive.designwindmill.com which is the SITE_URL)

Not sure I want to try it again with Manticore 9 though ;-)))

@jsuto
Copy link
Owner
jsuto commented Apr 19, 2025

Administrator is not supposed to search email by design. A regular user or an auditor account may do so. Btw. if you insist, then set this in config-site.php:

$config['ADMIN_CAN_POWER_SEARCH'] = 1;

Regarding the wrong sitename, the UI executes these commands: hostname -f and uname -a. Btw. what's in the browser's URL? It should be the SITE_URL value.

Anway, I can see that probably you deployed the archive to the mail server. Technically you can do that, however, from a disaster recovery point I suggest a different host, dedicated to the archive.

@fs2810
Copy link
Author
fs2810 commented Apr 19, 2025

Can I change the behavior for it NOT to use the Hostname? ;-)

This is my wife's server and too say it's a small business would be an understatement, therefore there is just one server available.
You are right of course, but I compensate with backups that include the archive.

One other thing I noticed: Now the piler-smtp process is started automatically.
This messed up the mail reception (postfix).

I commented out the start sequence in /etc/init.d/rc.piler but can I change a config file somewhere to stop it from starting?
Mails come into the archive via pilerimport via an IMAP account only.

Thanks for your help!

Happy Easter! :-)

@jsuto
Copy link
Owner
jsuto commented Apr 19, 2025

On ubuntu you can start piler and piler-smtp daemons using systemctl or the rc.piler script. Unfortunately your best bet is to fix the used script, and comment out piler-smtp and probably you don't need piler daemon either if using pilerimport only. So in that case just don't start the daemons. Btw. piler-smtp can listen on a specific interface or even port as well, eg. 127.0.0.1:2525.

If you want some other name from the actual hostname, then fix the code here: https://github.com/jsuto/piler/blob/master/webui/model/health/health.php#L246-L251

@fs2810
Copy link
Author
fs2810 commented Apr 21, 2025

Hi Janos,
I hardcoded the hostname in health.php, but the behavior did not change.

  #$hostname = exec("hostname -f");
  $hostname = "archive.designwindmill.com";

What I did notice: The health page now does now show "archive.designwindmill.com" instead of "mail.designwindmill.com".

  1. I open https://archive.designwindmill.com in my Browser
  2. I login as admin@local
  3. The browser redirects to https://mail.designwindmill.com/index.php?route=health/health (Which does not work, obviously)

If I manually change the URL to https://archive.designwindmill.com/index.php?route=health/health (replacing mail with archive), it works

The same thing happens when I change the password of a user.
You edit the user fields, click modify and the browser redirects to https://mail.designwindmill.com

I'm stumped and still hoping for a Easter miracle ;-)

@fs2810
Copy link
Author
fs2810 commented Apr 21, 2025

Argh ;-)

I think I got it now.
Looks like config-site.php has to reside at /usr/local/etc/piler and not at /var/piler/www ....

I think that is created by make postinstall, right?
It definitely had the wrong SITE_URL in there, probably used hostname -f to determine it...

@jsuto
Copy link
Owner
jsuto commented Apr 21, 2025

OK, I'm glad that it works by now.

@fs2810
Copy link
Author
fs2810 commented Apr 21, 2025

Sorry to bother you again, but I used the Timestamping feature with the Piler 1.2.
I did it with a cronjob running sign.php at specific times (to not overwhelm the TSA).

sign.php seems to be "deprecated" and I cannot find any mention of Timestamping in the Installation docs.
Is that feature gone/deprecated?

I found TrustedTimestamps.php but this seems to only provide the "class" for the action...
Haven't tried calling that directly...

Can I enable Timestamping with Piler 1.4.7?

@github-actions GitHub Actions
Copy link
github-actions bot commented May 6, 2025

This issue is stale because it has been open for 14 days with no activity.

@github-actions github-actions bot added the stale label May 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jsuto jsuto

Labels
stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jsuto @fs2810
0