Re: Strigi Loaded

2008-07-24

Sebastian is apologizing to me for writing code. This is a bit strange and I need to reply to his post. The discussion below is really more suited for a mailing list, but since I did not start it I have no choice in the matter.

First of all, why do you apologize for writing code? That is no reason for apology. What you could apologize for is not what you present, but how you present it. You have written some code and talk about it like it is the solution for a big problem. You fail, however, to clearly analyze these problems. The title of the blog is “Strigi Reloaded - The Answer to all our Problems?”. This is a title which is worthy of the Bild newspaper. Any title with a question mark behind a sentence which is not a question is a sign that the title is meaningless agitation. It is a method to gather support of fanboys in the absense of real arguments and to sow confusion. I’m not saying that this was your intent, but I am saying that using question marks in titles is a sign of bad writing.

But let’s talk content and not form. What are these Problems that deserve a capital P? The first two problems you observe are untrue. You say that strigidaemon has no method to suspend and resume indexing. However, it does. The DBus API of strigidaemon allows you to start and stop indexing. Second, you say that strigidaemon does not report what it is doing. This is also not true; it does report this. In fact, this is even part of the Xesam interface. The reporting is not very elaborate. It says ‘Idle’ or ‘Indexing’, but this can be extended.

Not such a good start for a blog in which you are announcing a fork of a desktop search daemon. The list continues with things you have done to integrate indexing more tightly with the desktop. This is a good and noble cause. But the way you approach it is wrong. You mention that you use KDirWatch, Solid, KFreeDiskSpace, KPassivePopup to blend into the KDE user experience. That’s very nice but no reason to fork a perfectly fine program. There is an elegant solution which I mention below.

The most serious problem you observe is watching files for changes. This is a major problem for all desktop search programs, but KDirWatch is not the solution.

KDirWatch is not suited for watching large amounts of files. It is a wrapper around inotify (default), FAM and QFileSystemWatcher (used on Windows. QFileSystemWatches is a small version of KDirWatch written by Trolltech. It only emits two signals: directoryChanged(const QString& path) and fileChanged(const QString& path).) To use this wrapper class you must link libkio which depends on libQtSvg and libQtGui (and all of X).`

What we need is a small crossplatform library for monitoring files. A dedicated library that wraps the native implementations for each of these functions in a clean API. A library that would work on any operating system and that would not link to anything that is not strictly necessary. A library that falls back cleanly to configurable polling when the native implementations are not suitable and that is modest in its use of memory and CPU.

It is unfortunate that you use classes that are part of the desktop environment for such low level functionality. This functionality should be in POSIX. Since it is not, it should be in a small helper library.

In conclusion, I like that you are putting effort in improving the desktop search experience. You’re beying paid for it after all. But forking a perfectly fine daemon because you want some improvements is not the way to go. I am happy that you managed to build something so quickly: it proves that libstreamanalyzer is awesome.

Now this is the right solution: write a KDE service that uses Solid, KFreeDiskSpace and KPassivePopup to control strigidaemon over DBus. In that way, you do not need to fork it. I am happy to add the required DBus calls to strigidaemon. In this way we can keep strigidaemon desktop independent, small and fast and we avoid a fork and months of flames. And as an added bonus, you should end up with a nice API for Xesam.

Comments

Follow up...

Obviously you feel personally insulted. Well, that is a pity. I will not comment much on your bashing of my writing. That is just ridiculous.

As for the technical side: whether this is the wrong approach is a matter of opinion. I have tried writing a service to control strigidaemon before and failed. This may be due to issues with strigidaemon or with my skills. Whatever it is, the presented solution was the only one that I could come up with to solve the problems we had. Problems that did not emerge recently, but have been around for a long time.

Some random notes:

You claim to know the better solution. Yet, the "better" solution does not exist and I will not spend my time implementing it. It would take much longer and have (as mentioned before) no benefit that I can count as valid.

Maintenance

Hi Sebastian,

Maybe I'm naive and my vision is too simplistic but to me it looks like you have two choices:

a) Fix stop/resume of strigidaemon, and add those DBus calls to strigidaemon (it's opensource and it's in the KDE repository, where you can commit). You probably never need to care about this again.

b) Fork strigidaemon and maintain this fork

Isn't b) going to be really expensive in terms of maintenance and keep up to date with libstreamsanalyzer and the plugins? Won't the maintenance of this fork keep you from working on Nepomuk?

Not quite

The problem is that I don't see a reason to (re)implement stuff without using the amazing power of Qt/KDE. Everything is that much simpler. And much faster to code. Learning the inner workings of strigidaemon already took enough of my time.

As for "maintaining" a fork: the actual lines of code that strigidaemon and my service "share" (in that they do the same thing) sum up to about 50 or something. So what has to be maintained mostly is the stuff around that, the features strigidaemon does not support (yet maybe).

>> The problem is that I

>> The problem is that I don't see a reason to (re)implement stuff without using the amazing power of Qt/KDE.

Bingo. All this cross-desktop stuff is cool in concept (yay share code) but realistically it means ditching the unique advantage that KDE has (awesome api and awesome toolkit) just to _potentially_ appease people from other desktops.

If other desktops were inclined to support strigidaemon, then that might be worth the effort, but realistically, they've already invented their own systems (tracker, beagle) and there really is not much chance for any gnome distribution to adopt strigi (or any Windows system for that matter). Try to make it generic to every system and it lacks the integration to actually succeed on any of them.

Tracker will always be ahead in terms of seamless integration with Gnome, Windows search will always integrate best with Windows, and Spotlight is already far ahead of everyone at integration. So what's the point of having a cross-platform system when the problem is solved on other platforms? Better to take advantage of KDE's api and make it the best possible engine for KDE desktops.

Better to take advantage of KDE's api

"So what's the point of having a cross-platform system when the problem is solved on other platforms? Better to take advantage of KDE's api and make it the best possible engine for KDE desktops"

Does this make it harder to build/maintain a Windows/Mac port? Because if it does then that is an important issue that really does need to be taken into consideration.

I'm looking forward to a proper and full Windows port. But if things like this get in its way then in the long run you're harming KDE's development and user base.

I can see both sides

I can see both of your points. Trueg has some very good points however:

* strigidaemon, it seems, is pretty small by itself. all the actual work is elsewhere
* you have different goals - Trueg wants something more tightly integrated etc.Jos wants something small and platform independant.

Just from these two points, it seems that trueg took the right path in just writing another program that replaces strigidaemon.

You could even have both programs in the same source tree - no? Would that be a good compromise for you Jos?

What is the underlying

What is the underlying problem that KDirWatch isn't suited for watching large amounts of files? It indeed seems to be redundant to recode a strigi wrapper around different filewatching implementations if KDirWatch is essentially that. But otherwise I see the architectural advantage of your approach.
Couldn't the work done by Sebastian still be reused anyway, by replacing his direct calls to libstreamanalyzer with dbus calls to strigidaemon? (and his additional information retrieval could be copied to strigidaemon, or are they kde-dependant also?).

All except the

All except the libstreamanalyser calls is KDE-dependant. That is why coding it was so simple, why maintaining it is so simple, why it works very well.