Discuss Repairing data and data structures

From semantic-mediawiki.org

Contents

Thread titleRepliesLast modified
Could not get URL after 3 tries.321:49, 11 July 2019
rebuildData.php gone?221:34, 4 March 2016
Data repair and upgrade after update513:08, 16 October 2015
rebuildData.php freezes at certain page IDs414:06, 21 January 2015
My property list is mainly empty - what happened and how do I fix it?1409:16, 22 August 2014
Names still correct ?415:26, 13 May 2014

Could not get URL after 3 tries.

I successfully added this to cron.

cd "/opt/smw/core/extensions/SemanticMediaWiki/maintenance"; export MW_INSTALL_PATH="/opt/smw/core" && nice -10 php SMW_refreshData.php -v -n 1000 --startidfile /var/run/smw/smwrefresh-startid > /var/log/smw/smwrefreshdata.log

But for some IDs, I get the error - "Could not get URL after 3 tries."

Should I be concerned about this error? Or will runJobs ultimately take care of them too?

Thanks!

18:38, 24 July 2013
  1. Nice -10 is high priority. Do you really mean this?
05:49, 25 July 2013

Thanks Alex for catching that. I meant Nice 10.

16:05, 29 August 2013
 

This was actually never answered. I just ran into this too. This comes from the External Data extension unsuccessfully trying to fetch data from the external source, which is no longer there. See system message "externaldata-db-could-not-get-url". Thus this has nothing to do with SMW. I just pops up during the data rebuild since the respective pages are being parsed.

21:12, 11 July 2019
 

rebuildData.php gone?

I am trying with SMW 2.3.1 (on MW 1.26.2). rebuildData.php does not seem to be available anymore? Has that been replaced by rebuildall.php? Should one (I?) update this Help page correspondingly?

Unfortunately, running rebuildall.php (after update.php) on my imported existing data gets this error while refreshing link tables:

PHP Catchable fatal error:  Argument 1 passed to SMW\Subobject::__construct() must be an instance of Title, null given, called in /home/mw_crm7dev/htdocs/w/extensions/SemanticMediaWiki/src/Query/ProfileAnnotator/QueryProfileAnnotatorFactory.php on line 29 and defined in /home/mw_crm7dev/htdocs/w/extensions/SemanticMediaWiki/includes/Subobject.php on line 42
PHP Stack trace:
PHP   1. {main}() /home/mw_crm7dev/htdocs/w/maintenance/rebuildall.php:0
PHP   2. require_once() /home/mw_crm7dev/htdocs/w/maintenance/rebuildall.php:67
PHP   3. RebuildAll->execute() /home/mw_crm7dev/htdocs/w/maintenance/doMaintenance.php:103
PHP   4. RefreshLinks->execute() /home/mw_crm7dev/htdocs/w/maintenance/rebuildall.php:60
PHP   5. RefreshLinks->doRefreshLinks() /home/mw_crm7dev/htdocs/w/maintenance/refreshLinks.php:58
PHP   6. RefreshLinks::fixLinksFromArticle() /home/mw_crm7dev/htdocs/w/maintenance/refreshLinks.php:175
PHP   7. AbstractContent->getSecondaryDataUpdates() /home/mw_crm7dev/htdocs/w/maintenance/refreshLinks.php:248
PHP   8. AbstractContent->getParserOutput() /home/mw_crm7dev/htdocs/w/includes/content/AbstractContent.php:230
PHP   9. WikitextContent->fillParserOutput() /home/mw_crm7dev/htdocs/w/includes/content/AbstractContent.php:497
PHP  10. Parser->parse() /home/mw_crm7dev/htdocs/w/includes/content/WikitextContent.php:331
PHP  11. Parser->internalParse() /home/mw_crm7dev/htdocs/w/includes/parser/Parser.php:439
PHP  12. Parser->replaceVariables() /home/mw_crm7dev/htdocs/w/includes/parser/Parser.php:1239
PHP  13. PPFrame_DOM->expand() /home/mw_crm7dev/htdocs/w/includes/parser/Parser.php:3342
PHP  14. Parser->braceSubstitution() /home/mw_crm7dev/htdocs/w/includes/parser/Preprocessor_DOM.php:1177
PHP  15. PPFrame_DOM->expand() /home/mw_crm7dev/htdocs/w/includes/parser/Parser.php:3665
PHP  16. Parser->braceSubstitution() /home/mw_crm7dev/htdocs/w/includes/parser/Preprocessor_DOM.php:1177
PHP  17. PPFrame_DOM->expand() /home/mw_crm7dev/htdocs/w/includes/parser/Parser.php:3442
PHP  18. Parser->braceSubstitution() /home/mw_crm7dev/htdocs/w/includes/parser/Preprocessor_DOM.php:1177
PHP  19. Parser->callParserFunction() /home/mw_crm7dev/htdocs/w/includes/parser/Parser.php:3523
PHP  20. call_user_func_array() /home/mw_crm7dev/htdocs/w/includes/parser/Parser.php:3789
PHP  21. SMW\ParserFunctionFactory->SMW\{closure}() /home/mw_crm7dev/htdocs/w/includes/parser/Parser.php:3789
PHP  22. SMW\ShowParserFunction->parse() /home/mw_crm7dev/htdocs/w/extensions/SemanticMediaWiki/src/ParserFunctionFactory.php:273
PHP  23. SMW\AskParserFunction->parse() /home/mw_crm7dev/htdocs/w/extensions/SemanticMediaWiki/includes/parserhooks/ShowParserFunction.php:67
PHP  24. SMW\AskParserFunction->doFetchResultsForRawParameters() /home/mw_crm7dev/htdocs/w/extensions/SemanticMediaWiki/includes/parserhooks/AskParserFunction.php:120
PHP  25. SMW\AskParserFunction->createQueryProfile() /home/mw_crm7dev/htdocs/w/extensions/SemanticMediaWiki/includes/parserhooks/AskParserFunction.php:185
PHP  26. SMW\Query\ProfileAnnotator\QueryProfileAnnotatorFactory->newJointProfileAnnotator() /home/mw_crm7dev/htdocs/w/extensions/SemanticMediaWiki/includes/parserhooks/AskParserFunction.php:198
PHP  27. SMW\Subobject->__construct() /home/mw_crm7dev/htdocs/w/extensions/SemanticMediaWiki/src/Query/ProfileAnnotator/QueryProfileAnnotatorFactory.php:29

Any hints?

18:43, 4 March 2016

rebuildData.php is still present in current master and so it should at your end. rebuildall.php is a core script that currently does not seem to play with SMW.

19:00, 4 March 2016
 

> PHP Catchable fatal error: Argument 1 passed to SMW\Subobject::__construct() must be an instance of Title, null given,

See [0] for the analysis of the issue.

> I am trying with SMW 2.3.1 (on MW 1.26.2). rebuildData.php

Is expected to work as described by the documentation (otherwise our tests would fail and you would see a red status alert on [1]).

[0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1318

[1] https://github.com/SemanticMediaWiki/SemanticMediaWiki

Cheers

21:34, 4 March 2016
 

Data repair and upgrade after update

I am getting fatal PHP errors while updating the SMW database after an upgrade...

2015-10-15 13:23:13 SMW\UpdateJob MACBEATH_Jimmy (id=57732) STARTING PHP Fatal error: Class 'ParamDefinition' not found in /var/www/wiki/extensions/SemanticMediaWiki/includes/query/SMW_QueryProcessor.php on line 664


And....


PHP Fatal error: Class 'ParamDefinition' not found in /var/www/wiki/extensions/SemanticMediaWiki/includes/query/SMW_QueryProcessor.php on line 664


PHP Fatal error: Class 'ParamProcessor\StringParam' not found in /var/www/wiki/extensions/Maps/includes/ServiceParam.php on line 18


Several similar errors revolving around this ParamProcessor, I installed Validator and set it up in LocalSettings, bit worrying.


Is this a known problem?


Debian Squeeze

PHP 5.3.3-7 MySQL 5.1.73-1

15:15, 15 October 2015

> I am getting fatal PHP errors while updating the SMW database after an upgrade...

From where to where did you upgrade?

Did you follow the steps as outlined in one of the mentioned upgrade guides [0]?

> PHP Fatal error: Class 'ParamDefinition' not found

Please see [1].

[0] https://semantic-mediawiki.org/wiki/Help:Installation

[1] https://semantic-mediawiki.org/wiki/Help:Installation#Troubleshooting

15:29, 15 October 2015

I upgraded from MW 1.22 - 1.25 and ran the database upgrade, SMW I upgraded to the latest version from 1.8.5.3, so quite an old version. I followed the instructions re: Composer and now I get:

MediaWiki requires the PSR-3 logging library to be present. This library is not embedded directly in MediaWiki's git repository and must be installed separately by the end user. Please see mediawiki.org for help on installing the required components.

Having to go AFK now, bit surprised how difficult this process has been, I had it up and running quite well before.

16:48, 15 October 2015

> MediaWiki requires the PSR-3 logging library to be present. This library is not embedded directly in MediaWiki's git repository and must be installed separately by the end user. Please see mediawiki.org for help on installing the required components.

This message is generated by MW itself and has not really something to do with SMW (whether you would have installed it via Composer or the Tarball).

In regards to the message, if you run composer update from the MW base directory then it should install the libraries as requested by MW.

> Having to go AFK now, bit surprised how difficult this process has been, I had it up and running quite well before.

Looking at [0, 1], I personally find the process fairly simple but it varies with experience. Now (as of 1.25), that MW uses Composer itself to manage external libraries, SMW is just one component to leverage the infrastructure necessary in handling external dependencies.

[0] https://vimeo.com/82255034 [1] https://vimeo.com/127356533

17:05, 15 October 2015
 

If you think the instructions are not as easy to follow as expected or need adaption in order to make them more self-explanatory (one reason why we have the videos), we are more than happy to see them improved based on user experience.

Please don't hesitate to make adjustments (after all it is a wiki) so other users can participate on the gained experience.

17:15, 15 October 2015

I'm not sure I'm currently qualified to give advice given how badly I seemed to break my install! I think the problem was the classic one of following outdated advice and relying on Google search too much.

I started with a fresh install following the video and found Composer to in fact be very good, though I think it would be helpful to be a bit clearer about it's function. I really had no idea what it was even supposed to be doing for quite a while.

I think I'm back on track now anyway, lesson learned that maybe I should keep more abreast of changes (just need to find the time!).

Thanks, S

13:07, 16 October 2015
 
 
 
 

rebuildData.php freezes at certain page IDs

I'm having an issue with a query which generates Unexpected non-MediaWiki exception encountered, of type "RuntimeException" error with a simple semantic search based on a category. I'm trying to refresh the semanic data, but when I run rebuildData.php -v the script first froze at ID 11. I restarted the script at ID 11 and it froze again. I edited the page to remove the semantic content (it was present for testing only) and the script continued and then froze at ID 24, which does not contain any semantic data. Again, the script froze when restarted at ID 24. I executed rebuildData.php -v -f which deleted and recreated the tables fine, but froze again at ID 24 during the data refresh. I cannot figure out what is causing the script to freeze.

Configuration:

  • MediaWiki 1.23.7 (8136f24)
  • PHP 5.4.20 (cgi-fcgi)
  • MySQL 5.6.14
  • Semantic MediaWiki 2.0
15:37, 5 January 2015

Sorry but above information are not sufficient to make any proposal on how to solve the issue. It would be best if you can provide a stack trace and/or the query that is involved on the mentioned page.

Furthermore, you should list all SMW related extensions being used that may or may not cause the described issue.

If you have a stack trace then it is recommended to post the information on the issue tracker.

17:12, 5 January 2015

Thanks - I was also having problems with the MediaWiki runjobs.php hanging on execution - Support thread. I thought the two issues may be related - and it turns out they were.

  • Disabling caching (CACHE_NONE) allowed both maintenance scripts to run to completion.
13:47, 21 January 2015

For the sake of completion, above run-time error was referring to the Jobclass (copied from the thread).

0.0312   1.5M  Start command line script runjobs.php
0.0468   1.8M  [caches] main: WinCacheBagOStuff, message: WinCacheBagOStuff, parser: WinCacheBagOStuff
0.0624   3.0M  [caches] LocalisationCache: using store LCStoreCDB
0.0780   4.0M  Fully initialised
0.1092   5.0M  Connected to database 0 at 127.0.0.1
0.1092   5.0M  Query mwdev (1) (slave): SELECT /* JobQueueDB::doGetSiblingQueuesWithJobs  */  DISTINCT job_cmd  FROM `job`   WHERE job_cmd IN ('refreshLinks','refreshLinks2','htmlCacheUpdate','sendMail','enotifNotify','fixDoubleRedirect','uploadFromUrl','AssembleUploadChunks','PublishStashedFile','null')   
0.1092   5.0M  LoadBalancer::reuseConnection: this connection was not opened as a foreign connection
0.1092   5.0M  Query mwdev (2) (slave): SELECT /* JobQueueDB::claimRandom  */  job_id,job_cmd,job_namespace,job_title,job_timestamp,job_params,job_random,job_attempts,job_token,job_token_timestamp,job_sha1  FROM `job`   WHERE job_cmd = 'refreshLinks' AND job_token = ''  LIMIT 88,1  
0.1092   5.0M  Query mwdev (3) (slave): SELECT /* JobQueueDB::claimRandom  */  job_id,job_cmd,job_namespace,job_title,job_timestamp,job_params,job_random,job_attempts,job_token,job_token_timestamp,job_sha1  FROM `job`   WHERE job_cmd = 'refreshLinks' AND job_token = '' AND (job_random >= '18174153')  ORDER BY job_random ASC LIMIT 1  
0.1092   5.0M  DatabaseBase::query: Writes done: UPDATE `job` SET job_token = 'X'
0.1092   5.0M  Query mwdev (4) (slave): UPDATE /* JobQueueDB::claimRandom  */  `job` SET job_token = '2d61852a9a7731265203632d5f49e0f1',job_token_timestamp = '20150108174628',job_attempts = job_attempts+1 WHERE job_cmd = 'refreshLinks' AND job_id = '235' AND job_token = ''
0.1716   5.0M  LoadBalancer::reuseConnection: this connection was not opened as a foreign connection
13:57, 21 January 2015
 

I'm not sure what the CACHE_NONE has to do with rebuildData but maybe we try to add a --no-cache option [0].

[0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/749

14:06, 21 January 2015
 
 
 

My property list is mainly empty - what happened and how do I fix it?

I noticed that my in-line queries weren't working. So I looked at some pages that had set the properties - the pages were OK, but the properties seemed to have been ignored.

When I list all the properties, they are there, but the show (0 users).

I've tried running rebuildData.php on all IDs, and touching LocalSettings.php - but it remains the same.

How do I get it to re-link all the properties again?

08:53, 20 August 2014

This very much sounds like this worry. Just answered to this thread on the list though it will probably not help the cause. :(

09:31, 20 August 2014

Thank you for the suggestion - my problem is similar, but, fortunately, not quite the same.

The properties are there, and they don't show 'invalid content' - just no references.

I'm trying this at the moment, as recommended:

running this three times to rebuild the properties:

php rebuildData.php -ftpv

Now running this twice (maybe three times, it doesn't take too long) to rebuild them:

php rebuildData.php -v

Then I'm going to do a normal 'update', remove the cache and hope..

If that doesn't work, I'm not sure what the next step is.

09:46, 20 August 2014

I don't know if this is connected. Mainly the rebuildData runs through OK, but there are these:

PHP Notice: Undefined index: REMOTE_ADDR in /home/kchcl200/public_html/wiki/extensions/userInfo/userInfo.php on line 43 PHP Notice: Undefined index: REMOTE_ADDR in /home/kchcl200/public_html/wiki/extensions/userInfo/userInfo.php on line 58 PHP Notice: Undefined variable: os in /home/kchcl200/public_html/wiki/extensions/userInfo/userInfo.php on line 83 PHP Notice: Undefined variable: bw in /home/kchcl200/public_html/wiki/extensions/userInfo/userInfo.php on line 133 PHP Notice: Undefined variable: bw in /home/kchcl200/public_html/wiki/extensions/userInfo/userInfo.php on line 158


and:

PHP Warning: preg_split() expects parameter 3 to be long, string given in /home/kchcl200/public_html/wiki/extensions/ParserFunctions/ParserFunctions_body.php on line 811 (1990) Processing ID 1990 ... PHP Warning: preg_split() expects parameter 3 to be long, string given in /home/kchcl200/public_html/wiki/extensions/ParserFunctions/ParserFunctions_body.php on line 811

I'll have to fund page ID 1989 and1990 and see what's going on there.

09:52, 20 August 2014

I've found the pages - it was a very complicated template - that I don't need. I've deleted it and I hope things will get better.

10:22, 20 August 2014

The first lot of notices comes from the unmaintained UserInfo extension, so this is not connected to the problem. Indeed sometimes a template may contain syntax which breaks things or triggers warnings. Usually some falsely applied syntax within the template is the cause. This happens once in a while, but I think that this is not really connected to the issue.

12:46, 20 August 2014
 

I've narrowed this down.

The problem is that properties are not created and don't have their count increased when a page is saved.

They are recognised as properties on the page.

Categories are saved.

I've put the versions at the bottom - but PHP and MySQL are at different versions - the wikis are both running the same SMW (also listed at the bottom)

This is the test I've run. The first wiki is the original wiki. The second is the copy of it running on another machine.

Test of two wiki's:

First wiki:


http://kchclinics.com/wiki/index.php?title=Sandpit:Property_test

==Property Test==
__NOEDITSECTION__

[[Has type::Text]]
[[Has colour::Orange]]
[[Location::Ouagadougou]]
[[Population::1,475,223]]
[[Capital of::Burkina Faso]]
[[Category:City]]
[[Category:African]]

===================
Create Page:

Left hand column shows 'Browse properties'
Items show up as:

Blue - Text
Red - Orange 1,475,223 Burkina Faso - City African
Black - Ouagadougou

================================
Browse property page shows:

Sandpit:Property test
Capital of	Burkina Faso 
Has colour	Orange 
Location	Ouagadougou  
Population	1,475,223 
Categories	City  , African  
Modification date	22 August 2014 07:34:45  
Has type	Text  

---
Categories page shows African & City
---
Properties Page:

Shows:

Capital of - 1 use
Has Type   - 4 uses
Has colour - 1 use
Location   - 75 uses
Population - 9 uses

---------------------------------------------------------
Second Wiki

===================
Create Page:

Left hand column DOES NOT show 'Browse properties'
Items show up as:

Blue - Text
Red - Orange 1,475,223 Burkina Faso - City African
Black - Ouagadougou
---
Properties Page:

Does NOT show:

Capital of
Has colour
Population

Does show:

Has Type - but only 1 use
Location - but '0' uses


---
Categories page shows African & City



================================================
The physical set up is:

First Machine

Apache version	2.2.27
PHP version	5.3.28
MySQL version	5.5.34-cll-lve
Architecture	x86_64
Operating system	linux
Dedicated IP Address	69.28.199.12
Path to sendmail	/usr/sbin/sendmail
Path to Perl	/usr/bin/perl
Perl version	5.10.1
Kernel version	2.6.32-531.17.1.lve1.2.60.e

Second Machine

Apache version	
PHP version	5.5.9
MySQL version	5.5.38
Architecture	x86_64
Operating system	linux
Kernel version	Ubuntu 14.04.1

=== 
SMW versions Machine One:

Semantic Compound Queries (Version 0.3.4)	A parser function that displays multiple semantic queries at the same time	Yaron Koren
Semantic Drilldown (Version 1.4)	A drilldown interface for navigating through semantic data	Yaron Koren and others
Semantic Extra Special Properties (Version 1.2.1)	Adds some extra special properties to all pages	Leo Wallentin, James Hong Kong and others
Semantic Forms (Version 2.7)	Forms for adding and editing semantic data	Yaron Koren, Stephan Gambke and others
Semantic Forms Inputs (Version 0.7)	Additional input types for Semantic Forms	Stephan Gambke and others
Semantic Image Input (Version 0.3.0 alpha)	Adds an image input that pulls images from Wikipedia to Semantic Forms	Jeroen De Dauw
Semantic Internal Objects (Version 0.7.6)	Setting of internal objects in Semantic MediaWiki	Yaron Koren
Semantic Maps (Version 3.0.4)	Provides the ability to view and edit coordinate data stored with the Semantic MediaWiki extension	Jeroen De Dauw
Semantic MediaWiki (Version 1.9.2)	Making your wiki more accessible - for machines and humans (online documentation)	Markus Krötzsch, Jeroen De Dauw, James Hong Kong and others
Semantic Result Formats (Version 1.9.1)	Additional result formats for Semantic MediaWiki queries	James Hong Kong, Stephan Gambke, Jeroen De Dauw, Yaron Koren and others
Semantic Watchlist (Version 0.2.2)	Lets users be notified of specific changes to Semantic MediaWiki data	Jeroen De Dauw for WikiWorks
SemanticFormsSelect (Version 1.1)	Generate a select field in Semantic Form which values are from query	Jasonzhang
Special pages
Access Log (Version 2.0)	Logs user access to MediaWiki	Aleksandar Bojinovic, Peter Kin-Fong Fong
CheckUser (Version 2.3)	Grants users with the appropriate permission the ability to check user's IP addresses and other information	Tim Starling and Aaron Schulz
EmailPage (Version 2.3.0, 2013-11-30)	Send rendered HTML page to an e-mail address or list of addresses using phpmailer	User:Nad
MaintenanceShell (Version 0.4.0)


===
SMW versions machine Two

Semantic Compound Queries (Version 0.3.4)	A parser function that displays multiple semantic queries at the same time 	Yaron Koren
Semantic Drilldown (Version 1.4)	A drilldown interface for navigating through semantic data 	Yaron Koren and others
Semantic Extra Special Properties (Version 1.2.1)	Adds some extra special properties to all pages 	Leo Wallentin, James Hong Kong and others
Semantic Forms (Version 2.7)	Forms for adding and editing semantic data 	Yaron Koren, Stephan Gambke and others
Semantic Forms Inputs (Version 0.7)	Additional input types for Semantic Forms 	Stephan Gambke and others
Semantic Image Input (Version 0.3.0 alpha)	Adds an image input that pulls images from Wikipedia to Semantic Forms 	Jeroen De Dauw
Semantic Internal Objects (Version 0.7.6)	Setting of internal objects in Semantic MediaWiki 	Yaron Koren
Semantic Maps (Version 3.0.4)	Provides the ability to view and edit coordinate data stored with the Semantic MediaWiki extension 	Jeroen De Dauw
Semantic MediaWiki (Version 1.9.2)	Making your wiki more accessible - for machines and humans (online documentation) 	Markus Krötzsch, Jeroen De Dauw, James Hong Kong and others
Semantic Result Formats (Version 1.9.1)	Additional result formats for Semantic MediaWiki queries 	James Hong Kong, Stephan Gambke, Jeroen De Dauw, Yaron Koren and others
Semantic Watchlist (Version 0.2.2)	Lets users be notified of specific changes to Semantic MediaWiki data 	Jeroen De Dauw for WikiWorks
SemanticFormsSelect (Version 1.1)	Generate a select field in Semantic Form which values are from query 	Jasonzhang
Special pages
Access Log (Version 2.0)	Logs user access to MediaWiki 	Aleksandar Bojinovic, Peter Kin-Fong Fong
CheckUser (Version 2.3)	Grants users with the appropriate permission the ability to check user's IP addresses and other information 	Tim Starling and Aaron Schulz
EmailPage (Version 2.3.0, 2013-11-30)	Send rendered HTML page to an e-mail address or list of addresses using phpmailer 	User:Nad
MaintenanceShell (Version 0.4.0)	Wiki interface for maintenance scripts 	Andrew Fitzgerald and Timo Tijhof
Parser hooks
and http://www.shadowritter.net Chklist (Version 0.1.1)	Add checkbox on your wiki page 	Shadowritter
DynamicPageList (Version 2.01)	A highly flexible report generator for MediaWikis 	Gero Scholz
GetUserName (Version 1.0)	Get current user's name. 	Ejcaputo
Maps (Version 3.0.1)	Enables embedding of dynamic maps into wiki pages, geocoding of addresses and other geographical operations. 	Jeroen De Dauw
MsUpload (Version 9.5.1)	Enables easy uploads (when available also via drag&drop) for one or multiple files simultaneously 	info@ratin.de | Ratin
ParserFunctions (Version 1.5.1)	Enhance parser with logical functions 	Tim Starling, Robert Rohde, Ross McClure and Juraj Simlovic
Strtotime (Version 1.0)	Represents time as a string containing a US English date format 	Xavier Atero
SubPageList (Version 1.1)	Allows to list and count subpages 	Jeroen De Dauw
Variables
UserInfo (Version 1.0)
09:16, 22 August 2014
 

Allowing others to join in: related thread on the mailing list

12:48, 20 August 2014
 

I have a feeling that the advice should, maybe, have been to run:

php rebuildData.php -f

once - because this rebuilds everything from scratch.

Then this a few times:

php rebuildData.php -vcpt

to rebuild all the tables

then to build the remaining table with:

php rebuildData.php -v

At least I'm trying this now - it didn't seem to make sense to run with the -f option more than once.

Am I missing something?

13:16, 20 August 2014

Ah, new options to the script. Hmm ..., did I document this? No I would do the classic php rebuildData.php -fv once and when php rebuildData.php -v once. I do not think that what you are doing will work / is meant to work. If you want to rebuild specific tables thereafter you may use all these "cpt" options.

14:15, 20 August 2014

I'd really like to know how to understand what happened - so it doesn't happen again.... as well, of course, fixing it now.

The -v option is, as I understand it, supposed to visit each page, check if there are any properties on it, and, if there are, then recreate them. This should bring the properties back.

The fact that it hasn't, after running it a number of times, is a concern - where should I be looking to see what it's not doing?

The properties themselves are there, and not showing red any more, so it seems to have sorted out the table of properties itself, just not the pointers to where the properties are used.

that's why I thought that the -cpt options, particularly the -p option, might help.

Is there an error log somewhere that might help show what's not working?

14:22, 20 August 2014
 
 
 
 

I'm still trying to solve this.

I still have the property list showing all properties as unused -- such as:

Agenda of type Page (0 uses)

To try to fix this, I took all the property names and created a scratch page which declared them as properties. Like this:

[[Agenda::test of property]]

When I save the page, it looks as if it has understood this:

-some of these are accepted and show 'Test_of_property' in black since it is of type 'Text' - some show that they're understood and the property page doesn't exist, by showing read - some give an error '!' because the property is incorrectly set - eg. 'Test_of_property' is not type 'Date' - Some show blue and are a link to 'Test_of_property' (which doesn't, of course, exist.

So it looks as if that part of SMW is working.

However, with a page with properties, if a wiki is working, on the left side-bar, you get the 'Browse Properties' option. When you click on that, you see all the properties.

On my wiki, that option does not appear.

So, in effect, SMW is treating the page as if it has no properties on it.

This seems to be the underlying problem.

Any suggestions about what might be causing this?

07:57, 21 August 2014

I don't know if this will help. I'm posting this here, rather than the mailing list, because it is long and probably only of interest to a specialist. I thought that it might help diagnose the problem.

It's the SMW configuration output:


{

   "smwgScriptPath": "/wiki/extensions/SemanticMediaWiki",
   "smwgIP": "/home/kchcl200/public_html/wiki/extensions/SemanticMediaWiki/",
   "smwgDefaultStore": "SMWSQLStore3",
   "smwgSparqlDatabase": "SMWSparqlDatabase",
   "smwgSparqlQueryEndpoint": "http://localhost:8080/sparql/",
   "smwgSparqlUpdateEndpoint": "http://localhost:8080/update/",
   "smwgSparqlDataEndpoint": "http://localhost:8080/data/",
   "smwgSparqlDefaultGraph": "",
   "smwgHistoricTypeNamespace": false,
   "smwgNamespaceIndex": 100,
   "smwgShowFactbox": 1,
   "smwgShowFactboxEdit": 3,
   "smwgToolboxBrowseLink": true,
   "smwgInlineErrors": true,
   "smwgUseCategoryHierarchy": true,
   "smwgCategoriesAsInstances": true,
   "smwgLinksInValues": true,
   "smwgDefaultNumRecurringEvents": 100,
   "smwgMaxNumRecurringEvents": 500,
   "smwgBrowseShowInverse": false,
   "smwgBrowseShowAll": true,
   "smwgSearchByPropertyFuzzy": true,
   "smwgTypePagingLimit": 200,
   "smwgConceptPagingLimit": 200,
   "smwgPropertyPagingLimit": 25,
   "smwgQEnabled": true,
   "smwgQMaxLimit": 10000,
   "smwgIgnoreQueryErrors": true,
   "smwgQSubcategoryDepth": 10,
   "smwgQEqualitySupport": 1,
   "smwgQSortingSupport": true,
   "smwgQRandSortingSupport": true,
   "smwgQDefaultNamespaces": null,
   "smwgQComparators": "<|>|!~|!|~|≤|≥|<<|>>",
   "smwStrictComparators": false,
   "smwgQMaxSize": 12,
   "smwgQMaxDepth": 4,
   "smwgQFeatures": 63,
   "smwgQDefaultLimit": 50,
   "smwgQMaxInlineLimit": 500,
   "smwgQPrintoutLimit": 100,
   "smwgQDefaultLinking": "all",
   "smwgQConceptCaching": 1,
   "smwgQConceptMaxSize": 20,
   "smwgQConceptMaxDepth": 8,
   "smwgQConceptFeatures": 59,
   "smwgQConceptCacheLifetime": 1440,
   "smwgResultFormats": {
       "table": "SMW\\TableResultPrinter",
       "list": "SMW\\ListResultPrinter",
       "ol": "SMW\\ListResultPrinter",
       "ul": "SMW\\ListResultPrinter",
       "broadtable": "SMW\\TableResultPrinter",
       "category": "SMWCategoryResultPrinter",
       "embedded": "SMWEmbeddedResultPrinter",
       "template": "SMW\\ListResultPrinter",
       "count": "SMW\\ListResultPrinter",
       "debug": "SMW\\ListResultPrinter",
       "feed": "SMW\\FeedResultPrinter",
       "csv": "SMW\\CsvResultPrinter",
       "dsv": "SMWDSVResultPrinter",
       "json": "SMW\\JsonResultPrinter",
       "rdf": "SMWRDFResultPrinter"
   },
   "smwgResultAliases": {
       "feed": [
           "rss"
       ]
   },
   "smwgQuerySources": [
   ],
   "smwgPDefaultType": "_wpg",
   "smwgAllowRecursiveExport": false,
   "smwgExportBacklinks": true,
   "smwgMaxNonExpNumber": 1000000000000000,
   "smwgEnableUpdateJobs": true,
   "smwgNamespacesWithSemanticLinks": {
       "102": true,
       "103": false,
       "104": true,
       "105": false,
       "108": true,
       "109": false,
       "NS_MAIN": true,
       "NS_TALK": true,
       "NS_USER": true,
       "NS_USER_TALK": true,
       "NS_PROJECT": true,
       "NS_PROJECT_TALK": false,
       "NS_IMAGE": true,
       "NS_IMAGE_TALK": false,
       "NS_MEDIAWIKI": true,
       "NS_MEDIAWIKI_TALK": false,
       "NS_TEMPLATE": true,
       "NS_TEMPLATE_TALK": false,
       "NS_HELP": true,
       "NS_HELP_TALK": false,
       "NS_CATEGORY": true,
       "NS_CATEGORY_TALK": false,
       "NS_General": true,
       "NS_Services": true,
       "NS_Policy": true,
       "NS_Risk": true,
       "NS_Procedures": true,
       "NS_Requirements": true,
       "NS_Improvements": true,
       "NS_Ethics": true,
       "NS_Financials": true,
       "NS_Documents_of_Record": true,
       "NS_Hospitality": true,
       "NS_Incidents": true,
       "NS_Specialties": true,
       "NS_Potential_Conflicts": true,
       "General": true,
       "Services": true,
       "Policy": true,
       "Risk": true,
       "Procedures": true,
       "Requirements": true,
       "Improvements": true,
       "Ethics": true,
       "Financials": true,
       "Documents_of_Record": true,
       "Hospitality": true,
       "Incidents": true,
       "Specialties": true,
       "Potential_Conflicts": true,
       "NS_Pharmacy": true,
       "Pharmacy": true,
       "NS_Audit": true,
       "Audit": true,
       "NS_Purchasing": true,
       "Purchasing": true,
       "NS_Lab": true,
       "Lab": true,
       "NS_Recruitment": true,
       "Recruitment": true,
       "NS_Radiology": true,
       "Radiology": true,
       "NS_Family_Medicine": true,
       "Family_Medicine": true,
       "NS_Anaesthetics": true,
       "Anaesthetics": true,
       "NS_Marketing": true,
       "Marketing": true,
       "NS_Gynaecology": true,
       "Gynaecology": true,
       "NS_Dietetics": true,
       "Dietetics": true,
       "NS_Accounts": true,
       "Accounts": true,
       "NS_IT": true,
       "IT": true,
       "NS_Nursing": true,
       "Nursing": true,
       "NS_Gastroenterology": true,
       "Gastroenterology": true,
       "NS_Fetal_medicine": true,
       "Fetal_medicine": true,
       "NS_Paediatrics": true,
       "Paediatrics": true,
       "NS_HR": true,
       "HR": true,
       "NS_Claims": true,
       "Claims": true,
       "NS_Internal_Medicine": true,
       "Internal_Medicine": true,
       "NS_Hepatology": true,
       "Hepatology": true,
       "CATEGORY": true,
       "CATEGORY-TALK": true,
       "NS-CATEGORY": true,
       "NS-CATEGORY-TALK": true,
       "HELP": true,
       "HELP-TALK": true,
       "NS-HELP": true,
       "NS-HELP-TALK": true,
       "HR-TALK": true,
       "NS-HR": true,
       "NS-HR-TALK": true,
       "IMAGE": true,
       "IMAGE-TALK": true,
       "NS-IMAGE": true,
       "NS-IMAGE-TALK": true,
       "IT-TALK": true,
       "NS-IT": true,
       "NS-IT-TALK": true,
       "MAIN": true,
       "MAIN-TALK": true,
       "NS-MAIN": true,
       "NS-MAIN-TALK": true,
       "MEDIAWIKI": true,
       "MEDIAWIKI-TALK": true,
       "NS-MEDIAWIKI": true,
       "NS-MEDIAWIKI-TALK": true,
       "PROJECT": true,
       "PROJECT-TALK": true,
       "NS-PROJECT": true,
       "NS-PROJECT-TALK": true,
       "TALK": true,
       "TALK-TALK": true,
       "NS-TALK": true,
       "NS-TALK-TALK": true,
       "TEMPLATE": true,
       "TEMPLATE-TALK": true,
       "NS-TEMPLATE": true,
       "NS-TEMPLATE-TALK": true,
       "USER": true,
       "USER-TALK": true,
       "NS-USER": true,
       "NS-USER-TALK": true,
       "Accounts-TALK": true,
       "NS-Accounts": true,
       "NS-Accounts-TALK": true,
       "Anaesthetics-TALK": true,
       "NS-Anaesthetics": true,
       "NS-Anaesthetics-TALK": true,
       "Audit-TALK": true,
       "NS-Audit": true,
       "NS-Audit-TALK": true,
       "Claims-TALK": true,
       "NS-Claims": true,
       "NS-Claims-TALK": true,
       "Dietetics-TALK": true,
       "NS-Dietetics": true,
       "NS-Dietetics-TALK": true,
       "Documents_of_Record-TALK": true,
       "NS-Documents_of_Record": true,
       "NS-Documents_of_Record-TALK": true,
       "Ethics-TALK": true,
       "NS-Ethics": true,
       "NS-Ethics-TALK": true,
       "Family_Medicine-TALK": true,
       "NS-Family_Medicine": true,
       "NS-Family_Medicine-TALK": true,
       "Fetal_medicine-TALK": true,
       "NS-Fetal_medicine": true,
       "NS-Fetal_medicine-TALK": true,
       "Financials-TALK": true,
       "NS-Financials": true,
       "NS-Financials-TALK": true,
       "Gastroenterology-TALK": true,
       "NS-Gastroenterology": true,
       "NS-Gastroenterology-TALK": true,
       "General-TALK": true,
       "NS-General": true,
       "NS-General-TALK": true,
       "Gynaecology-TALK": true,
       "NS-Gynaecology": true,
       "NS-Gynaecology-TALK": true,
       "Hepatology-TALK": true,
       "NS-Hepatology": true,
       "NS-Hepatology-TALK": true,
       "Hospitality-TALK": true,
       "NS-Hospitality": true,
       "NS-Hospitality-TALK": true,
       "Improvements-TALK": true,
       "NS-Improvements": true,
       "NS-Improvements-TALK": true,
       "Incidents-TALK": true,
       "NS-Incidents": true,
       "NS-Incidents-TALK": true,
       "Internal_Medicine-TALK": true,
       "NS-Internal_Medicine": true,
       "NS-Internal_Medicine-TALK": true,
       "Lab-TALK": true,
       "NS-Lab": true,
       "NS-Lab-TALK": true,
       "Marketing-TALK": true,
       "NS-Marketing": true,
       "NS-Marketing-TALK": true,
       "Nursing-TALK": true,
       "NS-Nursing": true,
       "NS-Nursing-TALK": true,
       "Paediatrics-TALK": true,
       "NS-Paediatrics": true,
       "NS-Paediatrics-TALK": true,
       "Pharmacy-TALK": true,
       "NS-Pharmacy": true,
       "NS-Pharmacy-TALK": true,
       "Policy-TALK": true,
       "NS-Policy": true,
       "NS-Policy-TALK": true,
       "Potential_Conflicts-TALK": true,
       "NS-Potential_Conflicts": true,
       "NS-Potential_Conflicts-TALK": true,
       "Procedures-TALK": true,
       "NS-Procedures": true,
       "NS-Procedures-TALK": true,
       "Purchasing-TALK": true,
       "NS-Purchasing": true,
       "NS-Purchasing-TALK": true,
       "Radiology-TALK": true,
       "NS-Radiology": true,
       "NS-Radiology-TALK": true,
       "Recruitment-TALK": true,
       "NS-Recruitment": true,
       "NS-Recruitment-TALK": true,
       "Requirements-TALK": true,
       "NS-Requirements": true,
       "NS-Requirements-TALK": true,
       "Risk-TALK": true,
       "NS-Risk": true,
       "NS-Risk-TALK": true,
       "Services-TALK": true,
       "NS-Services": true,
       "NS-Services-TALK": true,
       "Specialties-TALK": true,
       "NS-Specialties": true,
       "NS-Specialties-TALK": true,
       "User": true,
       "User-TALK": true,
       "NS-User": true,
       "NS-User-TALK": true
   },
   "smwgPageSpecialProperties": [
       "_MDAT"
   ],
   "smwgDeclarationProperties": [
       "_PVAL",
       "_LIST"
   ],
   "smwgTranslate": false,
   "smwgAdminRefreshStore": true,
   "smwgAutocompleteInSpecialAsk": true,
   "smwgAutoRefreshSubject": true,
   "smwgAutoRefreshOnPurge": true,
   "smwgAutoRefreshOnPageMove": true,
   "smwgContLang": {
   },
   "smwgMaxPropertyValues": 3,
   "smwgQSubpropertyDepth": 10,
   "smwgNamespace": ".kchclinics.com",
   "smwgMasterStore": null,
   "smwgIQRunningNumber": 0,
   "smwgCacheType": -1,
   "smwgCacheUsage": {
       "smwgWantedPropertiesCache": true,
       "smwgWantedPropertiesCacheExpiry": 3600,
       "smwgUnusedPropertiesCache": true,
       "smwgUnusedPropertiesCacheExpiry": 3600,
       "smwgPropertiesCache": true,
       "smwgPropertiesCacheExpiry": 3600,
       "smwgStatisticsCache": true,
       "smwgStatisticsCacheExpiry": 3600
   },
   "smwgFixedProperties": [
   ],
   "smwgPropertyLowUsageThreshold": 5,
   "smwgPropertyZeroCountDisplay": true,
   "smwgShowHiddenCategories": true,
   "smwgFactboxUseCache": true,
   "smwgFactboxCacheRefreshOnPurge": true,
   "smwgQueryProfiler": {
       "smwgQueryDurationEnabled": false
   },
   "smwgEnabledSpecialPage": [
       "Ask",
       "RunQuery"
   ],
   "smwgOnDeleteAction": {
       "smwgDeleteSubjectAsDeferredJob": false,
       "smwgDeleteSubjectWithAssociatesRefresh": false
   },
   "smwgCanonicalNames": {
       "102": "Property",
       "103": "Property_talk",
       "104": "Type",
       "105": "Type_talk",
       "108": "Concept",
       "109": "Concept_talk"
   }

}

Return to Special:SMWAdmin

11:50, 21 August 2014
 
 

Names still correct ?

Can it be that rebuildData.php is now named SMW_refreshData.php ?

13:04, 13 May 2014

No, what makes you think so?

14:24, 13 May 2014

The fact, that I didn't find any file by the fist name, but found the other one, and it did what I expected the first file to do and worked with the parameters, just as the article describes...

14:30, 13 May 2014

To wit, according to Help:rebuildData.php, "This maintenance script depreciated the former "SMW_refreshData.php" script starting with SMW 1.9.2 and it is strongly encouraged to make the transition to this new script."

So it appears that the opposite of what Heinrich has stated.

15:21, 13 May 2014

Okay, that explains it, too.

15:26, 13 May 2014