Wikimedia Belgium bewikimedia https://be.wikimedia.org/wiki/Wikimedia_Belgium MediaWiki 1.39.0-wmf.23 first-letter Media Special Talk User User talk Wikimedia Wikimedia talk File File talk MediaWiki MediaWiki talk Template Template talk Help Help talk Category Category talk TimedText TimedText talk Module Module talk Translations Translations talk Gadget Gadget talk Gadget definition Gadget definition talk 14 2414 19229 15200 2022-08-15T14:04:47Z Geertivp 171 Geertivp moved page [[Category:Free content licenses]] to [[Category:Free licenses]] wikitext text/x-wiki [[Category:Copyright]] 27s9tcr6f9vc96w8s8xd6rmy9x4zndx 0 2918 19234 18922 2022-08-16T09:51:14Z Geertivp 171 Wikimania 2022 wikitext text/x-wiki '''OpenRefine''' is a powerful data manipulation tool to validate, prepare data to be uploaded to, uploaded, or retrieved from, [[Wikidata]] and [[Wikimedia Commons]]. Multiple plugins exist. You can import and export data from and to multiple formats. It is more powerful than [[Excel]], because it interfaces with external systems, but a bit more difficult to use. It is using open standards. It has a read and write interface to Wikidata, and Wikimedia Commons. It can export to [[QuickStatements]] for large batch operations, or you can amend Wikidata interactively. == What is it? == Originally called ''Google Refine'', an automated tool to manipulate lists of data, based on rows and columns. A front-end to Wikidata. Import and Export to other formats like Excel, CSV, Google Spreadsheets, etc. == Install the tool == See: * https://openrefine.org *https://docs.openrefine.org/manual/installing *https://openrefine.org/download.html Windows: Download the zip file. Unzip. Possibly install the Java Run-time Environment (JRE). Linux: Prerequisites: sudo apt install default-jre [[Chromebook]]; install the Linux version. == Runtime environment == === Run === * On Windows: Run openrefine.exe from Explorer, which will open a browser (localhost webserver). * On Linux: ** Run the shell script "refine" (backend) **Open the application in the browser: http://127.0.0.1:3333 (frontend) === Storage === Local storage on the client device. === Stop === Save your data. Close the DOS window, or exit the application (^c). == Functionality == [[File:20200613 Presentation OpenRefine Workshop (Dutch).pdf|thumb|Presentation OpenRefine]] * Data import (multiple targets) **Input from CSV, Excel, Google Spreadsheet, XML file, paste buffer (very practical) **Skip empty rows *Selectively delete rows based on facets (filter query) *Cleanup: merge data, detect and correct outliers * Transform: split strings into columns, concatenate columns, based on GREL = General Refine Expression Language * Reconcile: validate and get ID data from e.g. [[Wikidata]] **Choose an instance, or reconcile against no particular type **Verify if you got the right item (check homonyms based upon descriptions or statements) **Choose the right homonym **Flag the item for creation if it does not exist yet *Create columns based on Wikidata statements from reconciled items *Enrich: get additional data by ID from external databases *Verify the data quality * Create [[Wikidata schema|Wikidata schemas]] (prepare data upload: item labels, descriptions, alias, statements) *Upload to Wikidata **Pay attention not to create duplicate items or statements ***create prerequisite items ***first amend existing items ****create a minimum list of statements ***then create new items; multiple targets are possible (use multiple facets when required) ****add all required statements => no risk for duplicate statements since new items are created *Extensions === Menu commands === *Reconcile against no type (when inputting Q-numbers) * Convert to text, before uploading a date *White space (should be handled by Wikibase?) **Trim leading and trailing spaces **Collapse repeated white space === GREL functions === * Replace special characters: value.replace(".0" ,'0') value.replace("”" ,'"') value.replace("“" ,'"') value.replace("’" ,"'") * Extract first sentence from a text: (avoid plagiaat) value.match(/([A-Za-z0-9éë ,:"'()-]+)[.].*/).toString() * Make first character lowercase toLowercase(value.substring(0,1))+value.substring(1) == Options == * You can set the user interface language via Preferences userLang == Unresolved == * How to add rows? == Use case == * Prepare [[ListeriaBot]] lists to create and amend Wikipedia pages linked to a [[WikiProject]] (example: [[w:nl:Wikipedia:Wikiproject/Gendergap/Ontwerp/België/Wiki Women Design/ListeriaBot/Query|Wiki Women Design]]); see [[d:User:Geertivp/training/Wikidata/Wiki projects|How to setup Wiki projects]]. == Known problems == * Uploading to Wikidata can take a long time; you are '''not''' notified via a message "transaction in progress" ** You can export to [[QuickStatements]] (error prone; more manual work) === Duplicate items === * Avoid creating duplicate items, aliases, statements, and values: ** Reconciling might fail (slight different label, wrong instance, homonyms) ** Manual intervention might be required (manual search via the GUI) ** Reconcile in multiple cycles (filter already reconciled items) ** Run <code>validate_item.py</code> to get matching Q-numbers * You can merge any duplicate items afterwards (please be careful that it is really duplicate; e.g. museum against building) == Documentation == *[https://openrefine.org Kit and documentation] *https://www.wikidata.org/wiki/Wikidata:Tools/OpenRefine *https://www.wikidata.org/wiki/Wikidata:Tools/OpenRefine/Editing/Tutorials/Video * https://www.wikidata.org/wiki/Help:Dates * [https://openrefine.org/documentation.html Online courses] * [https://github.com/OpenRefine/OpenRefine/wiki/Documentation-For-Users User documentation] * [https://github.com/OpenRefine/OpenRefine OpenRefine on GitHub] * [https://github.com/OpenRefine/OpenRefine/blob/master/CONTRIBUTING.md Contributing to OpenRefine] * [https://docs.openrefine.org/technical-reference/architecture Documentation about OpenRefine's architecture] == Sessions and projects == * [[wmania:Hackathon/Schedule|Wikimania 2022]] * https://phabricator.wikimedia.org/T314803 == See also == * [[Wikimedia Belgium Projects/ListeriaBot/Query|Wikimedia Belgium Projects]] * [[d:Wikidata:Tools/OpenRefine|Wikidata]] * [[c:Commons:OpenRefine|Commons]] *[[QuickStatements]] [[Category:Tools]] [[Category:Wikidata]] [[Category:Google Spreadheets]] sagcvontat5ql03w98f8ocgw2mavztv 14 3522 19230 2022-08-15T14:04:47Z Geertivp 171 Geertivp moved page [[Category:Free content licenses]] to [[Category:Free licenses]] wikitext text/x-wiki #REDIRECT [[:Category:Free licenses]] m0hnrhng3mttufx8tbh5c40151t6fxa 0 3523 19231 2022-08-15T14:08:31Z Geertivp 171 description and examples of the CC BY-SA free license wikitext text/x-wiki '''CC BY-SA''' ('''share alike''') is one of the [[Free license|free licenses]] that is much used. It is the primary license for Wikipedia and for most other [[Wikimedia platforms]]. It is ideal for persons to allow others to use ones work, while assuring: * the author must be named (source reference) * the license must always stay "share alike" * a link to the copyright must be provided == Examples == * [[c:File:Hadewijch gedicht1 HsGent f49r.jpg|File:Hadewijch gedicht1 HsGent f49r.jpg]] - The library of the [[Universiteit Gent|University of Ghent]] has published their manuscripts as CC BY-SA. Therefore all images can be used for [[Wikipedia]]. == See also == * [[Free license]] [[Category:Free licenses]] 8tf7bmslc91yq5nvop9m2113fg3l8cn 0 3524 19232 2022-08-15T14:14:49Z Geertivp 171 Created page with "'''Michiel Hendryckx''' is a famous Flemish photographer and Wikipedian. He worked for [[De Standaard]]. == Work == * https://www.openmonumenten.be/nieuws/fotowedstrijd-michiel-hendryckx-daagt-vlaanderen-uit-om-massaal-erfgoedfotos-te-maken * https://www.youtube.com/watch?v=u2emxxV_QYI [[Category:Photography]]" wikitext text/x-wiki '''Michiel Hendryckx''' is a famous Flemish photographer and Wikipedian. He worked for [[De Standaard]]. == Work == * https://www.openmonumenten.be/nieuws/fotowedstrijd-michiel-hendryckx-daagt-vlaanderen-uit-om-massaal-erfgoedfotos-te-maken * https://www.youtube.com/watch?v=u2emxxV_QYI [[Category:Photography]] s6kdvs3d25xk6yj8ehillwc2d9wm3pq 0 3525 19233 2022-08-15T15:40:18Z Geertivp 171 Created page with "[[Preservation]] of objects is urgently needed, for several reasons: * progress in technology (disappearing technology) * catastrophes (fire, water, storm, earth quakes) == See also == * [[:en:wikipedia:There is a deadline|Wikipedia:There is a deadline]]" wikitext text/x-wiki [[Preservation]] of objects is urgently needed, for several reasons: * progress in technology (disappearing technology) * catastrophes (fire, water, storm, earth quakes) == See also == * [[:en:wikipedia:There is a deadline|Wikipedia:There is a deadline]] 2iu0lxz3jsuwic9l5cxxglypk58e4v3