Repository navigation
Integrate with Crowdin localization project #82
Description
Activity
Happy to help create a Gatsby source plugin for Crowdin! That'd be really powerful. And a lot of people would use it including sooner than later gatsbyjs.org
From a bit of research, it looks like it'd be pretty easy to pull data using their API:
- https://support.crowdin.com/api/download/
- Guide for creating a source plugin https://www.gatsbyjs.org/docs/create-source-plugin/ — this guide is a bit wordy right now. When someone wants to tackle this, I could take a pass at improving the guide + adding a mini-tutorial on creating source plugins.
I spent a bunch of time adding localization support to https://www.gatsbyjs.org/packages/gatsby-source-contentful/ as well so have some opinions on this. See the example site for Contentful + its source
Fantastic 😄 There's a good chance we'll end up collaborating on this soon then.
Netlify also has a way of doing redirects based on
Accept-Languageheader and GeoIP. For example, if someone from Italy (or someone with their browser or OS set to request Italian content) visits the home page and they don't have a language cookie already set, we can automatically redirect to the Italian version. We use this on the Yarn site - You can look at how Yarn does it for inspirationHow does this impact urls (eg /docs/, /docs//)?
I'd suggest
/<lang>/(eg./<lang>/docs/<file>), this is consistent with other Facebook projects that use localization:
https://yarnpkg.com/en/docs/getting-started
https://flow.org/en/docs/getting-started/If we run into any performance problems, @ericnakagawa has volunteered to work with Netlify and Crowdin to simplify or speed up this syncing process. Ideally this sync process will be fast enough to allow Netlify's PR previews to also be used to test localized builds of the site.
We have perf issues with the Yarn site, but a lot of it is because we're using Jekyll. Most of the build time is actually the Jekyll build time rather than CrowdIn. We had a prototype using Hugo that was much faster. I don't think CrowdIn takes a significant amount of time there.
From a bit of research, it looks like it'd be pretty easy to pull data using their API:
Would the React site actually need a custom API integration? It could be useful, but CrowdIn has a CLI tool (https://support.crowdin.com/cli-tool/) which we might be able to use with less effort. For Yarn we just run
crowdin-cli download -b masterto download all the localized files as part of the build (in that case, there's a bunch of Markdown files as well as some yml files for language strings). Then you just build the site like normal and everything just works, as long as the files are in the correct locations.How does this impact urls (eg /docs/, /docs//)?
I'd suggest // (eg. //docs/), this is consistent with other Facebook projects that use localization:
Don't have time to fully digest this response now- but wanted to mention that we should ensure whatever change we make does not break pre-existing links (like how links broke for Jest).
Would the React site actually need a custom API integration
It doesn't need it but it seems simpler perhaps to just query the data through graphql instead of writing it out first then pulling it in.
But yeah, definitely more straightforward to just use the cli.
Would you checkin the translated content?
but wanted to mention that we should ensure whatever change we make does not break pre-existing links
Jest is limited in what it can do, because it's using GitHub Pages. With Netlify, you can generate some redirects as part of the build. Just need to redirect "raw" URLs (
/docs/foo) to the localized URL (/en/docs/foo), either with/en/hard-coded or using Netlify's language redirects that I mentioned earlier (so it'd redirect/docs/footo/<user's preferred language>/docs/foo).Check it out with Yarn: https://yarnpkg.com/docs/getting-started redirects to the right place. :)
Would you checkin the translated content?
We don't for Yarn. It's pulled in as part of the build process for the site. Since CrowdIn tracks the history for the translations, we didn't really need to also have Git track it.
Some sites do commit the translated files though. Up to you - Either way is fine.
Would you checkin the translated content?
No
or using Netlify's language redirects that I mentioned earlier (so it'd redirect /docs/foo to /<user's preferred language>/docs/foo).
Cool! 👍
Here's how the Yarn site builds all the redirects from non-localized URLs to localized URLs: https://github.com/yarnpkg/website/blob/master/_redirects#L48-L70 (this would also be how you handle your redirects from 'old' URLs to new localized URLs). Unfortunately the Yarn one is super difficult to understand because it's using the Jekyll templating syntax. Somehow @thejameskyle got it working though. You could likely make a nicer JS script to do the same thing for the React site.
The resulting redirects file looks something like this:
/docs/cli/add/ /en/docs/cli/add/ 302 Language=en /docs/cli/add/ /fr/docs/cli/add/ 302 Language=fr /docs/cli/add/ /pt-BR/docs/cli/add/ 302 Language=pt-br /docs/cli/add/ /zh-Hans/docs/cli/add/ 302 Language=zh-cnHere's the entire language redirects section: https://gist.github.com/Daniel15/d84b39e8a260ee2a7d627751d015181b
Docs: https://www.netlify.com/docs/redirects/#geoip-and-language-based-redirects
Might be able to use a regex or placeholders or something rather than having to generate all the redirects like that.
@bvaughn wrote a nice redirect API for Gatsby + added it to gatsby-plugin-netlify so this setting up redirects would be pretty easy :-)
Hi, the official doc in English is being updated frequently, but the contents in Crowdin seems to be out of sync with the newest doc. I think it's still based on the doc several months ago, and I'm seeing no doc for error boundaries ready for translation.
I'm new to Crowdin, but I want to believe there is a way to merge the updates in the original doc and enable people to translate the changed part incrementally. Is there any plan for this? I'd rather not see an overwhelming one-time update of the original doc.
25 remaining items
Hi, I logged in to Crowdin after a while, and found the "translated" percentage of Japanese has dropped to 65%. It was 100% (i.e., translation completed) in September last year, but no one but translators themselves had a chance to read the translated docs :)
So can someone set up a timeline and a guideline for this on which we can rely on? You have already gathered dozens of volunteers at Crowdin even though you have not officially called for volunteers. Has the localization project started officially, in the first place? Are you going to call for more collaborators, in the blog for example? When are we expected to finish the translation? I don't think the entire Crowdin project may be abandoned, but since there is no timeline nor document to guide the translators, I don't know when we can restart translating to keep up with the current doc. I don't even know what those "master" and "test-17" folders mean. (BTW, I just found the translation percentage for the "master" folder is now 1%... Does this mean almost all the effort we already made on the "old" master is going to disappear soon?)
Reacted by seungha-kim, Sunki Baek, y-iyama and YToriiCan we expect some progress before the release of React hooks? For now, articles related to hooks are not even listed in the Crowdin repository, and it appears to me that this effort has been half-abandoned recently. It's sad to see developers in my country have to depend on fairly insufficient personal blog articles when such a big change is approaching to the ecosystem.
By the way, does anyone know why materials in Crowdin are in HTML format rather than Markdown? HTML tags, especially all those
<code>s, are very cumbersome to handle.ping @bvaughn @ericnakagawa
Sorry, I don't have any updates/estimates for this. The core team is spread very thin and I don't think anyone owns this effort at the moment 😞
I see. Meanwhile, is it possible to allow translators to download/export their existing contributions from Crowdin? If I understand correctly, this can be done by changing the project settings. Then each translator may host the translated version elsewhere on the internet, say, my GH pages.
That's something I think @ericnakagawa would have to manage. I don't know much about our Crowdin setup. Not even sure if I would have the necessary permissions.
To give an update — if everything works out, we expect a person to jump on finishing the integration in one of the next weeks. Can't promise to get it done before Hooks are stable though but it's on our radar.
Reacted by Soichiro MikiFor the record, here are some problems I'm currently seeing as a translator.
- Translations are not Markdown-based but HTML-based. I believe all translated materials should be version-controlled.
- There are several "root" directories on the Crowdin repo, and it is not clear which to work on.
- New commits to the original doc are not automatically synched. For example there are no articles about Hooks yet in the Crowdin repository.
- Some elements in the front matter (document IDs, etc) are wrongly marked as translatable.
- Some code examples are marked as translatable and some are not.
- This may be specific to CJK languages, but italics are not always rendered properly for those languages, so we need a different CSS styles for
<em>'s.
Translations are not Markdown-based but HTML-based.
For Yarn we use Markdown in CrowdIn (at https://i18n.yarnpkg.com/), so this might just be an issue with how it's configured for React. Should be straightforward to fix it 😃
- Reacted by Dima Tisnek
- Reacted by Soichiro Miki, Dima Tisnek and Sam Richards
- added a commit that references this issue
on Jul 1, 2020 - added a commit that references this issue
on Mar 21, 2023
Migrated from react/react/issues/8063
Coordinate with docs translators via @ericnakagawa
This issue involves multiple steps:
Integrate with Crowdin
This could be done via a custom Gatsby plug-in. We've already written a couple. (It would be nice to work with @KyleAMathews to share this plug-in back with Gatsby so that other users could benefit from it as well.)
If we run into any performance problems, @ericnakagawa has volunteered to work with Netlify and Crowdin to simplify or speed up this syncing process. Ideally this sync process will be fast enough to allow Netlify's PR previews to also be used to test localized builds of the site.
Misc thoughts and questions
/docs/<file>,/docs/<lang>/<file>)?