Any plans on including `sourcekit-lsp`? The latest toolchains for Ubuntu includes it.
I couldn't get it to work correctly on Fedora 30. Like it compiles fine, but I get nothing out of the lsp, the returned results are always empty. In VSCode, i'm getting `failed to create SwiftPMWorkspace: cannotDetermineHostToolchain`.
Hmm, sure, let me give it a try. I'd be interested to know how to get it to work properly in case more tweaking has to be done in the packaging.
So I've been looking into this quite a bit; you're right in that it's included if Swift is built from the master branch on Github; It built properly on Fedora using F29, which still has Python 2. It is not part of the releases, however; if you look at https://github.com/apple/sourcekit-lsp/tags, there is no release or tag that is in sync with Swift's "-RELEASE". That plus its stated on the README that it's under heavy development, *and* the fact that I too have been unable to get it working at all, makes me worried that it would lead to a crummy user experience with Swift in general. I'm going to keep this on the radar but at the moment I don't think this is going to be able to be added; if you'd like to take a crack at it, patches welcome!
So I've noticed that the swift build-script takes '--sourcekit-lsp [BOO]' and '--install-sourcekit-lsp [BOOL]' to build sourcekit-lsp along swift. I got swift to build fine on F30, but fails when building it with sourekit-lsp due to missing toolchain. When building a toolchain, sourcekit-lsp is included by default which means it is part of the release. Check out https://github.com/apple/sourcekit-lsp/releases/tag/swift-5.1.1-RELEASE I'd say since it's part of the release, it should be really considered.
Okay, I'll give it a try.
I am able to include sourcekit-lsp into the package. However, I haven't had any success getting it to work with VSCode and am not exactly sure *how* it's supposed to work, which obviously isn't a great thing. Also, I posted a question on the Swift Forums (https://forums.swift.org/t/no-release-5-1-1-of-the-forked-llvm-project/30368) about there not being an explicit 5.1.1 release tarball of the llvm-project (not the same as llvm) and so I opted for a snapshot that was close to when Swift-5.1.1 was released. I have not submitted it to Koji because I'm not sure it's really working properly. If you want to test it, I am hosting the sourcekit-lsp version on Copr at https://copr.fedorainfracloud.org/coprs/tachoknight/swift-lang/ if you'd like to give it a try.
When installing the sourcekit-lsp version, I keep getting a Failed dependencies error: error: Failed dependencies: libFoundationXML.so()(64bit) is needed by swift-lang-5.1.1-0.4.20191004git4242edd.fc30.x86_64 Installing it with --nodeps but no success with sourcekit-lsp. I tried nvim with lsp and vscode, the lsp server works but returns nothing. The same issue when building sourcekit-lsp from their master branch. I'll keep testing and report back if there's any updates.
Hey Ayman- The missing library is my fault; it wasn't packaged. I have a new build for x64_64 at https://tachoknight.fedorapeople.org/swift-lang/ if you'd like to try it. That said, it seems Apple has done some project reorganization; certain sub-projects have been deprecated for new ones, etc. I'm going to "start over" by trying to adhere as close as possible to their new layout as ultimately 5.1.2 or whatever the next -RELEASE version is will likely require it. I'll include sourcekit-lsp into it. In the meantime, if you can test the versions above I'd love feedback. Also, some general pointers on how to use the thing would be helpful as currently it packages, but I don't really have any idea how to judge whether it works or how it works; getting the REPL working was easy by comparison, it either started properly or it didn't. :)
Hey Ron, I really appreciate the work you're doing, thank you! So I've tried the new build and it installs fine but doesn't come with sourcekit-lsp. Tried building sourcekit-lsp from source but the build fails in swift-package-manager. So I'm back to the packages from copr. sourcekit-lsp is a language-server, it helps with writing code. The guys behind it provide a plugin to work with VSCode. Once you install it and configure it, you should be able to get code autocompletion, hints, etc. Now I've tried it on an Ubuntu instance on a VM with swift-5.1-RELEASE-ubuntu18.04 toolchain and it worked out of the box! I just had to install the plugin and set the sourcekit-lsp binary path in VSCode. Below are two pictures showing how it works on Ubuntu and how the lsp logs look like on Fedora after setting tracing to verbose. https://imgur.com/a/xVKM2zd https://github.com/apple/sourcekit-lsp/tree/master/Editors#visual-studio-code https://github.com/JohnSundell/Splash https://langserver.org/
The big difference I've noticed between the Apple Ubuntu builds and mine is that they pull straight from source on GitHub, whilst I'm using their tgz archives. The benefit of the former is that all the various projects are pulled down, including sourcekit-lsp as well as swift-format and swift-syntax. swift-format has no releases (i.e. no archive files to download), and swift-syntax is not in sync with the latest dev builds. To illustrate the differences in building: when you start Swift's REPL or do a `swift --version` on Ubuntu, you'll see that both swift and clang have git hashes, while the Fedora ones do not; this is because part of the building decorates the banners with the hash from the .git directory, while the archive files don't have the .git directory. So I thought about trying to copy what Apple is doing but with Fedora 31 and newer, which uses Python 3, that isn't possible because of differences in how 2 and 3 handle certain things. I have a pull request (https://github.com/apple/swift/pull/26296) which makes it all work under Fedora (and some of which I patch when building the RPMs), but Apple has not indicated anywhere I've found (mostly looking in the Swift forums) about when they're going to make a push to make Swift fully Python 3-compliant. TL;DR: I can't build everything that sourcekit-lsp needs because some sub-projects don't have releases, and I can't build everything from source because of Python versioning issues. I am updating to the 2019-11-05 snapshot, and will be making a COPR build as soon as my underpowered server finishes my test build (takes about four hours); I'll keep at it but wanted to let you know what's going on in case you don't see any meaningful progress. That said, I do totally want to have this feature now myself, so I'll definitely keep at it!
Your package spec helped me get swift-lang built properly, thanks for this. I have some info to report on the off chance it'll be helpful. While I was playing around, I noticed that your most recent package spec does not produce a working `sourcekit-lsp`. I used VSCode and `vim-lsp`, and although no errors were given, no completions or language server functionality was either. Using your package spec to build a .tar.gz file and then installing the dependencies, followed by extracting this to somewhere on my hard drive and setting `PATH` (without the package installed), worked. e.g. ``` tar -xzf swift-5.1.2-fedora.tar.gz mkdir -p /opt/swift mv usr/ /opt/swift/usr export PATH=$PATH:/opt/swift/usr/bin ``` I tested that by using my Vim plugin and VSCode. In that case, `sourcekit-lsp` produced completions against the standard library, member variables, etc... So without digging any further, I would guess that something is up with the relative file structure that's causing the problem. Finally, when I was building the image on an empty Docker image, I noticed that despite installing all of the dependencies in the spec file with `yum build-dep`, `make` was not installed and the build failed the first time. There's a good chance this is due to my ignorance (I'm relatively new to using the Fedora distribution) but I figured I'd include it here in the off chance it's significant.
Hi John- So we're on the same page, the swift-5.1.2-fedora.tar.gz file you used came from building Swift via `rpmbuild` right (i.e. you were building it from the spec file)? Also make is not necessary because Swift uses cmake to build.
> So we're on the same page, the swift-5.1.2-fedora.tar.gz file you used came from building Swift via `rpmbuild` right (i.e. you were building it from the spec file)? My procedure was roughly: ``` docker run -it fedora:31 bash dnf install fedora-packager fedora-review cd ~/ mkdir packaging cd packaging # probably missing some steps here fedpkg --release f31 local ``` The build failed for the above reason (make was missing) so I installed it and continued the build by running ``` cd swift-build ./swift/ swift/utils/build-script --preset=buildbot_linux,no_test install_destdir=$PWD/build installable_package=$PWD/build/swift-5.1.2-fedora.tar.gz ``` But the tar file that came out would probably look the same after the build completed successfully anyway. I tested your changes by running: ``` sudo dnf copr enable tachoknight/swift-lang sudo dnf update swift-lang ``` > Also make is not necessary because Swift uses cmake to build. Doesn't CMake just create build files for different downstream build systems? So when "make" is run it's calling into `Makefile`s that have been generated by CMake, but still use `make` to execute? So wouldn't a CMake project that has been processed with the "Unix Makefile" generator still require that make be installed for the build to run? It's possible that this dependency problem has been masked by usually doing the build on machines that have make installed (as most development machines do). I can try to reproduce this later if you'd like. Sorry if my report here is overly vague, but I don't have my terminal history available.
Right, I said CMake, but I really meant Ninja; you're correct that CMake generates files for downstream builders, which in this case is ninja. Oh, so you built Swift from scratch, from GitHub? I do that too except it fails 100% of the time because they haven't merged my pull requests to make everything python 3 compatible. I would, if you be so kind, to know what steps you actually did so I can compare. One thing that is always going to be different is that if you build from source without patching, you'll get a working swift installation that assumes everything is in known locations, for example the swift binary expects to find lldb in the same directory (usr/bin) and for packaging, that has to be placed elsewhere (/usr/libexec/swift-lldb in this case). My concern is that the sourcekit binary is doing something similar; I haven't been able to actually get sourcekit working to validate what's going on, but I'll keep at it.
I was using a Python virtual environment to get around the Python problem. However, this just underscores why I should put together a set of repro steps. :) I'll get to that this evening or tomorrow. Thanks for your time.
Any updates?
Was hoping to get some info from John; I've been working on it in the background when Real Life™ isn't getting in the way, which it has been lately. Also, when Apple stops releasing "swift-5.1-DEVELOPMENT-SNAPSHOT-" versions, it typically means they're doing big changes or at least changes that will break my build scripts so will have to figure out what's going on there. Anyway, TL;DR: not much yet other than coming up with new ways to break stuff. :)
just as an additional bit of info, my current process has me learning exactly how the vscode plugin works and communicates with clangd; not being much of a javascript/npm user this has been a bit of a journey, but work is, in fact proceeding, if not always in a completely straight direction. If anyone knows more about the integration and can help short-circuit this, I'd appreciate the info!
Hey Ron & Ayman, Sorry for disappearing. That Real Life guy is relentless. Here are my repro steps: ``` sudo docker run -it fedora:31 bash dnf -y install fedora-packager fedora-review which make mkdir -p ~/packaging cd ~/packaging git clone https://src.fedoraproject.org/rpms/swift-lang git reset --hard d968601519670f3fee5b06a03dc0cd2173ee00b4 cd swift-lang dnf -y builddep ./swift-lang.spec fedpkg --release f31 local dnf -y install ~/packaging/x86_64/swift-lang-5.1.2-0.1.20191107git71def56.fc31.x86_64.rpm # To test sourcekit-lsp dnf -y install neovim cd ~ git clone https://github.com/indianajohn/dotfiles/ git checkout swift ln -s $HOME/dotfiles/.config $HOME/.config curl -fLo ~/.local/share/nvim/site/autoload/plug.vim --create-dirs \ https://raw.githubusercontent.com/junegunn/vim-plug/master/plug.vim nvim +'PlugInstall --sync' +qall nvim hello.swift # Type print("hello") # No standard library completions dnf rm swift-lang cd ~/packaging/swift-lang mkdir -p /usr/local/opt/swift tar -xzf ~/packaging/swift-lang/swift-5.1.2-fedora.tar.gz -C /usr/local/opt/swift export PATH=$PATH:/usr/local/opt/swift/usr/bin nvim hello.swift # Standard library completions work ``` Standard library completions look like, for example: ``` pri use of unresolved identifier 'pri' ~ print(items: Any...) function Void ~ print(items: Any..., separator: String, terminator: String) function Void ~ print(items: Any..., separator: String, terminator: String, to: &TextOutputStream) function Void ~ print(items: Any..., to: &TextOutputStream) function Void ~ private keyword ``` After typing "pri" in insert mode in Vim. The VSCode plugin likely uses the same mechanism, so the repro steps will be siimilar, as will the failure mode. Also, more clarity on why `make` is needed; if not, then in this minimal Docker environment the following error pops up: ``` ICU for C/C++ 61.2 is ready to be built. === Important Notes: === Data Packaging: library This means: ICU data will be linked with ICU. A static data library will be built. A shared data library will be built. To locate data: ICU will use the linked data library. If linked with the stub library located in stubdata/, the application can use udata_setCommonData() or set a data path to override. ** WARNING: make may not be GNU make. This may cause ICU to fail to build. Please make sure that GNU make is in your PATH so that the configure script can detect its location. checking the version of "make"... /root/packaging/swift-lang/swift-source/icu/icu4c/source//configure: line 9081: make: command not found ``` Hope this is helpful. Best Regards, John
Sorry, missed a `cd dotfiles` under `git clone https://github.com/indianajohn/dotfiles/`.
Hi John, Can you share your swift-5.1.2-fedora.tar.gz?
Okay, I think I've found the problem. Gonna try something tonight, but because it takes a full four+ hours to compile everything (sometimes longer :( ), it'll probably be sometime tomorrow night at the earliest when I can update this ticket with what I've found. The working theory is that I'm not including the version of clangd that is built as part of the toolchain, but instead am relying on the one from the clang-tools-extra package. I think this version of clangd doesn't understand swift, similarly to the way the packaged version of lldb also does not understand Swift and that's why I had to put it in /usr/libexec/swift-lldb so that it could live on the system alongside the regular one. The plan is to move the Swift version of clangd to the same directory (/usr/libexec/swift-lldb) and modify the source on sourcekit-lsp via a patch to look for clangd in that directory, instead of the default bin directory. The file in question to be changed is source kit-lsp/Sources/SKCore/Toolchain.swift in case anyone at home is keeping score.[1] Once I get that all worked out and rebuilt I'll see how it goes with neovim per John's info; I was able to get sourcekit working on neovim using an Ubuntu-based system and it worked fine so I have a baseline to compare against. [1] American baseball reference :)
Created attachment 1643094 [details] screenshot of working sourcekit-lsp on fedora Good news, it works! I built a test version with my patch and installed neovim and am able to get code-completion (the neovim suggestion is great because that I can test from the terminal). BTW, everything that I use to build Swift for Fedora is here: https://github.com/tachoknight/swift-lang-packaging-fedora. That's my raw, working repo, unlike the official Fedora one that doesn't include the shell scripts, python code, etc. I have a copr repo that I can make a build for testing, or I can just push it to the official repo and hopefully folks can test it before it goes live. :)
FYI, if you want to try it *now*, the RPMs are available at https://tachoknight.fedorapeople.org/swift-lang/. Those are the ones I used. Also, on my Fedora workstation, I installed it and sure enough, Visual Studio Code works great with sourcekit-lsp. One thing that probably needs to be considered is that installing the VSCode plugin isn't part of the installation process of Swift; the user would still have to go through the process of adding it to VSCode because there's no guarantee they have it installed or even want it, regardless of the fact they're installing Swift. Regardless, I think there's some rule about not automatically installing plugins to other apps; I may be wrong about this so I'll check.
FYI I just tested this on my machine and it works as expected. Thanks for looking into it.
Tested here, works as expected. Thanks for your work!
FYI I can't deploy a new version because there seems to be a weird Glibc bug in rawhide that is preventing it from building correctly. I am going to submit a bug report against the Glibc project and we'll see what happens, but as it is I'd suggest sticking with the version downloadable from the link above for the time being. The next Fedora release I can do will close this ticket so you'll know when it's ready. :)
FEDORA-2019-8316fb625b has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-8316fb625b
FEDORA-2019-c5c1f95c38 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-c5c1f95c38
FEDORA-EPEL-2019-603d479a37 has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-603d479a37
swift-lang-5.1.3-0.1.20191213git005fc1f.el8 has been pushed to the Fedora EPEL 8 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-603d479a37
swift-lang-5.1.3-0.1.20191213git005fc1f.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-8316fb625b
swift-lang-5.1.3-0.1.20191213git005fc1f.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-c5c1f95c38
swift-lang-5.1.3-0.1.20191213git005fc1f.el8 has been pushed to the Fedora EPEL 8 stable repository. If problems still persist, please make note of it in this bug report.