Freebsd compat #2834

shamazmazum · 2017-04-10T06:28:08Z

Hi. This is my second set of commits for FreeBSD compatibility. The first is for hidapi here: supercollider/hidapi#8

With these commits you will able to build Supercollider on FreeBSD smoothly (provided you have Qt5Positioning which is not in the port collection). Btw., what is Qt5Positioning for? Is it really used in Supercollider?

mossheim

Thanks for this! There are some issues:

This fails to build now. Looking over the Travis logs that's because you have some #elifs where there should be #elses. It's also because you missed a thread declaration in UIUGens.mm.
Instead of decomposing usage of thread to std::thread, I think it might be better to go the other way and typedef SC_Thread std::thread; like SC_Lock. This makes it easier to test/swap out thread implementations other than std::thread. We can discuss.

mossheim · 2017-04-10T12:11:46Z

QtCollider/QObjectProxy.cpp

-#else
+#elif defined __FreeBSD__
+# include <stdlib.h>
+#elif


Should be #else

mossheim · 2017-04-10T12:11:59Z

QtCollider/primitives/prim_QObject.cpp

-#else
+#elif defined __FreeBSD__
+# include <stdlib.h>
+#elif


Should be #else

mossheim · 2017-04-10T12:20:02Z

Also forgot to say, this PR should definitely be built on Windows before merging.

shamazmazum · 2017-04-10T14:47:13Z

I updated my patches. Travis is still unhappy. One of the tests failed:

[!]   TestNoOps:test_arithmetics (0.0s)ERROR: UnitTest halted with onFailure handler.

I do not see how this is relevant to what I have done.
Unfortunately, I cannot build on Windows(

mossheim · 2017-04-10T16:11:59Z

lang/LangSource/PyrObject.cpp

@@ -1169,7 +1169,7 @@ void buildBigMethodMatrix()
 	double t0 = elapsedTime();
 #endif

-	const int hw_concurrency    = thread::hardware_concurrency();
+	const int hw_concurrency    = std::thread::hardware_concurrency();


(missed one here)

Thanks. I did that intentionally. I do not know C++, so I thought declared type will not work as a namespace. (Or you will not be able to call its class methods, or whatever they called). Will fix it tomorrow.

mossheim · 2017-04-10T16:14:21Z

Thanks! Hm... UnitTest uses forking, so it could be related to a change to threads. I've restarted the Travis build in case it was a fluke.

I can build on Windows, I was just making a note for myself or anyone else interested. :)

shamazmazum · 2017-04-10T20:12:43Z

That's strange. No changes were done to program logic, just renaming. Oh, I see green check. That is good

bagong · 2017-04-11T09:29:38Z

So Windows builds fine with this VS2013/2015 MinGW 4.8 Msys2 64. So no build obstacles. Can somebody explain what that means implementation-wise? How would one go about to find out what this actually means? ;) The usual start sc, boot server, do ().play is not enough, I guess? ;)

It's nice to have an occasional update of the support of more exotical Unixes I am curious: which Qt-version does FreeBSD use, which compiler and compiler version? Are FreeBSD, OpenBSD and NetBSD close enough to assume they might be covered as well? As to the necessity of Qt5Positioning, I do think the SC build currently requires a few Qt-libraries which are actually unused. I guess it happened in an optimistic moment Qt-support in SC...

There are source files which include <sys/sysctl.h> and have "using std::thread" at the same time. This causes name clash on FreeBSD. Define a new type SC_Thread to avoid this.

shamazmazum · 2017-04-11T14:42:00Z

Changed that last std::thread. All must be good now. Can this be merged now? Thanks

shamazmazum · 2017-04-11T14:53:38Z

Answering to @bagong. I currently have Qt 5.7.1 from ports. FreeBSD 11's base system compiler is currently clang 3.8, you can also install llvm+clang up to 4.0. GCC is also present both in FreeBSD's source tree and ports, but is not built by default. By myself, I do not have any need in GCC.

Are FreeBSD, OpenBSD and NetBSD close enough to assume they might be covered as well

All BSD flavors develop independently, but sometimes something from one BSD goes to another. I cannot say if my patch is useful for any other BSD (or even if this problem occurs in any other variant of BSD).

mossheim · 2017-04-11T16:37:36Z

So no build obstacles. Can somebody explain what that means implementation-wise? How would one go about to find out what this actually means?

@bagong The main change is just resolving a namespace collision that happens on FreeBSD. Building on Windows was not strictly necessary but interactions between namespaces and OS headers can be affected in odd ways sometimes. A good illustration of why to never put using std::x in a header...

mossheim · 2017-04-11T16:40:12Z

And just to clarify, the reason why I did think it was a good idea was because a file that was only included while building on macOS showed a missed replacement.

mossheim

Thanks again!

mossheim · 2017-04-11T16:42:00Z

This is ready to merge IMO, I'd rather someone else look it over too. @danstowell @llloret? (you approved the hidapi changes).

danstowell

All fine except for one I have a comment on

danstowell · 2017-04-11T16:49:44Z

lang/LangPrimSource/PyrPrimitive.cpp

@@ -4264,7 +4264,7 @@ void initMIDIPrimitives();
 	initMIDIPrimitives();
 #endif

-#if !defined(_WIN32) && !defined(SC_IPHONE) && !defined(__OpenBSD__) && !defined(__NetBSD__) && !defined(__APPLE__)
+#if defined __linux__


Are we confident that this change is safe? It's probably better to be conservative and not assume that linux is defined on all platforms that can use LID (i.e. the old way). However I don't know if there are cases in the wild where this change would be a problem. Personally I'd say stick with the verbose way, exclusion rather than inclusion.

@danstowell the LID file is only included if LINUX is defined in Cmake, which is only defined if CMAKE_SYSTEM_NAME is "Linux", so I think this is safe. Do you see any issues arising from that? __linux__ is after all only defined on the Linux kernel.

I raised the question because I wanted this to be a conscious decision, rather than because I had strong views. I'll go along with your argument, yes. No need to hold up the merge.

bagong · 2017-04-12T02:08:37Z

It breaks in supernova, but that's probably intentional...
Which brings me to another question, @shamazmazum , how do you run jackd, or which audio backend do you use for sc? I can get jackd to start with an oss backend, and scsynth uses it, but I get frequencies almost a fifth higher than they should be?

mossheim · 2017-04-12T03:00:29Z

It breaks in supernova, but that's probably intentional...

@bagong More info please

bagong · 2017-04-12T09:43:00Z

First break is in server/supernova/server/main.cpp:

[ 34%] Building CXX object server/supernova/CMakeFiles/supernova.dir/server/main.cpp.o
/usr/home/rainer/Projects/supercollider/server/supernova/server/main.cpp:229:2: error: platform not supported
#error platform not supported
 ^
1 error generated.
*** Error code 1

Stop.

Which makes sense, as main.cpp 211ff 'resolve home' only covers Linux, Apple, and Windows, other unixes are not covered, although that would be easy in that particular case.

There are a few places in supernova main.cpp where non-Apple/Windows just covers Linux. If you create an outlet for FreeBSD in those cases the one way or the other, it will ultimately break in the linker stage (not necessarily at 99%, but always with reference to sc_ugen_factory:load_plugin_folder/close_handles):

[ 99%] Linking CXX executable supernova
libsupernova.a(sc_ugen_factory.cpp.o): In function `nova::sc_ugen_factory::load_plugin_folder(boost::filesystem::path const&)':
/usr/home/rainer/Projects/supercollider/server/supernova/sc/sc_ugen_factory.cpp:(.text+0x10b0): undefined reference to `nova::sc_ugen_factory::load_plugin(boost::filesystem::path const&)'
libsupernova.a(sc_ugen_factory.cpp.o): In function `nova::sc_ugen_factory::~sc_ugen_factory()':
/usr/home/rainer/Projects/supercollider/server/supernova/sc/sc_ugen_factory.cpp:(.text._ZN4nova15sc_ugen_factoryD2Ev[_ZN4nova15sc_ugen_factoryD2Ev]+0xb): undefined reference to `nova::sc_ugen_factory::close_handles()'
c++: error: linker command failed with exit code 1 (use -v to see invocation)
--- server/supernova/supernova ---
*** [server/supernova/supernova] Error code 1

make[2]: stopped in /usr/home/rainer/Projects/supercollider/build
1 error

Maybe that would be easy to fix. But the fact that FreeBSD doesn't seem support ALSA (?) made me wonder more generally, how SC would cope with the audio backend in FreeBSD land.

Btw, there is a SC 3.8 (!) package in FreeBSD, I was surprised to see. I haven't tried it yet, should do so. But I'd like to hear from @shamazmazum first, how he actually runs the server before going through another trial-error odyssey.

shamazmazum · 2017-04-12T14:48:01Z

@bagong I run server with "Server.local.boot" or "s.boot".
Supernova is built by default only on Linux, isn't it? So I did not try to port it. Btw, thanks for the hint to search in port collection. I think it's relatively new port, that's why I missed it.

@danstowell What platforms are currently supported? Windows, Linux, Apple family, BSD family, right? From that list only Linux has LID, as I can understand. Look at the commit history. For some reason it started with few !defined and with time it became very long definition.

At first it was !defined(SC_WIN32) && !defined(SC_IPHONE) && !defined(__OpenBSD__)

With 0b623d9 is became !defined(SC_WIN32) && !defined(SC_IPHONE) && !defined(__OpenBSD__) && !defined(__APPLE__) and what is more important, separated from initSerialPrimitives(). Later with 14f1ac7 it became !defined(SC_WIN32) && !defined(SC_IPHONE) && !defined(__OpenBSD__) && !defined(__NetBSD__) && !defined(__APPLE__) .

My point is that at first it had some sense: both linux and apple had initSerialPrimitives(). So you could write #if defined(___linux___) || defined(___APPLE__) or #if !defined(SC_WIN32) && !defined(SC_IPHONE), it was the same. Later, when Serial Primitives got their own section, condition became by one !defined() longer rather than be just one defined(). Then another !defined()conditions were added just for historical reasons.

mossheim · 2017-04-12T14:51:32Z

Maybe that would be easy to fix.

It would be. Some code paths are only turned on by the macro DLOPEN in Supernova, which is only defined via CMake if the system has libdl. But FreeBSD's dlopen function is in libc, so it could technically support that code, the logic is just a little off.

See primecoin/primecoin#22 and aspnet/KestrelHttpServer#94 for similar discussions.

bagong · 2017-04-12T15:02:53Z

@bagong I run server with "Server.local.boot" or "s.boot".
Supernova is built by default only on Linux, isn't it? So I did not try to port it.

There is no obligation to build SuperNova of course, but I think it's good to keep it in mind, at least for us here.

As to the server question: I assumed that you used s.boot ;) My question is about the backend used by jackd. I couldn't just start jackd (to wouldn't "just boot"), I had to define arguments first to make it boot with oss and use the sample rate my hardware was set to. But my main question: does the server play the frequencies you ask it to play? I get a wide difference between what I set and what I hear. Is that not so for you?

Btw, thanks for the hint to search in port collection. I think it's relatively new port, that's why I missed it.

Pretty similar ;)

shamazmazum · 2017-04-12T15:31:51Z

@brianlheim I was able to build supernova, but got many runtime errors. First of all it cannot find any UGen and also report some errors about threads. Already removed that build, so cannot paste them here. Btw, are UGens loaded with dlopen()?

@bagong Jack starts just well with jack -r -d oss. I tried to play Sin wave on 440 Hz and got 440 Hz exactly. (But then you set another sampling frequency on Jack (default is 48kHz), your wave sounds like clipped, suppose that's bug in jack).

bagong · 2017-04-12T18:24:28Z

Ah, okay @shamazmazum . What I see is that playback seems to be at a constant samplerate, regardless of what sc and jackd think it is. At 44.100 the audible frequency is about a fifth higher, at 48.000 a minor third, at 88.200 it's a lot lower, at 96.000 even lower. Same when playing back audio files. If you get oddities with the sampleRate switch too, it might be worth to inspect that a bit more. I am not saying it's the fault of your port, of course not, but - while it could be my environment (virtual machine) - it might also be that the chain scsynth -> jackd -> oss -> audiohardware doesn't work as it should on NetBSD. Chromium, btw, plays back the 440Hz test tone in a Youtube video correctly for all of 44.100 up to 96000 on my system. Doesn't mean much, but at least shows that it could work ;).

This is more informational of course, but a bit more testing is probably advisable before we assume FreeBSD is a supported platform now. I's suspect 'can be built now' is more apt. And from that point of view it might be nice fix the supernova build as well. We have a similar situation on Raspbian, where supernova can be built, but produces a lot of runtime errors. If the build hurdle is out of the way, somebody might think it worthwhile to have a look what actually happens (although that's likely harder to fix than the build)...

This btw is what jackd posts on startup:

Copyright 2001-2009 Paul Davis, Stephane Letz, Jack O'Quinn, Torben Hohn and others.
jackd comes with ABSOLUTELY NO WARRANTY
This is free software, and you are welcome to redistribute it
under certain conditions; see the file COPYING for details

JACK compiled with System V SHM support.
loading driver ..
oss_driver: /dev/dsp : 0x10/2/63257 (4096)
oss_driver: indevbuf 4096 B, outdevbuf 4096 B
oss_driver: not using barrier mode, (single thread)

And we should mention that -r means 'none-realtime mode`

shamazmazum · 2017-04-12T18:57:29Z

@bagong

oss_driver: /dev/dsp : 0x10/2/63257 (4096)

My is

oss_driver: /dev/dsp : 0x1000/2/44100 (4096)

so I thought the third number is the sample rate. But 63257 seems senseless to me. Btw. jackd is started on that sample rate, which is currently set on virtual channel by audio driver, no matter what jackd is told to set (e.g. jackd -r -d oss -r 192000). So no wonder that you hear tune on different frequency than it should be. And if you set sample rate to 48000 on virtual channel (by sysctl dev.pcm.X.play.vchanrate=48000) you will probably hear tunes as they should be. I think SuperCollider can't do anything about it. Supercollider<->jackd interaction is working fine, it's jackd<->kernel interaction which is broken.

bagong · 2017-04-12T19:51:33Z

Yea, it's the sample rate, you are right, also fit's my hear experience (higher below 63256, lower above 63256). Sysctl dev.pcm gives for me:

$ sysctl dev.pcm
dev.pcm.0.bitperfect: 0
dev.pcm.0.buffersize: 16384
dev.pcm.0.ac97rate: 47442
dev.pcm.0.rec.vchanformat: s16le:2.0
dev.pcm.0.rec.vchanrate: 63256
dev.pcm.0.rec.vchanmode: fixed
dev.pcm.0.rec.vchans: 1
dev.pcm.0.play.vchanformat: s16le:2.0
dev.pcm.0.play.vchanrate: 63256
dev.pcm.0.play.vchanmode: fixed
dev.pcm.0.play.vchans: 1
dev.pcm.0.eapd: 1
dev.pcm.0.%parent: pci0
dev.pcm.0.%pnpinfo: vendor=0x8086 device=0x2445 subvendor=0x1ab8 subdevice=0x0400 class=0x040100
dev.pcm.0.%location: slot=31 function=4 dbsf=pci0:0:31:4
dev.pcm.0.%driver: pcm
dev.pcm.0.%desc: Intel ICH2 (82801BA)
dev.pcm.%parent:

Trying to set dev.pcm.0.rec.vchanrate on the command line seems not possible on my system. Maybe in some configuration file, which is used at boot time. Or it's broken all together (on my system)? Browsing Google seems to show quite a few postings with this kind of problem.
Makes sense that this is a problem between hardware and oss. Would probably be nice to have a place where one could store such information regarding SC on FreeBSD: cannot switch/detect hardware sample rate.

Let's leave it at that, just something I bumped into when trying to reproduce what you were doing. Really has nothing to do with your PR. Except one might feel inspired to inspect SCs limitations on FreeBSD if one facilitates its use ;)

ghost · 2017-04-26T21:27:00Z

every single plattform that supercollider may be optimized for, is a plus for the possibility to bring in new users. so i vote fixing that up

mossheim · 2017-05-18T22:44:58Z

What remains to be done here? I think modulo @danstowell's concern this is near mergeable, right?

danstowell · 2017-05-18T22:52:39Z

Go for it, IMHO

mossheim · 2017-05-18T23:39:45Z

Awesome. Thanks again for the discussion everyone, and for the code @shamazmazum

Link UIUGens against X11

f8f2726

mossheim suggested changes Apr 10, 2017

View reviewed changes

shamazmazum force-pushed the freebsd-compat branch from 63cca1e to adb042c Compare April 10, 2017 14:00

mossheim reviewed Apr 10, 2017

View reviewed changes

shamazmazum added 3 commits April 11, 2017 17:26

Avoid name clash on FreeBSD platform.

dbac7b1

There are source files which include <sys/sysctl.h> and have "using std::thread" at the same time. This causes name clash on FreeBSD. Define a new type SC_Thread to avoid this.

alloca is in stdlib.h on FreeBSD

a02586e

Remove reference to linux-specific stuff from any other platform

ccb379f

shamazmazum force-pushed the freebsd-compat branch from adb042c to ccb379f Compare April 11, 2017 14:38

mossheim approved these changes Apr 11, 2017

View reviewed changes

danstowell reviewed Apr 11, 2017

View reviewed changes

mossheim added comp: build CMake build system os: FreeBSD labels Apr 17, 2017

mossheim merged commit 2a105f1 into supercollider:master May 18, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Freebsd compat #2834

Freebsd compat #2834

shamazmazum commented Apr 10, 2017

mossheim left a comment

mossheim Apr 10, 2017

mossheim Apr 10, 2017

mossheim commented Apr 10, 2017

shamazmazum commented Apr 10, 2017

mossheim Apr 10, 2017

shamazmazum Apr 10, 2017

mossheim commented Apr 10, 2017

shamazmazum commented Apr 10, 2017

bagong commented Apr 11, 2017

shamazmazum commented Apr 11, 2017

shamazmazum commented Apr 11, 2017

mossheim commented Apr 11, 2017

mossheim commented Apr 11, 2017

mossheim left a comment

mossheim commented Apr 11, 2017

danstowell left a comment

danstowell Apr 11, 2017

mossheim May 18, 2017

danstowell May 18, 2017

bagong commented Apr 12, 2017 •

edited

Loading

mossheim commented Apr 12, 2017

bagong commented Apr 12, 2017

shamazmazum commented Apr 12, 2017

mossheim commented Apr 12, 2017

bagong commented Apr 12, 2017 •

edited

Loading

shamazmazum commented Apr 12, 2017 •

edited

Loading

bagong commented Apr 12, 2017 •

edited

Loading

shamazmazum commented Apr 12, 2017

bagong commented Apr 12, 2017

ghost commented Apr 26, 2017

mossheim commented May 18, 2017

danstowell commented May 18, 2017

mossheim commented May 18, 2017

Freebsd compat #2834

Freebsd compat #2834

Conversation

shamazmazum commented Apr 10, 2017

mossheim left a comment

Choose a reason for hiding this comment

mossheim Apr 10, 2017

Choose a reason for hiding this comment

mossheim Apr 10, 2017

Choose a reason for hiding this comment

mossheim commented Apr 10, 2017

shamazmazum commented Apr 10, 2017

mossheim Apr 10, 2017

Choose a reason for hiding this comment

shamazmazum Apr 10, 2017

Choose a reason for hiding this comment

mossheim commented Apr 10, 2017

shamazmazum commented Apr 10, 2017

bagong commented Apr 11, 2017

shamazmazum commented Apr 11, 2017

shamazmazum commented Apr 11, 2017

mossheim commented Apr 11, 2017

mossheim commented Apr 11, 2017

mossheim left a comment

Choose a reason for hiding this comment

mossheim commented Apr 11, 2017

danstowell left a comment

Choose a reason for hiding this comment

danstowell Apr 11, 2017

Choose a reason for hiding this comment

mossheim May 18, 2017

Choose a reason for hiding this comment

danstowell May 18, 2017

Choose a reason for hiding this comment

bagong commented Apr 12, 2017 • edited Loading

mossheim commented Apr 12, 2017

bagong commented Apr 12, 2017

shamazmazum commented Apr 12, 2017

mossheim commented Apr 12, 2017

bagong commented Apr 12, 2017 • edited Loading

shamazmazum commented Apr 12, 2017 • edited Loading

bagong commented Apr 12, 2017 • edited Loading

shamazmazum commented Apr 12, 2017

bagong commented Apr 12, 2017

ghost commented Apr 26, 2017

mossheim commented May 18, 2017

danstowell commented May 18, 2017

mossheim commented May 18, 2017

bagong commented Apr 12, 2017 •

edited

Loading

bagong commented Apr 12, 2017 •

edited

Loading

shamazmazum commented Apr 12, 2017 •

edited

Loading

bagong commented Apr 12, 2017 •

edited

Loading