The title is a bit confusing as open-source separation of ... reads like source separation, which this is not. Rather, it is a pitch detection algorithm which also classifies the instrument the pitch originated with.
I think it's really neat, but the results look like it could take more time to fix the output than using a manual approach (if really accurate results are required).
Is “source separation” better known as “stem separation” or is that something else? I think the latter term is the one I usually hear from musicians who are interested in taking a single audio file and recovering (something approximating) the original tracks prior to mixing (i.e. the “stems”).
Audio Source Separation I think is the general term used in research. It is often applied to musical audio though, where you want to do stem separation - that's source separation where you want to isolate audio stems, a term referring to audio from related groups of signals, e.g. drums (which can contain multiple individual signals, like one for each drum/cymbal).
Stem separation refers to doing it with audio playback fidelity (or an attempt at that). So it should pull the bass part out at high enough fidelity to be reused as a bass part.
This is a partly solved problem right now. Some tracks and signal types can be unmixed easier than others, it depends on what the sources are and how much post-processing (reverb, side chaining, heavy brick wall limiting and so on)
I'd agree with the partly. I have yet to find one that either isolates an instrument as a separate file or removes one from the rest of the mix that does not negatively impact the sound. The common issues I hear are similar to the early internet low bit rate compression. The new "AI" versions are really bad at this, but even the ones available before the AI craze were still susceptible
I'm far (far) from an expert in this field, but when you think about how audio is quantized into digital form, I'm really not sure how one solves this with the current approaches.
That is: frequencies from one instrument will virtually always overlap with another one (including vocals), especially considering harmonics.
Any kind of separation will require some pretty sophisticated "reconstruction" it seems to me, because the operation is inherently destructive. And then the problem becomes one of how faithful the "reproduction" is.
This feels pretty similar to the inpainting/outpainting stuff being done in generative image editing (a la Photoshop) nowadays, but I don't think anywhere near the investment is being made in this field.
Very interested to hear anyone with expertise weigh in!
I won't say expertise, but what I've done recently:
1) used PixBim AI to extract "stems" (drums, bass, piano, all guitars, vocals). Obviously a lossless source like FLAC works better than MP3 here
2) imported the stems to ProTools.
3) from there, I will usually re-record the bass, guitars, pianos and vocals myself. Occassionally the drums as well.
This is a pretty good way I found to record covers of tracks at home, re-using the original drums if I want to, keeping the tempo of the original track intact etc. I can embellish/replace/modify/simplify parts that I re-record obviously.
It's a bit like drawing using tracing paper, you're creating a copy to the best of your ability, but you have a guide underneath to help you with placement.
It's not really digital quantisation that's the problem, but everything else that happens during mixing - which is a much more complicated process, especially for pop/rock/electronic etc., than just "sum all the signals together".
There's a bunch of other stuff that happens during and after summing which makes it much harder to reliably 100% reverse that process.
I didn't mean to say that quantization was the problem, just that you're basically trying to pick apart a "pixel" (to continue my image-based analogy) that is a composite of multiple sounds (or partially-transparent image layers).
I was sincere when I said:
> I'm really not sure how one solves this with the current approaches.
I was hoping someone would come along and say it is, in fact, possible. :)
I'm a Data Scientist currently consulting for a project in the Real Estate space (utilizing LLMs).
I understand the article is hyperbole for perhaps comedic purposes, and actually do perhaps 80% align with a lot of the authors views, but it's a bit much.
There is industry-changing tech which has become available, and many orgs are starting to grasp it. I won't deny that there's probably a large percentage of projects which fall under what the author describes, but these claims are doing a bit of a disservice to the legitimately amazing projects being worked on (and the competent people performing that work).
> I'm a Data Scientist currently consulting for a project in the Real Estate space (utilizing LLMs).
Consultants are obviously making huge amounts of money implementing LLMs for companies. The question is whether the company profits from it afterwards.
Time will tell, but I would cautiously say say yes.
Note that I don't usually work in that particular space (I prefer simple solutions and don't follow the hype), didn't sell myself using 'AI' (I was referred), and also would always tell a client if I believe there isn't much sense in a particular ask.
This particular project really uniquely benefits from this technology and would be much harder, if possible at all, otherwise.
Would you recommend to still get into freelance consulting (with a ML background) at this point in time? Or will the very technology you're consulting about, replace you very soon? AutoML, LLMs etc..
I'd say it depends on what your other options are. I don't think the technology will replace me soon, even at the rate I see it improving. At this point it's still a tool we can use to deliver faster, if we use it wisely.
Especially about ChatGPT et al. - I use it daily, but having the proper foundation to discern and verify its output shows me that it's still very far from being a competent programmer for any but the 'easy' tasks which have been solved hundreds of times over.
Like I hinted, I also view all of this hype sceptically. I dislike the 'we need AI in our org now!' types and am not planning on taking on projects if I don't see their viability. But there's obviously still a lot of demand and people offering services like those in TFA who're just looking to cash in, and that seems to work.
If you can find projects you believe you can make a difference in with your background, why not give it a shot?
Pretty cool! Tried it with RHCPs Dani California (https://lamucal.ai/songs/red-hot-chili-peppers/dani-californ...) and there's a lot of wrong chords still. Impressive nonetheless, and already quite useful in the song-part recognition (assuming it's all the ML)! Lyrics seem right too.
The source separation only seems to be available when downloading their app, which I didn't do, so I can't comment on that.
I downloaded and tried their app, experiencing the audio source separation feature, and ended up with five tracks (piano, vocals, drums, bass, and others). It sounds pretty good, but unfortunately, there is no guitar track.
I’ve been using Windows throughout my childhood and start of my CS career - now I use Windows for specific software (audio/music) and Linux for developing (about 8 years I guess). I had a 1-year stint with macOS because I was developing an iOS app, and have been the troubleshooter for people with macs at my previous job, so I consider myself somewhat ‘multilingual’ when it concerns OSs.
As a power user, Linux is just so much nicer. I constantly get frustrated, especially with macOS, about stuff that I can’t easily. In Linux my stuff works and if it doesn’t it can be made to work (usually). In Windows/Mac it’ll often take considerable effort to make the system work the way I want, or it’s just not possible.
I think with proprietary software ‘it just works’ is only a thing if you’re happy with the basic experience that is tuned to the average person. If you have more complex needs, you should be using Linux (and if you know your stuff or use the right distro, things will likely also ‘just work’).
Well, weird movements in games should be a thing of the past in the near future, as we can begin to extract motion capture data from videos of normal people acting normally.
I think it depends on the type of game you have, but I wouldn't underestimate this type of technology for say, open world games where it might make the game more immersive due to convincing realism.
> Well, weird movements in games should be a thing of the past in the near future, as we can begin to extract motion capture data from videos of normal people acting normally.
I think you misread my posts. We don't have awkward animations because our mocap isn't good enough, we have awkward animations because typical human motion looks awkward - our brains just mostly ignore that.
People are awkward; we don't actually want characters in games/movies/etc to be like real people. Very few movies, for example, would be well served by conversations frequently and for non-plot-related reasons being interrupted by loud noises, having people talk over each other and nonverbally try to figure out who gets to speak, having characters ask "What?" and then begin to reply without waiting for the answer because their brain caught up half a second later, etc.
But I don’t think I can agree with what you meant then - why would our brain mostly ignore it in real life but not in video games?
Where is the transition from something feeling real (in an immersive way) to us not liking it because it feels awkward, and why does it happen?
I’m imagining a “perfect simulation” game which is like real life in ways that matter/don’t get in your way in terms of gameplay - I think everyone would be awed (of course this can be argued though). What would need to degrade in terms of realism for it to seem awkward/not be immersive anymore?
I agree with the movie example, but in a game you don’t have to watch the mundane - it’s just background “noise” to make the world believable.
That already exists in some forms, most notably perhaps in Dubler 2.
I’m also working on an open source version of something similar, albeit quite involved and aiming at professional real-time performance and drumkit augmentation (if all goes well to share on HN this month for the first time :))
Recently released free short course (1 hour-ish minus the project) on getting structured output from LLMs. I found it quite useful to quickly get an idea of how this can be done!
I've had Tinnitus since I was 14 (when I went to a concert and stood in front of the speakers).
A couple of pieces of advice to people who might be struggling with their tinnitus:
1. You need to learn to cope with it - once you're used to it, it will mostly fade into the background and be manageable. Accepting that it'll never be silent again was very difficult, but that's the only thing hat helped me feel better in the end.
2. Wear ear plugs when it gets too loud! It's too easy to get irreversible damage to your hearing, and that's the only thing you can really do - prevent it.
Curiously, yesterday I woke up at night because the tinnitus had gotten louder again - stupidly, I played drums the other day at a jam session without earplugs. I could punch myself for that one, and see it as (yet another) wakeup call to be more careful.
Prolonged exposure to loud sounds, short extremely loud sounds (explosions), ototoxic drugs (some antibiotics, chemo..) and substances (toluene..) and viral infections that spread into inner ear can all cause cochlear damage and therefore tinnitus.
Covid vaccines do sometimes lead to increased tinnitus symptoms. But you can’t draw any conclusions from that, because getting COVID often leads to increased tinnitus. I’m not sure whether it’s known yet, but it very well may be that on balance there are fewer cases of tinnitus associated with the vaccine than with the virus. Also, BTW, flu vaccines and catching the flu both have reports of tinnitus increase. My theory: any inflammation event may be likely to increase tinnitus symptoms.
What difference does it make if covid causes it? Most people took the vaccine, pretty much everybody got covid anyway. The vaccine was voluntary (with a lot of unethical coercion).
What do you mean what difference does it make? Isn’t it clear that you cannot attribute tinnitus to the covid vaccine, if the covid virus (or any virus, or any vaccine) causes tinnitus as much or more often than the vaccine does?
If the per-capita rate of onset tinnitus symptoms when getting the vaccine is lower than the rate of onset tinnitus when catching covid, then the vaccine isn’t just not implicated, it’s effectively helpful at reducing tinnitus, as a byproduct of reducing cases and/or severity of covid illness.
Pretty much everyone got covid anyway. The vaccine was an unnecessary intervention that didn't do anything to stop covid and caused harm to many people. Just admit you were conned into taking it.
Haven't you noticed, after smearing, slandering and discriminating against those who chose not to take it, there is very little interest in following up with comparisons between the two groups? You would have expected scientists to be really interested in comparing the groups considering it was done under Trump's operation warp speed, and used a novel technology never before approved for human use. Instead the vaccine free are ignored. Tells me all I need to know.
I haven’t had covid, as far as I know. Of course I saw the recommendations to get the vaccine, but I didn’t see any smearing or slandering or conning, I don’t know what you’re referring to. The company I work for did allow vaccinated people to return to work before unvaccinated people, and to me that seemed like a prudent choice at the time, but the vaccine requirement was dropped at my work a long time ago. I’m sorry that your choice had unfairly negative consequences for you, especially if you felt bullied.
Please keep in mind that it doesn’t really matter if a lot of vaccinated people still got covid later. That was expected, because the ‘vaccine’ was not a covid cure. If the spread was slowed and the symptoms were reduced significantly, then the vaccine was successful. There has been lots of science on the unvaccinated, and it found that they died and were hospitalized from covid at much higher rates than vaccinated people. I believe there is plenty of science still happening on the secondary effects of the vaccine, so being unaware of it doesn’t mean it’s not happening. I’m not aware of “harm to many people” who took the vaccine. What harm are you referring to, and how many people were harmed, exactly? Did that harm happen less often or more often than to people who got covid? Just like with tinnitus, you can’t take anecdotes out of context when it happens to someone vaccinated if you don’t compare it to people who weren’t vaccinated.
BTW, while the covid vaccine was an untested vaccine at first (like all vaccines before trials), it did go through trials and it was not untested technology. mRNA therapies had been used in other non-covid trials for a decade, and tested against other infectious diseases for several years before the covid vaccine was developed. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7956899/
Think about it: at this point, we have more data on the effects of the covid vaccine than almost any other vaccine or drug in all of human history. Concerns about it being untested did maybe make sense in March of 2020, but it has now been thoroughly tested and so those concerns don’t make that much sense to hold on to anymore, right?
So I take it you weren’t very interested in talking about tinnitus?
And yes it does matter that people caught covid after as we were promised the opposite. Some people lost everything as a result of the totalitarian lockdowns so we could wait on the savior of a vaccine that turned out to b a complete failure. Some people lost their lives to the useless "vaccine".
No mRNA product had ever been approved for human us despite the decades of research. Why not?
And these "vaccines" caused tinnitus in many people, so it is relevant.
You’re talking about tabloid journalism?? The CNN link has no smearing, slandering or discrimination. It simply recommends the vaccine and says there are consequences to not taking it, which is true. It certainly has a lean toward recommending the vaccine, and it’s true that there could be consequences to taking it. That’s just a bit of bias, not slander or smear. Do you have personal experience that’s making you upset about the Covid vaccine?
Yes the vaccine might be relevant to tinnitus, and like I said Covid and the flu and catching a cold has also lead to many many people saying those things caused tinnitus. Claiming the vaccine itself is the direct cause of the tinnitus is unjustified and therefore misleading. If the cause is inflammation, then anything that causes inflammation can lead to tinnitus, whether it’s a vaccine or catching a flu or hitting your head. The tinnitus may have nothing to do with the Covid vaccine specifically, it may be nothing more than a byproduct of activating your immune system.
I got vaccinated three times (with Pfizer). No ill effects on my hearing/tinnitus (I was monitoring it). Then I got COVID (Omicron), was quite sick for several days (lost smell) and it seems the tinnitus worsened a bit in one ear. So... your mileage may vary, as with everything.
They do make the tinnitus more noticeable for sure! By blocking out sounds which would otherwise “mask” the tinnitus, it can become noticeable if it’s otherwise pretty quiet. But in conditions where you really need earplugs I think you still won’t really notice it that much when wearing earplugs (very loud situations).
I actually started recording with S1 v1 in my teens because I got it for cheap. After some longer breaks, I kept coming back to it (and buying updates) - it really is good!
Compared with the Ableton workflow my bandmates are using nowadays, I still prefer S1.
I think it's really neat, but the results look like it could take more time to fix the output than using a manual approach (if really accurate results are required).