0

I have a data set that includes image arrays, point clouds, audio waveforms, and plain numerical data. I want to use unsupervised learning to group the data based on relatedness. So, if the audio and video are changing simultaneously, then the algorithm should group them together. If I am not mistaken, this is called heterogenous data clustering.

My data looks like this:

Audio
[[first frame's audio samples] [second set] [third set] ...

Video
4D array of shape (1654, 500, 128, 3)
              # of imgs   l    w   channels

Gyro
[[roll, pitch, yaw], [roll, pitch, yaw], [roll, pitch, yaw], [roll, pitch, yaw].....

And a bunch of 1D numerical data

Is there a way to do this?

I am new to this so if there is some critical information missing, let me know.

DragonflyRobotics
  • 135
  • 7
  • What are your findings so far? Why can't you use k-means? What's the problem you're having? Provide more details about why you're stuck. – nbro Mar 03 '22 at 08:52
  • From my research, k-means is used for numerical data and not high-dimensional data that changes shapes across categories. – DragonflyRobotics Mar 04 '22 at 22:05
  • See this paper [Head-to-head comparison of clustering methods for heterogeneous data: a simulation-driven benchmark](https://www.nature.com/articles/s41598-021-83340-8) or [PyMix - The Python mixture package - a tool for clustering of heterogeneous biological data](https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-11-9). They could be useful. Feel free to provide an answer to your own question, once you find it. – nbro Mar 06 '22 at 00:26
  • Thanks! I will check it out. – DragonflyRobotics Mar 06 '22 at 23:56
  • Is there a python implementation of this? This seems theoretical. – DragonflyRobotics Mar 06 '22 at 23:58
  • I see some example code but I don't know how to make it work with my data. – DragonflyRobotics Mar 07 '22 at 00:06
  • I don't know if there's any Python implementation. Check out on Github or Gitlab. – nbro Mar 10 '22 at 09:18

0 Answers0