Skip to content

Commit dadb838

Browse files
Merge pull request #588 from BigBrownMF69/feat/artist-watch-musicbrainz
Read Artist Watch discographies from MusicBrainz too
2 parents c8b04f6 + 9c5213d commit dadb838

7 files changed

Lines changed: 1294 additions & 35 deletions

File tree

‎docs/features/playlist-monitor.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,8 @@ On each sweep Downtify lists the artist's discography, compares it against the r
7676

7777
The discography is read from **Deezer** as well as YouTube Music, because YouTube Music's artist page doesn't always list everything: Drake's leaves out *Views*, for example. The Deezer artist is found by name on the watch's first check (the most followed one with exactly that name) and kept from then on. A release both catalogues list is downloaded once: its songs are linked to the files the other one brought in. Deezer also lists the singles an artist is only a guest on, like another artist's `Song (feat. Drake)`; those are skipped, while a release the artist shares top billing on (a joint album or single) is downloaded. If one catalogue doesn't answer, the sweep goes ahead with the other. When an existing **New releases only** watch reads Deezer for the first time, everything Deezer lists at that point is skipped too, like the back catalogue was.
7878

79+
**MusicBrainz** (musicbrainz.org, the open music encyclopedia Lidarr is built on) fills in what both of them miss: mixtapes, older and rarer releases. Each of its *release groups* is one album, EP or single with all of its editions; Downtify downloads one official edition of each: explicit rather than clean, the one with the most tracks (a deluxe edition holds the standard one). A group with no official edition (a bootleg: a big rapper has dozens of fan-made "street" compilations of songs he already released) is skipped. A group YouTube Music or Deezer already lists under the same name and type is left to them. The MusicBrainz artist is found on the watch's first check through the artist's Deezer page or YouTube Music channel (MusicBrainz links to both), else by name, but only when exactly one artist has exactly that name, and kept from then on. As with Deezer, the groups an artist is only a guest on (`Song (feat. Drake)`) are skipped. Mixtapes and soundtracks are downloaded; live albums, compilations, remixes, DJ mixes and demos, which mostly repeat the artist's songs, are not. MusicBrainz asks for at most one request a second, so a big artist's first check takes a few minutes; later checks cost a request or two. Settings `artist_watch_musicbrainz` (on) and `artist_watch_musicbrainz_kinds` (`["mixtape", "soundtrack"]`; also `live`, `compilation`, `remix`, `dj-mix`, `demo`) control it.
80+
7981
::: info Why the discography comes from YouTube Music
8082
Spotify's public embed exposes an artist's name and a top-tracks preview, but **not** their discography — reading that would need Spotify API credentials, which Downtify deliberately doesn't use. So a Spotify artist link is resolved to its name and matched to the same artist on YouTube Music, which is also where the audio is fetched from. The upside: every release Downtify can see is one it can actually download. The trade-off: the link follows the YouTube Music artist with exactly that name (case and punctuation aside), YouTube Music's best match when several share it. For a common name, check that the watch is the artist you meant, and paste the YouTube Music artist URL directly if it isn't. A Spotify artist whose exact name no YouTube Music artist has can only be added that way: Downtify doesn't guess.
8183
:::

‎downtify/api.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -703,6 +703,11 @@ def _clamp_retry_budget(value: Any) -> int:
703703
# YouTube Music's artist page doesn't list every release of every
704704
# artist (see ``monitor._artist_discography``).
705705
'artist_watch_deezer': True,
706+
# ...and MusicBrainz's release groups that neither lists (mixtapes,
707+
# older releases), with the secondary types let in by
708+
# ``artist_watch_musicbrainz_kinds`` (``musicbrainz.KINDS`` names).
709+
'artist_watch_musicbrainz': True,
710+
'artist_watch_musicbrainz_kinds': ['mixtape', 'soundtrack'],
706711
'max_parallel_downloads': 3,
707712
'download_delay_seconds': 0,
708713
'download_retry_budget_minutes': 0,

‎downtify/library_songs.py‎

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,12 @@
2020
2121
Deezer adds a few more spellings of the very same recording: ``(Album
2222
Version)``, ``(Explicit Version)``, ``(Explicit)``, ``- Original Mix``
23-
and the like are folded away too (:data:`_SAME_RECORDING_TAG`). Words
24-
that name another recording - ``Clean``, ``Radio Edit``, ``Live``,
25-
``Remix`` - are kept, and the length must agree as well.
23+
and the like are folded away too (:data:`_SAME_RECORDING_TAG`). So is
24+
the film a soundtrack song comes from, which Indian labels add on one
25+
catalogue and not another: ``Tum Hi Ho (From "Aashiqui 2")`` is the
26+
``Tum Hi Ho`` MusicBrainz and Spotify list. Words that name another
27+
recording - ``Clean``, ``Radio Edit``, ``Live``, ``Remix`` - are kept,
28+
and the length must agree as well.
2629
"""
2730

2831
from __future__ import annotations
@@ -42,10 +45,12 @@
4245
SAME_SONG_SECONDS = 3.0
4346

4447
#: Title tags that name no other recording: the album's (explicit)
45-
#: version, as one catalogue spells it and another leaves out.
48+
#: version, or the film a song is from, as one catalogue spells it and
49+
#: another leaves out.
4650
_SAME_RECORDING_TAG = re.compile(
4751
r'\s*(?:[([]\s*(?:album|explicit|original|lp)(?:\s+(?:version|mix))?\s*'
48-
r'[)\]]|\s-\s(?:album|explicit|original|lp)\s+(?:version|mix)\s*$)',
52+
r'[)\]]|\s-\s(?:album|explicit|original|lp)\s+(?:version|mix)\s*$'
53+
r'|[([]\s*from\s[^)\]]*[)\]])',
4954
re.IGNORECASE,
5055
)
5156

‎downtify/monitor.py‎

Lines changed: 142 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414

1515
from loguru import logger
1616

17-
from . import deezer, m3u, providers, spotify
17+
from . import deezer, m3u, musicbrainz, providers, spotify
1818
from .artist_release_store import ArtistReleaseStore
1919
from .cover_cache import CoverArtCache
2020
from .download_policy import DownloadPolicy, DownloadPolicyBlocked
@@ -332,6 +332,9 @@ class MonitoredPlaylist:
332332
# Artist watches: the artist's Deezer id, whose discography is read
333333
# alongside YouTube Music's ('' until found).
334334
deezer_id: str = ''
335+
# Artist watches: the artist's MusicBrainz id, whose release groups
336+
# fill in what the other catalogues miss ('' until found).
337+
musicbrainz_id: str = ''
335338

336339
@property
337340
def source(self) -> str:
@@ -491,6 +494,15 @@ def _init_db(self) -> None:
491494
)
492495
except sqlite3.OperationalError:
493496
pass
497+
# Migration: the same artist on MusicBrainz (see
498+
# _musicbrainz_releases).
499+
try:
500+
conn.execute(
501+
'ALTER TABLE monitored_playlists ADD COLUMN '
502+
"musicbrainz_id TEXT NOT NULL DEFAULT ''"
503+
)
504+
except sqlite3.OperationalError:
505+
pass
494506

495507
def add_playlist(
496508
self,
@@ -606,6 +618,7 @@ def update_playlist(
606618
'new_only',
607619
'baseline_pending',
608620
'deezer_id',
621+
'musicbrainz_id',
609622
}
610623
updates = {k: v for k, v in kwargs.items() if k in allowed}
611624
if not updates:
@@ -640,7 +653,8 @@ def retarget_playlist(
640653
"""UPDATE monitored_playlists
641654
SET spotify_id = ?, name = ?, url = ?,
642655
last_checked = NULL, last_track_count = 0,
643-
baseline_pending = new_only, deezer_id = ''
656+
baseline_pending = new_only, deezer_id = '',
657+
musicbrainz_id = ''
644658
WHERE id = ?""",
645659
(spotify_id, name, url, playlist_id),
646660
)
@@ -938,6 +952,11 @@ def _row_to_playlist(row: sqlite3.Row) -> MonitoredPlaylist:
938952
else False
939953
),
940954
deezer_id=str(row['deezer_id'] or '') if 'deezer_id' in keys else '',
955+
musicbrainz_id=(
956+
str(row['musicbrainz_id'] or '')
957+
if 'musicbrainz_id' in keys
958+
else ''
959+
),
941960
)
942961

943962

@@ -1547,6 +1566,8 @@ def _cb(pct: float, message: str, provider: Optional[str] = None) -> None:
15471566
#: Prefix of a release id from Deezer, telling it apart from a YouTube
15481567
#: Music browse id in ``seen_albums`` and ``artist_release_tracks``.
15491568
DEEZER_RELEASE = 'deezer:'
1569+
#: Prefix of a release id from MusicBrainz (a release group's id).
1570+
MUSICBRAINZ_RELEASE = 'mb:'
15501571

15511572

15521573
def _artist_discography(
@@ -1556,35 +1577,44 @@ def _artist_discography(
15561577
) -> list[dict[str, Any]]:
15571578
"""Every release of a watched artist that a catalogue lists.
15581579
1559-
YouTube Music's artist page doesn't always list everything - Drake's
1560-
shows only his albums from 2021 on - so, with ``artist_watch_deezer``
1561-
on, Deezer's discography of the same artist is read too. A release
1562-
both list is processed twice, but its songs are downloaded once: the
1563-
sweep links a song the library (or this sweep) already has. Raises
1564-
only when no source answered.
1580+
YouTube Music's artist page doesn't always list everything - it
1581+
leaves out Drake's *Views* - so, with ``artist_watch_deezer`` on,
1582+
Deezer's discography of the same artist is read too, and with
1583+
``artist_watch_musicbrainz`` on, MusicBrainz's release groups that
1584+
neither of them lists (mixtapes, older or rarer releases - see
1585+
:func:`_musicbrainz_releases`). A release two catalogues list is
1586+
processed twice, but its songs are downloaded once: the sweep links
1587+
a song the library (or this sweep) already has. Raises only when no
1588+
source answered.
15651589
"""
15661590

15671591
found: list[dict[str, Any]] = []
1568-
failed = 0
1569-
try:
1570-
found += providers.artist_albums_from_channel_id(playlist.spotify_id)
1571-
except Exception:
1572-
failed += 1
1573-
logger.opt(exception=True).warning(
1574-
'YouTube Music discography failed for artist "{}"', playlist.name
1575-
)
1576-
if not settings.get('artist_watch_deezer'):
1577-
if failed:
1578-
raise RuntimeError('No discography source answered')
1579-
return found
1580-
try:
1581-
found += _deezer_releases(playlist, db)
1582-
except Exception:
1583-
failed += 1
1584-
logger.opt(exception=True).warning(
1585-
'Deezer discography failed for artist "{}"', playlist.name
1592+
asked = failed = 0
1593+
sources = [
1594+
(
1595+
'YouTube Music',
1596+
lambda: providers.artist_albums_from_channel_id(
1597+
playlist.spotify_id
1598+
),
15861599
)
1587-
if failed == 2:
1600+
]
1601+
if settings.get('artist_watch_deezer'):
1602+
sources.append(('Deezer', lambda: _deezer_releases(playlist, db)))
1603+
if settings.get('artist_watch_musicbrainz'):
1604+
sources.append((
1605+
'MusicBrainz',
1606+
lambda: _musicbrainz_releases(playlist, db, settings, found),
1607+
))
1608+
for name, read in sources:
1609+
asked += 1
1610+
try:
1611+
found += read()
1612+
except Exception:
1613+
failed += 1
1614+
logger.opt(exception=True).warning(
1615+
'{} discography failed for artist "{}"', name, playlist.name
1616+
)
1617+
if failed == asked:
15881618
raise RuntimeError('No discography source answered')
15891619
return found
15901620

@@ -1629,17 +1659,99 @@ def _deezer_releases(
16291659
return releases
16301660

16311661

1662+
def _musicbrainz_releases(
1663+
playlist: MonitoredPlaylist,
1664+
db: PlaylistMonitorDB,
1665+
settings: dict[str, Any],
1666+
listed: list[dict[str, Any]],
1667+
) -> list[dict[str, Any]]:
1668+
"""The watched artist's MusicBrainz release groups that the other
1669+
catalogues (*listed*) don't have, as release summaries.
1670+
1671+
The MusicBrainz artist is found on the first check - through the
1672+
Deezer artist found just before, the YouTube Music channel, or an
1673+
unambiguous name (:func:`musicbrainz.artist_id_for`) - and kept on
1674+
the watch. Only the artist's own groups are kept, not the ones they
1675+
are a guest on, and only groups whose secondary types are all let in
1676+
by ``artist_watch_musicbrainz_kinds`` (mixtapes and soundtracks by
1677+
default; live albums, compilations, remixes... when asked).
1678+
1679+
A group another catalogue lists under the same name and type is left
1680+
to that catalogue: MusicBrainz fills gaps, it doesn't fetch every
1681+
release a third time. When a "new releases only" watch first finds
1682+
the artist, every group is recorded as already there, as for Deezer.
1683+
"""
1684+
1685+
artist_id = playlist.musicbrainz_id
1686+
linked = False
1687+
if not artist_id:
1688+
artist_id = musicbrainz.artist_id_for(
1689+
playlist.name,
1690+
deezer_id=playlist.deezer_id,
1691+
channel_id=playlist.spotify_id,
1692+
)
1693+
if not artist_id:
1694+
logger.info(
1695+
'No MusicBrainz artist settled for "{}" (none, or several '
1696+
'of that name)',
1697+
playlist.name,
1698+
)
1699+
return []
1700+
linked = True
1701+
kinds = musicbrainz.kinds_from_setting(
1702+
settings.get('artist_watch_musicbrainz_kinds')
1703+
)
1704+
releases = [
1705+
{
1706+
**musicbrainz.release_summary(group),
1707+
'album_id': MUSICBRAINZ_RELEASE + group['id'],
1708+
}
1709+
for group in musicbrainz.release_groups(artist_id)
1710+
if musicbrainz.wanted(
1711+
group, artist_id, musicbrainz.PRIMARY_TYPES, kinds
1712+
)
1713+
]
1714+
if linked:
1715+
db.update_playlist(playlist.id, musicbrainz_id=artist_id)
1716+
playlist.musicbrainz_id = artist_id
1717+
logger.info(
1718+
'Artist "{}" is MusicBrainz artist {} ({} release groups)',
1719+
playlist.name,
1720+
artist_id,
1721+
len(releases),
1722+
)
1723+
if playlist.new_only and not playlist.baseline_pending:
1724+
db.mark_albums_skipped(playlist.id, releases)
1725+
known = {
1726+
(file_name_key(str(a.get('name') or '')), release_type_of(a))
1727+
for a in listed
1728+
}
1729+
return [
1730+
r
1731+
for r in releases
1732+
if (file_name_key(r['name']), release_type_of(r)) not in known
1733+
]
1734+
1735+
16321736
def _release_tracks(
16331737
album: dict[str, Any], artist_name: str
16341738
) -> Optional[list[dict[str, Any]]]:
16351739
"""A release's tracks, from the catalogue that listed it.
16361740
1637-
``None`` for a Deezer release that isn't this artist's: Deezer lists
1638-
the singles an artist is a guest on among their own (see
1639-
:func:`_credited_release`).
1741+
``None`` for a release not to download, recorded as skipped: a
1742+
Deezer release that isn't this artist's (Deezer lists the singles an
1743+
artist is a guest on among their own - see :func:`_credited_release`),
1744+
or a MusicBrainz group without an official edition (a bootleg).
1745+
MusicBrainz groups were already narrowed to the artist's own
1746+
(:func:`musicbrainz.wanted`).
16401747
"""
16411748

16421749
album_id = str(album['album_id'])
1750+
if album_id.startswith(MUSICBRAINZ_RELEASE):
1751+
edition, songs = musicbrainz.release_group_tracks(
1752+
album_id[len(MUSICBRAINZ_RELEASE) :]
1753+
)
1754+
return songs if edition else None
16431755
if not album_id.startswith(DEEZER_RELEASE):
16441756
return providers.album_tracks_from_browse_id(album_id)
16451757
owner, main, songs = deezer.album_credits_from_id(

0 commit comments

Comments
 (0)