CHANGELOG.md

Change Log

All notable changes to this project will be documented in this file. See Conventional Commits for commit guidelines.

5.0.0 (2019-03-17)

Features

• reduce rules on parallel (#157) (bce6eb5)

BREAKING CHANGES

• 'meta' can't be passed as shared resource

• build: deprecated lang-detector

4.10.3 (2019-03-17)

Note: Version bump only for package metascraper

4.10.2 (2019-03-16)

Bug Fixes

• add author size limitation (#154) (c1df0d3)

4.10.1 (2019-01-25)

Note: Version bump only for package metascraper

4.10.0 (2019-01-13)

Features

• remove unnecessary dependency (8dbb1ac)

4.8.5 (2018-12-25)

Note: Version bump only for package metascraper

4.8.4 (2018-12-16)

Bug Fixes

• avoid use generic publisher (#141) (c13f833)

4.8.3 (2018-12-15)

Note: Version bump only for package metascraper

4.8.2 (2018-12-12)

Bug Fixes

• ensure buffer has enough space (#136) (8bd7be3), closes #issuecomment-253184229

4.8.1 (2018-12-07)

Bug Fixes

• use agent for Twitter request (#135) (ed3e1a2)

4.8.0 (2018-11-19)

Features

• fulfilled twitter promise (#133) (5e89fe5)

(2018-11-16)

4.7.0 (2018-11-16)

• Update dependencies (#132) (a256f0c), closes #132
• v4.7.0 (a7d53da)
• fix: reset token every 10min (cf8cd92)

4.6.1 (2018-10-28)

• v4.6.1 (dc664c3)
• fix: use onError for capture error (#130) (dcf9e23), closes #130

4.6.0 (2018-10-24)

• v4.6.0 (0ef7ad5)
• feat: get language from twitter payload (#129) (80d5ddf), closes #129

4.5.6 (2018-10-23)

• v4.5.6 (e90bb13)
• fix: add proxy support at youtube-dl (c2c0fea)
• fix: iterate over new formats detected (14db749)

4.5.5 (2018-10-21)

• v4.5.5 (679424c)
• build: setup conventional workflow (b3e0a9e)

4.5.5-alpha.0 (2018-10-21)

• Update Changelog (5107f87)
• v4.5.5-alpha.0 (a9acb05)
• build: add automate-release workflow (a6b0493)
• build: update dependencies (8597ea7)

4.5.4 (2018-10-17)

• Prefer mp4 video if it is available (#127) (2d92376), closes #127
• Update Changelog (b8d5445)
• v4.5.4 (5fc2f5c)

4.5.3 (2018-10-11)

• Get original tweet id (ac7e5e9)
• Update Changelog (f8167ed)
• v4.5.3 (e01b4b2)

4.5.2 (2018-10-05)

• Add tunnel support (#124) (084e9e6), closes #124
• Update Changelog (1319563)
• v4.5.2 (29bcc2c)

4.5.1 (2018-10-04)

• Revert "Avoid use Twitter API (#122)" (66ca721), closes #122
• Update Changelog (6481681)
• v4.5.1 (af004a4)

4.5.0 (2018-09-30)

• Refactor tests (b9d4c5f)
• Update Changelog (024d5f3)
• Use prerender for extracting HTML (4b6c034)
• v4.5.0 (3c6970b)

4.4.4 (2018-09-29)

• Avoid use Twitter API (#122) (71c7071), closes #122 /github.com/loklak/loklak_server/blob/7c513ae7949718399a7cf06bb19e80da362ef396/src/org/loklak/harvester/TwitterScraper.java#L486-L506
• Update Changelog (3b581b6)
• v4.4.4 (fd0c0a9)

4.4.3 (2018-09-26)

• Disable retry (5dc9c54)
• Update Changelog (65de378)
• Update rate limit window (82b1218)
• v4.4.3 (8ecb4f5)

4.4.2 (2018-09-21)

• Update Changelog (82e44fa)
• Update limit rates (d7d0052)
• v4.4.2 (f6df434)

4.4.1 (2018-09-20)

• Refresh token after expiration (c99b57b)
• v4.4.1 (817663a)

4.4.0 (2018-09-19)

• Move memoize token into an external dependency (#120) (96b3bcc), closes #120
• Update Changelog (32fc3dd)
• Update scripts (c64a111)
• v4.4.0 (933b562)

4.3.6 (2018-09-19)

• Add memoize for successive API calls (#119) (d615b36), closes #119
• Update Changelog (d4cf08b)
• Update contributors (ad4468c)
• Update scripts (760657b)
• v4.3.6 (4bc9178)

4.3.5 (2018-09-18)

• Throw qualified errors (#118) (efe7710), closes #118
• Update Changelog (6639bdb)
• Update dependencies (795aeb5)
• v4.3.5 (049ef06)

4.3.4 (2018-09-15)

• Fix lint (395c98c)
• Only get video from tweets urls from Twitter (1313fd7)
• Update Changelog (0f84e19)
• v4.3.4 (82a5abf)

4.3.3 (2018-09-09)

• Handle unsupported urls gracefully (93d37a3)
• Update Changelog (86b443f)
• Update scripts (3fa3fed)
• v4.3.3 (b7d0526)

4.3.2 (2018-09-07)

• Fix publisher (#117) (501940e), closes #117
• More authors to root project (57638a7), closes #116
• Update Changelog (59f0216)
• v4.3.2 (0ece288)

4.3.1 (2018-09-05)

• Exit early if no twitter videos detected (#115) (a69979f), closes #115
• Update Changelog (25cec6f)
• v4.3.1 (24c388e)

4.3.0 (2018-09-04)

• Add metascraper lang detector package (#114) (f379837), closes #114
• Add metascraper-audio (#113) (face7e1), closes #113
• Fix generate rule interface correctly (b41007e)
• Update Changelog (531093b)
• Update rules bundles (5131af0)
• v4.3.0 (1ab950c)

4.2.0 (2018-09-01)

• Add image url helper (b7e2987)
• Update Changelog (c0b9ef5)
• v4.2.0 (e035974)

4.1.0 (2018-09-01)

• Add greyscale option to Clearbit logo API (#110) (137b2f4), closes #110
• Add audio support (7d5ad34)
• Add audio support (9f09429)
• Fix tests (00584d3)
• Fix tests (6fd95b4)
• Lock standard version (82c6108)
• Lock standard version (b61e68d)
• Move helpers method into helpers package (b845f45)
• Move helpers method into helpers package (a807b70)
• Remove unnecesary test (fe3efa6)
• Remove unnecesary test (153f5d1)
• Rename video-provider → media-provider (2838e12)
• Rename video-provider → media-provider (77080db)
• Return a collection of urls (7f54518)
• Return a collection of urls (9ffcd67)
• Strict url validation (edfb8a0)
• Strict url validation (b63564f)
• Tweaks (10faffd)
• Tweaks (99a52ef)
• Update Changelog (4ddd209)
• Update dependencies (#112) (4ecf92d), closes #112
• v4.1.0 (6b08dfa)
• Video is a collection (97fcfae)
• Video is a collection (19405c2)
• video is string (1e3227b)
• video is string (41e5806)

4.0.3 (2018-08-28)

• Add condition to stop (e49a85c)
• Update Changelog (8766f13)
• v4.0.3 (a2027c9)

4.0.2 (2018-08-27)

• Update Changelog (321a2ab)
• Update peer dependency to metascraper 4 (a837299)
• v4.0.2 (a287d23)

4.0.1 (2018-08-26)

• Add tests (58c26f0)
• Improve publisher detection based on title (#109) (840ac8b), closes #109
• Update Changelog (2a3dc73)
• Update ISSUE_TEMPLATE.md (f35ca69)
• v4.0.1 (94c0281)

4.0.0 (2018-08-24)

• Remove autoload (9d3745b)
• Remove autoload (#106) (60b0adb), closes #106
• Remove lazy intializer (d63fcc8)
• Update Changelog (76d5329)
• Update dependencies (19aca42)
• Update Docs (7fd6b15)
• v4.0.0 (7901fb6)

3.13.2 (2018-08-18)

• Oh twitter (23f641d)
• Update Changelog (9c5463b)
• v3.13.2 (f4a0b21)

3.13.1 (2018-08-18)

• At least video need to be mp4 format (f6fd92a)
• Update Changelog (b012514)
• v3.13.1 (8928e95)

3.13.0 (2018-08-17)

• Better twitter video extraction (#104) (eb8778c), closes #104
• Unify lint-staged (4e2ae4d)
• Update Changelog (840ff6f)
• v3.13.0 (3327bab)

3.12.1 (2018-08-17)

• Update Changelog (718b0c8)
• Update dependencies (51c7c52)
• Use title instead of to-title-case (#103) (5d25245), closes #103
• v3.12.1 (9baa717)

3.12.0 (2018-08-15)

• Use twdown for twitter videos (0a74da4)
• v3.12.0 (355b404)

3.11.12 (2018-08-13)

• Fix typo (35b0ca6)
• Remove residual data (b66a28c)
• v3.11.12 (815ba2b)

3.11.11 (2018-08-13)

• Improve video detection (3a326ab)
• v3.11.11 (0d5ee77)

3.11.10 (2018-08-11)

• v3.11.10 (63b10e1)

3.11.9 (2018-08-11)

• Add file extension (3236005)
• Add providers script (68ece2b)
• Cache non supported urls as well (837c76c)
• Detect extractors without HTTP request (1a64a14)
• Ensure normalizeHttp is false by default (#101) (339d473), closes #101
• Fix linter (569a6bd)
• Remove postinstall script (f8734db), closes /github.com/rg3/youtube-dl/issues/17208#issuecomment-412262556
• Tweaks (c3bd078)
• Update Changelog (3c06b83)
• Update dependencies (234bd91)
• Update index.html (5587ade)
• Update index.html (3f16409)
• v3.11.9 (c8e22e2)

3.11.8 (2018-07-10)

• Update Changelog (1b7685f)
• Update dependencies (#100) (630baec), closes #100
• v3.11.8 (841f367)

3.11.7 (2018-07-06)

• Update Changelog (5c2e904)
• Update dependencies (#98) (44e2f03), closes #98
• v3.11.7 (4197edc)

3.11.6 (2018-06-02)

• Add abbility to use dom pseudo selectors (8677a0d)
• Update Changelog (5f3d1da)
• Update dependencies (4a74f0b)
• v3.11.6 (43ce89f)

3.11.5 (2018-05-29)

• Don't remove trailing slash (#91) (79f3330), closes #91
• Update Changelog (d016735)
• v3.11.5 (bf8fd25)

3.11.4 (2018-05-27)

• Expose validators for core rules (#89) (feb2a4b), closes #89
• Update Changelog (aba7526)
• Update dependency (#90) (a7b1457), closes #90
• v3.11.4 (ca4b7d8)

3.11.3 (2018-05-17)

• Revert youtube-dl-installer (33989b9)
• Update Changelog (7e3e1fb)
• v3.11.3 (49af94c)

3.11.2 (2018-05-17)

• Update Changelog (1e7bf0f)
• Update deps (56ec5e6)
• v3.11.2 (ffe6811)

3.11.1 (2018-05-17)

• Update Changelog (d8a629f)
• Use youtube-dl binary directly (#88) (2409771), closes #88
• v3.11.1 (6b80d23)

3.11.0 (2018-05-15)

• Avoid mutate state (9a56e7d)
• Ensure to auto load rules just once (cb18d36)
• Pass rules as argument (#87) (7c5e236), closes #87
• Pass rules support (766bff3)
• Update Changelog (0bcc5ce)
• v3.11.0 (1558856)

3.10.9 (2018-05-12)

• Add itemprop image (8e81660)
• Remove sailthru rules (a36e240)
• Remove unnecessary author rules (296fda7)
• Simplify publisher rules (47ec0e5)
• Update Changelog (9d49f87)
• v3.10.9 (810a1e5)

3.10.8 (2018-05-10)

• Update Changelog (2dc8f84)
• Update dependencies (#85) (672facc), closes #85
• Use microlinkhq/normalize-url (#83) (817fce6), closes #83
• v3.10.8 (8ce6ecc)

3.10.7 (2018-04-29)

• Be possible pass opts (d675cc9)
• Update Changelog (3a1e5da)
• Update meta (7a16eba)
• v3.10.7 (47eac4c)

3.10.6 (2018-04-12)

• Avoid extract video for non supported domains (19792e6)
• Update Changelog (ab464ac)
• v3.10.6 (4a80b36)

3.10.5 (2018-04-11)

• Ensure to resolve a valid logo url (e004596)
• Update Changelog (5727dde)
• Update README.md (7ad710d)
• v3.10.5 (71ff1f9)

3.10.4 (2018-04-10)

• Extract more video fields (69f40c5)
• Update Changelog (a67e147)
• v3.10.4 (754fc1c)

3.10.3 (2018-04-08)

• Ensure to return a http protocol url (f1595f1)
• Update Changelog (ded6cc4)
• v3.10.3 (f5cc2b1)

3.10.2 (2018-04-08)

• Improve mp4 detection (4f7f26b)
• Update Changelog (0f66a1b)
• Update README.md (3fedeaa)
• v3.10.2 (c4aa6b6)

3.10.1 (2018-04-08)

• Change implementation (b14e3a0)
• Update Changelog (bd7dc2d)
• v3.10.1 (fe3320c)

3.10.0 (2018-04-07)

• Add metascraper-social-video (#79) (51959cf), closes #79
• Ensure to concat rules as array (dff7a6d)
• Refactor and expose internal method (ab0a1ac)
• Update Changelog (1a88468)
• v3.10.0 (f8e5ed5)

3.9.3 (2018-03-29)

• Add copy clipboard button (8ad33bb)
• Avoid sort url query params (4fb8095)
• Remove unnecessary code (bd6e970)
• Update Changelog (cb89351)
• Update index.html (e392c60)
• v3.9.3 (b3e562d)

3.9.2 (2018-02-22)

• Call getValue() correctly (afe0915)
• Fix badge (fd6d2b0)
• Fix lint (c1eb1ef)
• Only return first title element value (da30009)
• Update Changelog (672ebef)
• Update deps (327eea8)
• v3.9.2 (a9c94bc)

3.9.1 (2018-02-17)

• Fix resolve absolute video URL (97a8026)
• Update Changelog (96b5d84)
• v3.9.1 (44a2d29)

3.9.0 (2018-02-13)

• Add missing test script (c594242)
• Get the highest image size (63ce77d)
• Omit date from tests (7511145)
• Remove unnecessary dependencies (21f2a1f)
• Support async logic into rules set (06fa950)
• Update Changelog (759b538)
• Update test mock (a4c6c54)
• v3.9.0 (41aa1bc)

3.8.0 (2018-02-11)

• Add metascraper video as default rules set (d14e1e1)
• Avoid mutation (f1d380c)
• Extract .getSuffix method (34f8565)
• Refactor URL helpers (4869eaf)
• Update Changelog (3161f79)
• v3.8.0 (dfb6d4a)

3.7.3 (2018-02-08)

• Add metascraper-lang tests (c5f73bb)
• Ensure get lang only for amazon urls (24d5a19)
• Remove unnecessary file (c2e90ce)
• Update Changelog (66a7424)
• Update timestamp (979c538)
• v3.7.3 (fe11521)

3.7.2 (2018-02-04)

• Fix get video extension from url (b98d2db)
• Update Changelog (fa47e32)
• v3.7.2 (498a0d5)

3.7.1 (2018-02-04)

• Ensure video is an absolute video url (b193dcd)
• Update Changelog (b3cac2a)
• v3.7.1 (c839265)

3.7.0 (2018-02-03)

• Expose load and autoload methods (327fae4)
• Update Changelog (3951637)
• Update docs (b75dc59)
• Update tests (274cb27)
• v3.7.0 (4c62dce)

3.6.4 (2018-02-01)

• Update Changelog (caa6855)
• Update get-video-id dependency (fc0bf94)
• v3.6.4 (eed68e0)

3.6.3 (2018-01-30)

• Update Changelog (09efd46)
• Use src poster value as image (d2e16c3)
• v3.6.3 (53e1461)

3.6.2 (2018-01-28)

• Avoid repeat parsing steps (566f9d1)
• Update Changelog (2d1cb53)
• Update README.md (56409d1)
• v3.6.2 (2653c7d)
• docs(readme): add Greenkeeper badge (777f511)

3.6.1 (2018-01-23)

• Add og:video test (1ced9f9)
• Update Changelog (011fae2)
• v3.6.1 (d80789e)

3.6.0 (2018-01-23)

• Add metascraper-video first approach (79e98c9)
• Check upgrade package dependencies before publish (5c70cdb)
• Ensure output is an object (dbfac33)
• Extract removeBy into titleize helper (627abfc)
• Normalize meta property tag (df96522)
• Remove date from snapshot (818fa90)
• Update Changelog (b5749c3)
• Update deps (281088f)
• v3.6.0 (c28a942)

3.5.0 (2018-01-23)

• Add METASCRAPER_CWD tests and documentation (73be24f), closes #52
• Add metascraper-youtube (a043f93)
• Add missing dep (31122be)
• Add YouTube rules package (4749216)
• Adjust style (272c2a3)
• Avoid get first directly, use .getValue helper (12dd21e)
• Avoid non deterministic date (3fc6130)
• Docs tweaks (9dac9f6)
• Move soundcloud test (ab03d97)
• Refactor (a7dea20)
• Remove browser mention (6cb0d73)
• Remove unnecessary deps (ddf4384)
• Update (d246295)
• Update README.md (c8609cd)
• Update scripts (8b158e0)
• Update testing setup (ba6253c)
• Update testing setup (d75c74f)
• Update toc (853f8d3)
• v3.5.0 (88a1e6f)

3.4.3 (2017-12-31)

• Improve author & description accuracy (7e14b48)
• Refactor (447280c)
• v3.4.3 (9ca8926)

3.4.2 (2017-12-30)

• Add metascraper-lang as part of defaults rule sets (5ec9472)
• Add get language from amazon urls (f6de069)
• Update Changelog (38ce74c)
• v3.4.2 (df4f572)

3.4.1 (2017-12-29)

• Ensure url is string (4402622)
• Update changelog (ad5efe8)
• Update README.md (f875152)
• v3.4.1 (d13dfb8)

3.4.0 (2017-12-29)

• Add a.co amazon urls (eb75363)
• Add language property (656bb97), closes #45
• Add missing readme (76201b0)
• Avoid non deterministic tests (d63d5aa)
• Ensure favicon path is absolute (266b404)
• Refactor (3a412be)
• Remove coveralls as dev dependency (2aab9e7)
• Tweaks (2ea632f)
• Update Changelog (f2bee3d)
• v3.4.0 (3f73396)

3.3.0 (2017-12-20)

• Add indidivual package doc (253ca09)
• Add more tests (e7143f6)
• Ensure to resolve absolute path (70b03bb)
• Refactor url helpers (9da1325)
• Regenerate changelog (9c314e5)
• Resolve closest node_modules folder (13a5498)
• Update package urls (5ed88e5)
• Use ~ version (895a0bd)
• v3.3.0 (4eabdec)

3.2.0 (2017-12-19)

• Add build command (8a23e81)
• Add clearbit-logo tests (3e87ec4)
• Add metascraper amazon (daa7c11)
• Add missing dep (03c970a)
• Add missing dependency (eae7d9a)
• Add npmrc (6e72266)
• Concat specific rules first (9ecc857)
• Customize selection color (a336d9a)
• Extract fixed favicon value into separate package (7a5619f)
• Extract titleize into helpers (ae3a515)
• Fix badge style (c17b46d)
• Fix test (1757ff1)
• Improve docs (9b00438)
• Remove unnecessary badges (7ab34eb)
• Resolve package properly (e61d33f)
• Setup better DX (2c47bfa)
• Titleize description without capitalization (048d398)
• Tweak doc style (ffd57f1)
• Unify changelog (8f0c23e)
• Update changelog (bc9d757)
• Update lint staged hooks (9ed1812)
• Update plugins directory (d989c14)
• Update test (2f327d9)
• Update tests snapshot (93bb754)
• v3.2.0 (7cead99)

3.1.0 (2017-12-17)

• Add metasraper-soundcloud (9814253)
• Add publish script (b96eb09)
• Add soundcloud example (1b3aa93)
• Fix linter (d1ce9ea)
• Improve plugins interface (563c583)
• Remove unnecessary test (7b7d249)
• Remove version from pkg (9acc123)
• Update docs (03d29e0)
• v3.1.0 (578ef2a)

3.0.0 (2017-12-15)

• Add CNAME (a5a1b59)
• Add custom travis install (5768cbf)
• Add design assets (2f1a19b)
• Add documentation site (2bb6933)
• Add lerna debug into gitignore (bbf5ea9)
• Add metascraper clearbit logo package (4ab612b)
• Add plugins documentation (e37a4ee)
• Add plugins documentation (6af5082)
• Add possibility to load rules from config file (71b83b3)
• Add support for plugins (f065096)
• Be possible specify config cwd (8b34f50)
• Migrate into monorepo (1bf8ee6)
• Optimize assets (3ac8a88)
• Refactor to be more readable (c85c9b8)
• Setup travis (b87bd58)
• Support load config (329452b)
• Update TOC (2edb17c)
• v3.0.0 (6df48b3)

2.0.0 (2017-12-11)

• Add anandtech case (da392f6)
• Add arstechnica case (0c27159)
• Add astier case (fdebba6)
• Add atlasobscura case (d2a8f22)
• Add bbc case (30f2cff)
• Add bench (57ddb65)
• Add env for testing (366eda1)
• Add favicon support (f413a31)
• Add github case (f9cb980)
• Add GitLab conf (fe3df07)
• Add googleblog case (bb76141)
• Add html sanitization (818a19b)
• Add instagram (2522915)
• Add issue template (8967e15)
• Add linkedin company profile (a103ea4)
• Add logo support (1ad56ae)
• Add meta and new docs (96a5601)
• Add missing rules (26d299b)
• Add missing timestamp selector (aebe5ef)
• add nytimes case (194a71a)
• Add pikabu test (4a27d08)
• Add prettier (d93dd39)
• Add req-all dep (9da03a5)
• Add smart quotes (61a4574)
• Add the guardian case (1c9068e)
• Add ToC (d6d7a6c)
• Add twitter case (cd42526)
• Add url and html as required options (a285914)
• Add vimeo test (3e9eff2)
• Add Wikipedia case (7c37449)
• Add wired case (fefd105)
• Add youtube case (486a964)
• Automatize snapshots (d7a640c)
• clean a tags (aea5787)
• Delete migration script (108e79c)
• Extract url helpers into utils (efd5694)
• Fix author (65de0ce)
• Fix output date (6c572f9)
• Fix output dates (5bf8d93)
• Fix test (3b04f96)
• Fix test (f58cccd)
• Fix test (6149cde)
• Fix test (7039590)
• Fix typo (e2138bd)
• Improve author connector (a8ae1aa)
• Improve docs (9676b93)
• Improve get url value (5020eb5)
• Improve logo detection (3040379)
• Improve logo detection (64d439a)
• Merge favicon into logo (d2d66fb)
• Merge props correctly (32b9610)
• Migrate from gitlab to travis (6e6f70a)
• Pretty (1892ea5)
• Publisher refactor (76bb282)
• Refactor (e9df18f)
• Refactor (cef0f1e)
• Refactor (20eee39)
• Refactor author (760b503)
• Refactor date (ae355f3)
• Refactor description (2d1ae48)
• Refactor image (6eb017f)
• Refactor testing (e120498)
• Refactor testing scaffold (31e5d02)
• Refactor url (2337f70)
• Release 2.0.0 (3a81306)
• Remove callback interface (7a1ef68)
• Remove debug (fab5432)
• Remove duplicate field (e3566ff)
• Remove node6 build (95b9f67)
• Remove non tested lines (1fcaa50)
• Remove npmignore (a24aaa9)
• Remove unnecessary code (c19f8ff)
• Remove whitespaces from title (b7ea017)
• Rename branding (a811e7a)
• Rename the-washington-post → washington-post (9967f1e)
• Setup depdendencies (40206ca)
• Support resolve URL (dbdba26)
• Support URL (9bb261d)
• Tweak (28cf2b9)
• Tweaks (1d7ee60)
• Update (d509c8a)
• Update dependencies (dbb54e6)
• Update deps (828d908)
• Update image url (33d99bf)
• Update license (84c0588)
• Update medium case (84dbfe5)
• Update nullable URL's (cc1d147)
• Update README.md (e9b8858)
• Use mocha for testing (07967ca)
• WIP (e3120a2)
• docs(readme): add Greenkeeper badge (5e92ea7)