{"data":{"space":"permanence","networksConfig":{"symbol":"DOT","decimals":10,"networks":[{"network":"polkadot","ss58Format":0,"assets":[{"symbol":"DOT","decimals":10}]}],"accessibility":"whitelist","whitelist":["1ZSPR3zNg5Po3obkhXTPR95DepNBzBZ3CyomHXGHK9Uvx6w","1xzcLSwo7xBFkJYZiL4EHaqFpuPTkH641E3V43W4cuk1bX6","12His7t3EJ38tjdBbivUzWQeaNCLKfMqtKp1Ed3xHMyCE9N3","12s6UMSSfE2bNxtYrJc6eeuZ7UxQnRpUzaAh1gPQrGNFnE8h","13EDmaUe89xXocPppFmuoAZaCsckaJy3deAyVyiykk1zKQbF","14333MZvbGkcq5CZ8fYHZiFYwHNDaW3uiErDKMb7oqnupWXn","14gMJV95zwxUsFEZDSC8mtBVifS6SypKJkfBKANkMsLZdeVb","14Gn7SEmCgMX7Ukuppnw5TRjA7pao2HFpuJo39frB42tYLEh","15fTH34bbKGMUjF1bLmTqxPYgpg481imThwhWcQfCyktyBzL","167YoKNriVtP4Nxk9F9GRV7HTKu5VnxaRq1pKMANAnmmTY9F","13znFMMjHyM2UvSewvaKMC2bLUcySRMzcM8BAMTzm1G2P5ju"],"strategies":["one-person-one-vote"],"version":"4"},"title":"[MS] DOT #1420 - SQD (fka Subsquid) - Public data indexing Infrastructure for Polkadot and Kusama (Q2 2024)","content":"https://polkadot.subsquare.io/referenda/1420\n\nProponent: Subsquid Lab Official - 13bfKSQXoBn3AMLtZaW6BKv797fqZzsD3PYF6xpJDir3tdoE\n\nBeneficiary: Subsquid Lab Official - 13bfKSQXoBn3AMLtZaW6BKv797fqZzsD3PYF6xpJDir3tdoE\n\nContact Details: Subsquid Labs GmbH 6300 Zug, Switzerland [mf@subsquid.io](mailto:mf@subsquid.io)\n\nShort description: Ongoing development and maintenance costs for public SQD Archives\n\nArchive raw data: [Archive Infrastructure Metrics](https://docs.google.com/spreadsheets/d/1qTLup1JU9XPsklciZDeHVbd-EtP1AvdP4zzI5rm4bSU/edit?gid=466566330#gid=466566330) - April, May, June\n\nRequested DOT: 58 852.91 DOT = $264,838.09\n\nPrevious proposals: [https://polkadot.polkassembly.io/referenda/1193](1193)\n\n**Motivation**\n\n**A) Archives**\n\nArchives are an important piece of Polkadot and Kusama data infrastructure provided by SQD (formerly Subsquid). They provide access to on-chain data for public chains in the Polkadot ecosystem. They are being used as a data source for [Squids](https://www.sqd.ai/cloud) and for efficient data exploration and ad-hoc queries.\n\nThe Archives, as performant data sources of historical on-chain data, are critical to the operation of backend APIs run by major projects in the Polkadot ecosystem (the list is not exhaustive):\n\n*   Tanssi Network\n*   Talisman Wallet\n*   Polimec\n*   Polkassembly\n*   Apillon\n*   Hydration\n*   Giant Squid API (maintained by LimeChain, see details below)\n*   SubWallet\n*   FiDi\n*   KodaDot\n*   RMRK\n*   ChainSafe (in particular the Multix multi-sig, as well as other projects)\n*   Phala Network\n*   Stellaswap\n*   Polkascan\n\nSince the first quarter of 2024, SQD has transitioned to a paid model for supporting parachains. However, public chains continue to be available for free use. Throughout the second quarter of 2024, we provided a free public data indexing service for Substrate blockchains, allowing interested parties to access indexed data on Substrate Events, Extrinsics, Storage Items, and EVM logs. Below is a list of the public chains we currently support at no cost:\n\n*   polkadot\n*   kusama\n*   asset-hub-kusama\n*   asset-hub-polkadot\n*   bridge-hub-polkadot\n*   collectives-polkadot\n*   rococo\n*   westend-collectives\n\nAs of Q1 2024, SQD has sunsetted support for Giant Squid, with LimeChain taking over its further development, detailed in the proposal at [https://polkadot.polkassembly.io/treasury/758](../treasury/758)\n\n**B) Contribution**\n\nDuring Q2’2024, we made significant contributions to support ink! v5 and address actual use cases that were highly demanded by the community:\n\n*   We provided support for manual finality confirmation in the **subsquid/substrate-processor** for cases where a chain does not expose finalization details.\n\n**Achievements**\n\n**Increase in Archival Data Demand and Infrastructure Scaling.**\n\nDuring Q2, archival data demand increased approximately ~28,1 times, from 110,6 GiB to 3.095,17 GiB. We addressed the increasing demand for archival data by enhancing our infrastructure's throughput and redundancy through deploying archives to OVHCloud in addition to GCE, while also empowering our maintenance and engineering teams to support the expanded infrastructure effectively.\n\n**Mainnet Launch and Decentralization of the SQD Network Enhancing the Polkadot Ecosystem:**\n\nIn Q2’24, we successfully launched the SQD mainnet, a pivotal advancement that significantly strengthens the Polkadot ecosystem's decentralization and resilience. By the end of the quarter, the SQD network comprised 726 Worker Nodes online, securely storing 529 TB of data and serving ~ 9 TB of data a day. This decentralized infrastructure enhances data processing and storage capabilities, ensuring higher security, fault tolerance, and scalability for the Polkadot network.\n\n**Advancements in Squid SDK Empowering Polkadot Developers:**\n\nEnabled manual configuration of finality confirmation for subsquid/substrate-processor in cases where a parachain does not expose finalization information.\n\n**Cloud API Upgrades**\n\nWe upgraded our Cloud API to offer enhanced capabilities that support the growth of the Polkadot ecosystem:\n\n*   Multiple API Tokens Support: Allows for more granular access control and better management of development workflows.\n*   Hasura Integration: Enables more efficient and powerful data querying and manipulation, facilitating the creation of responsive and data-driven applications.\n    \n*   Chain ID and SS58 Prefix support: Enhanced network identification and management for seamless interoperability across Polkadot parachains.\n    \n\n**Statistics and data**\n\n**![](https://i.ibb.co/fVsWg5nz/Chart-1.png)**\n\nOver the course of Q2 2024:\n\n*   Served a total of **3,505,904,767,634 (~3.19 TiB)** of data from the substrate based archives in Q2 2024\n*   Served 301 069 archive requests from substrate based archives in Q2 2024\n*   The highest daily total data transfer recorded for Substrate was **161.646 GiB** within a single 24-hour period.\n\n**Costing**\n\nThis proposal encompasses the combined costs for running Archives for Polkadot, Kusama, and other public chains, along with the development contributions, amounting to a total of **58 852.91 DOT**\n\n[Raw data for Archives can be seen here.](https://docs.google.com/spreadsheets/d/1qTLup1JU9XPsklciZDeHVbd-EtP1AvdP4zzI5rm4bSU/edit?gid=1549354589#gid=1549354589)\n\nThe proposal is submitted towards the Polkadot treasury. Here is a summary of the running costs for Archives, in a tabular format:\n\n![](https://i.ibb.co/JNZk2p9/Table-1.png)\n\nContribution costs:\n\n![](https://i.ibb.co/SwBQ74bG/Table-2.png)","contentType":"markdown","choiceType":"single","choices":["Aye","Nay","Abstain"],"startDate":1738886400000,"endDate":1741478400000,"snapshotHeights":{"polkadot":24634000},"realProposer":null,"proposerNetwork":"polkadot","version":"5","timestamp":1738966340},"address":"13znFMMjHyM2UvSewvaKMC2bLUcySRMzcM8BAMTzm1G2P5ju","signature":"0x9eb2b545c78fc0acc6479448f0ceb2c2c86bbe4b9cea2e091a0edd7317271f4da04076d1d876dad7b5b731dd61b2bbddab58f31e35373d7c99588967ce9f7687","version":"2"}