mirror of
https://github.com/discordjs/discord.js.git
synced 2026-03-12 01:23:31 +01:00
feat: Internal sharding (#2902)
* internal sharding * ready event * the square deal * the new deal * the second new deal * add actual documentation * the new freedom * the great society * federal intervention * some of requested changes * i ran out of things to call these * destroy this * fix: Client#uptime went missing * fix(Client): destroy the client on login failure This may happen duo invalid sharding config / invalid token / user requested destroy * fix(Client): reject login promise when the client is destroyed before ready * fix(WebSocketManager): remove redundancy in destroy method (#2491) * typo(ErrorMessages): duo -> duo to * typo(ErrorMessages): duo -> due * fix: docs and options * docs(WebSocketManager): WebSockethard -> WebSocketShard (#2502) * fix(ClientUser): lazily load to account for extended user structure (#2501) * docs(WebSocketShard): document class to make it visible in documentation (#2504) * fix: WebSocketShard#reconnect * fix: presenceUpdate & userUpdate * presenceUpdate wasn't really being handled at all * userUpdate handled incorrectly because as of v7 in the Discord API, it comes inside presenceUpdate * re-add raw event * member is now part of message create payload * feat: Add functionality to support multiple servers with different shards (#2395) * Added functionallity to spawn multiple sharding managers due to adding start and end shards * Small fixes and limiting shard amount to max recommended * Forgot a check in spawn() * Fixed indentation * Removed optiosn object documentation for totalShards * More fixes and a check that the startShard + amount doesnt go over the recommended shard amount * fix getting max recommended * Removed async from constructor (my fault) * Changed start and end shard to a shardList or "auto" + fixed some brainfarts with isNaN * Changed the loop and totalShard count calculation * shards are actually 0 based * Fixed a problem with the gateway and handled some range errors and type errors * Changed Number.isNan to isNaN and changed a few Integer checks to use Number.isInteger * Added check if shardList contains smth greater than totalShards; made spawn use totalShards again; shardList will be ignored and rebuild if totalShards is 'auto'; fixed docs * ShardingManager#spawn now uses a for..of loop; fixed the if statement inside the new for..of loop to still work as intended; made the totalShards be set to a new amount if smth manual is put into ShardingManager#spawn just like before; Fixed some spelling * internal sharding * ready event * the square deal * the new deal * the second new deal * add actual documentation * the new freedom * the great society * federal intervention * some of requested changes * i ran out of things to call these * destroy this * fix: Client#uptime went missing * fix(Client): destroy the client on login failure This may happen duo invalid sharding config / invalid token / user requested destroy * fix(Client): reject login promise when the client is destroyed before ready * fix(WebSocketManager): remove redundancy in destroy method (#2491) * typo(ErrorMessages): duo -> duo to * typo(ErrorMessages): duo -> due * fix: docs and options * docs(WebSocketManager): WebSockethard -> WebSocketShard (#2502) * fix(ClientUser): lazily load to account for extended user structure (#2501) * docs(WebSocketShard): document class to make it visible in documentation (#2504) * fix: WebSocketShard#reconnect * fix: presenceUpdate & userUpdate * presenceUpdate wasn't really being handled at all * userUpdate handled incorrectly because as of v7 in the Discord API, it comes inside presenceUpdate * Internal Sharding adaptation Adapted to internal sharding Fixed a bug where non ready invalidated sessions wouldnt respawn * Fixed shardCount not retrieving * Fixing style removed unnecessary parenthesis * Fixing and rebasing lets hope i didnt dun hecklered it * Fixing my own retardation * Thanks git rebase * fix: assigning member in message create payload * fix: resumes * fix: IS wont give up reconnecting now * docs: add missing docs mostly * fix: found lost methods * fix: WebSocketManager#broadcast check if shard exists * fix: ShardClientUtil#id returning undefined * feat: handle new session rate limits (#2796) * feat: handle new session rate limits * i have no idea what i was doing last night * fix if statement weirdness * fix: re-add presence parsing from ClientOptions (#2893) * resolve conflicts * typings: missing typings * re-add missing linter rule * fix: replacing ClientUser wrongly * address unecessary performance waste * docs: missing disconnect event * fix(typings): Fix 2 issues with typings (#2909) * (Typings) Update typings to reflect current ClientOptions * fix(Typings) fixes a bug with Websockets and DOM Types * fix travis * feat: allow setting presence per shard * add WebSocketManager#shardX events * adjust typings, docs and performance issues * readjust shard events, now provide shardId parameter instead * fix: ready event should check shardCount, not actualShardCount * fix: re-add replayed parameter of Client#resume * fix(Sharding): fixes several things in Internal Sharding (#2914) * fix(Sharding) fixes several things in Internal Sharding * add default value for shards property * better implement checking for shards array * fix travis & some casing * split shard count into 2 words * update to latest Internal Sharding, fix requested changes * make sure totalShardCount is a number * fix comment * fix small typo * dynamically set totalShardCount if either shards or shardCount is provided * consistency: rename shardID to shardId * remove Client#shardIds * fix: typo in GuildIntegrationsUpdate handler * fix: incorrect packet data being passed in some events (#2919) * fix: edgecase of ShardingManager and totalShardCount (#2918) * fix: Client#userUpdate being passed wrong parameter and fix a potential edgecase of returning null in ClientUser#edit from this event * fix consistency and typings issues * consistency: shardId instances renamed to shardID * typings: fix typings regarding WebSocket * style(.eslintrc): remove additional whitespace * fix(Client): remove ondisconnect handler on timeout * docs(BaseClient): fix typo of Immediate * nitpick: typings, private fields and methods * typo: improve grammar a bit * fix: error assigning client in WebSocketManager * typo: actually spell milliseconds properly
This commit is contained in:
Isabella
@@ -1,88 +1,281 @@
|
||||
const EventEmitter = require('events');
|
||||
const { Events, Status } = require('../../util/Constants');
|
||||
const WebSocketConnection = require('./WebSocketConnection');
|
||||
const WebSocketShard = require('./WebSocketShard');
|
||||
const { Events, Status, WSEvents } = require('../../util/Constants');
|
||||
const PacketHandlers = require('./handlers');
|
||||
|
||||
const BeforeReadyWhitelist = [
|
||||
WSEvents.READY,
|
||||
WSEvents.RESUMED,
|
||||
WSEvents.GUILD_CREATE,
|
||||
WSEvents.GUILD_DELETE,
|
||||
WSEvents.GUILD_MEMBERS_CHUNK,
|
||||
WSEvents.GUILD_MEMBER_ADD,
|
||||
WSEvents.GUILD_MEMBER_REMOVE,
|
||||
];
|
||||
|
||||
/**
|
||||
* WebSocket Manager of the client.
|
||||
* @private
|
||||
*/
|
||||
class WebSocketManager extends EventEmitter {
|
||||
class WebSocketManager {
|
||||
constructor(client) {
|
||||
super();
|
||||
/**
|
||||
* The client that instantiated this WebSocketManager
|
||||
* @type {Client}
|
||||
* @readonly
|
||||
*/
|
||||
this.client = client;
|
||||
Object.defineProperty(this, 'client', { value: client });
|
||||
|
||||
/**
|
||||
* The WebSocket connection of this manager
|
||||
* @type {?WebSocketConnection}
|
||||
* The gateway this WebSocketManager uses.
|
||||
* @type {?string}
|
||||
*/
|
||||
this.connection = null;
|
||||
this.gateway = undefined;
|
||||
|
||||
/**
|
||||
* An array of shards spawned by this WebSocketManager.
|
||||
* @type {WebSocketShard[]}
|
||||
*/
|
||||
this.shards = [];
|
||||
|
||||
/**
|
||||
* An array of queued shards to be spawned by this WebSocketManager.
|
||||
* @type {Array<WebSocketShard|number|string>}
|
||||
* @private
|
||||
*/
|
||||
this.spawnQueue = [];
|
||||
|
||||
/**
|
||||
* Whether or not this WebSocketManager is currently spawning shards.
|
||||
* @type {boolean}
|
||||
* @private
|
||||
*/
|
||||
this.spawning = false;
|
||||
|
||||
/**
|
||||
* An array of queued events before this WebSocketManager became ready.
|
||||
* @type {object[]}
|
||||
* @private
|
||||
*/
|
||||
this.packetQueue = [];
|
||||
|
||||
/**
|
||||
* The current status of this WebSocketManager.
|
||||
* @type {number}
|
||||
*/
|
||||
this.status = Status.IDLE;
|
||||
|
||||
/**
|
||||
* The current session limit of the client.
|
||||
* @type {?Object}
|
||||
* @prop {number} total Total number of identifies available
|
||||
* @prop {number} remaining Number of identifies remaining
|
||||
* @prop {number} reset_after Number of milliseconds after which the limit resets
|
||||
*/
|
||||
this.sessionStartLimit = null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sends a heartbeat on the available connection.
|
||||
* @returns {void}
|
||||
* The average ping of all WebSocketShards
|
||||
* @type {number}
|
||||
* @readonly
|
||||
*/
|
||||
heartbeat() {
|
||||
if (!this.connection) return this.debug('No connection to heartbeat');
|
||||
return this.connection.heartbeat();
|
||||
get ping() {
|
||||
const sum = this.shards.reduce((a, b) => a + b.ping, 0);
|
||||
return sum / this.shards.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Emits a debug event.
|
||||
* @param {string} message Debug message
|
||||
* @returns {void}
|
||||
* @private
|
||||
*/
|
||||
debug(message) {
|
||||
return this.client.emit(Events.DEBUG, `[ws] ${message}`);
|
||||
this.client.emit(Events.DEBUG, `[connection] ${message}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Destroy the client.
|
||||
* @returns {void} Whether or not destruction was successful
|
||||
* Handles the session identify rate limit for a shard.
|
||||
* @param {WebSocketShard} shard Shard to handle
|
||||
* @private
|
||||
*/
|
||||
destroy() {
|
||||
if (!this.connection) {
|
||||
this.debug('Attempted to destroy WebSocket but no connection exists!');
|
||||
async _handleSessionLimit(shard) {
|
||||
this.sessionStartLimit = await this.client.api.gateway.bot.get().then(r => r.session_start_limit);
|
||||
const { remaining, reset_after } = this.sessionStartLimit;
|
||||
if (remaining !== 0) {
|
||||
this.spawn();
|
||||
} else {
|
||||
shard.debug(`Exceeded identify threshold, setting a timeout for ${reset_after} ms`);
|
||||
setTimeout(() => this.spawn(), this.sessionStartLimit.reset_after);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Used to spawn WebSocketShards.
|
||||
* @param {?WebSocketShard|WebSocketShard[]|number|string} query The WebSocketShards to be spawned
|
||||
* @returns {void}
|
||||
* @private
|
||||
*/
|
||||
spawn(query) {
|
||||
if (query !== undefined) {
|
||||
if (Array.isArray(query)) {
|
||||
for (const item of query) {
|
||||
if (!this.spawnQueue.includes(item)) this.spawnQueue.push(item);
|
||||
}
|
||||
} else if (!this.spawnQueue.includes(query)) {
|
||||
this.spawnQueue.push(query);
|
||||
}
|
||||
}
|
||||
|
||||
if (this.spawning || !this.spawnQueue.length) return;
|
||||
|
||||
this.spawning = true;
|
||||
let item = this.spawnQueue.shift();
|
||||
|
||||
if (typeof item === 'string' && !isNaN(item)) item = Number(item);
|
||||
if (typeof item === 'number') {
|
||||
const shard = new WebSocketShard(this, item, this.shards[item]);
|
||||
this.shards[item] = shard;
|
||||
shard.once(Events.READY, () => {
|
||||
this.spawning = false;
|
||||
this.client.setTimeout(() => this._handleSessionLimit(shard), 5000);
|
||||
});
|
||||
shard.once(Events.INVALIDATED, () => {
|
||||
this.spawning = false;
|
||||
});
|
||||
} else if (item instanceof WebSocketShard) {
|
||||
item.reconnect();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a connection to a gateway.
|
||||
* @param {string} [gateway=this.gateway] The gateway to connect to
|
||||
* @returns {void}
|
||||
* @private
|
||||
*/
|
||||
connect(gateway = this.gateway) {
|
||||
this.gateway = gateway;
|
||||
|
||||
if (typeof this.client.options.shards === 'number') {
|
||||
this.debug('Spawning 1 shard');
|
||||
this.spawn(this.client.options.shards);
|
||||
} else if (Array.isArray(this.client.options.shards)) {
|
||||
this.debug(`Spawning ${this.client.options.shards.length} shards`);
|
||||
for (let i = 0; i < this.client.options.shards.length; i++) {
|
||||
this.spawn(this.client.options.shards[i]);
|
||||
}
|
||||
} else {
|
||||
this.debug(`Spawning ${this.client.options.shardCount} shards`);
|
||||
for (let i = 0; i < this.client.options.shardCount; i++) {
|
||||
this.spawn(i);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Processes a packet and queues it if this WebSocketManager is not ready.
|
||||
* @param {Object} packet The packet to be handled
|
||||
* @param {WebSocketShard} shard The shard that will handle this packet
|
||||
* @returns {boolean}
|
||||
* @private
|
||||
*/
|
||||
handlePacket(packet, shard) {
|
||||
if (packet && this.status !== Status.READY) {
|
||||
if (!BeforeReadyWhitelist.includes(packet.t)) {
|
||||
this.packetQueue.push({ packet, shardID: shard.id });
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (this.packetQueue.length) {
|
||||
const item = this.packetQueue.shift();
|
||||
this.client.setImmediate(() => {
|
||||
this.handlePacket(item.packet, this.shards[item.shardID]);
|
||||
});
|
||||
}
|
||||
|
||||
if (packet && PacketHandlers[packet.t]) {
|
||||
PacketHandlers[packet.t](this.client, packet, shard);
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks whether the client is ready to be marked as ready.
|
||||
* @returns {boolean}
|
||||
* @private
|
||||
*/
|
||||
checkReady() {
|
||||
if (this.shards.filter(s => s).length !== this.client.options.shardCount ||
|
||||
this.shards.some(s => s && s.status !== Status.READY)) {
|
||||
return false;
|
||||
}
|
||||
return this.connection.destroy();
|
||||
|
||||
let unavailableGuilds = 0;
|
||||
for (const guild of this.client.guilds.values()) {
|
||||
if (!guild.available) unavailableGuilds++;
|
||||
}
|
||||
if (unavailableGuilds === 0) {
|
||||
this.status = Status.NEARLY;
|
||||
if (!this.client.options.fetchAllMembers) return this.triggerReady();
|
||||
// Fetch all members before marking self as ready
|
||||
const promises = this.client.guilds.map(g => g.members.fetch());
|
||||
Promise.all(promises)
|
||||
.then(() => this.triggerReady())
|
||||
.catch(e => {
|
||||
this.debug(`Failed to fetch all members before ready! ${e}`);
|
||||
this.triggerReady();
|
||||
});
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Send a packet on the available WebSocket.
|
||||
* @param {Object} packet Packet to send
|
||||
* Causes the client to be marked as ready and emits the ready event.
|
||||
* @returns {void}
|
||||
* @private
|
||||
*/
|
||||
send(packet) {
|
||||
if (!this.connection) {
|
||||
this.debug('No connection to websocket');
|
||||
triggerReady() {
|
||||
if (this.status === Status.READY) {
|
||||
this.debug('Tried to mark self as ready, but already ready');
|
||||
return;
|
||||
}
|
||||
this.connection.send(packet);
|
||||
this.status = Status.READY;
|
||||
|
||||
/**
|
||||
* Emitted when the client becomes ready to start working.
|
||||
* @event Client#ready
|
||||
*/
|
||||
this.client.emit(Events.READY);
|
||||
|
||||
this.handlePacket();
|
||||
}
|
||||
|
||||
/**
|
||||
* Connects the client to a gateway.
|
||||
* @param {string} gateway The gateway to connect to
|
||||
* @returns {boolean}
|
||||
* Broadcasts a message to every shard in this WebSocketManager.
|
||||
* @param {*} packet The packet to send
|
||||
*/
|
||||
connect(gateway) {
|
||||
if (!this.connection) {
|
||||
this.connection = new WebSocketConnection(this, gateway);
|
||||
return true;
|
||||
broadcast(packet) {
|
||||
for (const shard of this.shards) {
|
||||
if (!shard) continue;
|
||||
shard.send(packet);
|
||||
}
|
||||
switch (this.connection.status) {
|
||||
case Status.IDLE:
|
||||
case Status.DISCONNECTED:
|
||||
this.connection.connect(gateway, 5500);
|
||||
return true;
|
||||
default:
|
||||
this.debug(`Couldn't connect to ${gateway} as the websocket is at state ${this.connection.status}`);
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Destroys all shards.
|
||||
* @returns {void}
|
||||
* @private
|
||||
*/
|
||||
destroy() {
|
||||
this.gateway = undefined;
|
||||
// Lock calls to spawn
|
||||
this.spawning = true;
|
||||
|
||||
for (const shard of this.shards) {
|
||||
if (!shard) continue;
|
||||
shard.destroy();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user