forked from chatwoot/chatwoot
-
Notifications
You must be signed in to change notification settings - Fork 47
feat: messages and contacts history fetch #71
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
CayoPOliveira
wants to merge
49
commits into
main
Choose a base branch
from
Cayo-Phellipe-Ramalho-de-Oliveira/CU-86a8xttnm/Fazer-fetch-do-historico-de-mensagens
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
49 commits
Select commit
Hold shift + click to select a range
a5a9c89
feat: add messaging history set handler for WhatsApp and restore cont…
CayoPOliveira b247261
feat: enhance messaging history processing and message handling in Wh…
CayoPOliveira d773f1c
feat: include MessagingHistorySet handler in IncomingMessageBaileysSe…
CayoPOliveira 49c1ac9
feat: add options to sync messages and contacts in WhatsApp settings
CayoPOliveira 2a33ae7
feat: implement initial sync for contacts and message history in setu…
CayoPOliveira 83dc898
chore: correct phone number variable usage in create_contact method
CayoPOliveira a999f8d
refactor: rename methods for avoid impact other baileys handlers
CayoPOliveira b08d42a
fix: correct argument order in history_handle_attach_media method call
CayoPOliveira 76c2184
fix: improve error logging for attachment download failure
CayoPOliveira 455c69a
fix: correct history_filename method in messaging_history.set
CayoPOliveira ae06267
chore: remove unused helper includes in messaging_history_set
CayoPOliveira d00f5e8
fix: variables names in attachment handling methods
CayoPOliveira 48e159c
fix: update fetch_message_history method to include phone_number para…
CayoPOliveira 35e4d2e
feat: add BAILEYS_MESSAGE_HISTORY_COUNT environment variable for mess…
CayoPOliveira a584148
fix: rename message_content method to history_message_content for cla…
CayoPOliveira 23d88ed
feat: implement fetch_message_history method to retrieve message hist…
CayoPOliveira 7f8275d
fix: simplify setup_channel_provider method by removing unnecessary s…
CayoPOliveira 0ec7602
fix: update process_messaging_history_set to conditionally sync conta…
CayoPOliveira 91ff533
fix: update create_contact method to include inbox parameter in Conta…
CayoPOliveira 345f3cf
fix: rename params name in message creation and attachment creation
CayoPOliveira ad35f31
fix: reduce default message history count to 5 in fetch_message_histo…
CayoPOliveira 2d01f7f
fix: refactor fetch_message_history methods to streamline parameters …
CayoPOliveira 5e534d2
fix: validate presence of key, message, and messageTimestamp in histo…
CayoPOliveira 16c0c1e
fix: add error handling for contact and message processing in process…
CayoPOliveira e2b634b
fix: add presence validation for key, message, and messageTimestamp i…
CayoPOliveira 40e7cda
fix: update history_create_message method to use raw_message directly…
CayoPOliveira b841a1b
fix: add contact attributes to ContactInboxWithContactBuilder in crea…
CayoPOliveira 80c5bcf
fix: update history_message_content_attributes to handle multiple uns…
CayoPOliveira 682d263
fix: remove media attachment handling methods from history_handle_att…
CayoPOliveira 8f03dbc
fix: remove redundant WhatsApp sync labels from inbox management loca…
CayoPOliveira 471b441
fix: update sync label order in inbox management localization file
CayoPOliveira 8745ec0
fix: correct phone number attribute in fetch_message_history method
CayoPOliveira 48a06ad
fix: enhance message flooding prevention and update fetch_message_his…
gabrieljablonski d86cf1e
chore: update fetch_message_history method to accept oldest_message p…
CayoPOliveira 456ff17
chore: add fetch_message_history to error handling methods in Whatsap…
CayoPOliveira 23a9246
fix: handle blank contact IDs and update name assignment logic in cre…
CayoPOliveira ffcc625
fix: update history_cache_message_source_id_in_redis to use set with …
CayoPOliveira 6327952
test: create spec in message model for skip message flooding validati…
CayoPOliveira e93d18a
fix: skip processing of blank contact IDs in messaging history set
CayoPOliveira 8cd5da3
refactor: remove unused process_status method from MessagingHistorySet
CayoPOliveira 1859630
refactor: simplify process_messaging_history_set method by removing u…
CayoPOliveira 0d47da5
refactor: simplify process_messaging_history_set and create_contact v…
CayoPOliveira 5f15fa4
fix: exclude unsupported message types from ignore processing list in…
CayoPOliveira 05a6569
refactor: update conversation creation to use history_conversation_pa…
CayoPOliveira b1cf9c8
test: add specs for messaging-history.set event handling
CayoPOliveira e38f893
refactor: streamline process_messaging_history_set by consolidating c…
CayoPOliveira 1537b1d
refactor: enhance history_handle_message and history_message_valid me…
CayoPOliveira 8cc5b26
test: add specs for sync_full_history and sync_contacts in setup_chan…
CayoPOliveira dde9ed4
chore: update fetch_message_history to ensure request body is properl…
CayoPOliveira File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
207 changes: 207 additions & 0 deletions
207
app/services/whatsapp/baileys_handlers/messaging_history_set.rb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,207 @@ | ||
module Whatsapp::BaileysHandlers::MessagingHistorySet # rubocop:disable Metrics/ModuleLength | ||
private | ||
|
||
def process_messaging_history_set | ||
provider_config = inbox.channel.provider_config | ||
|
||
return unless provider_config['sync_contacts'].presence || provider_config['sync_full_history'].presence | ||
|
||
process_contacts(params) | ||
process_messages(params) if provider_config['sync_full_history'].presence | ||
end | ||
|
||
def process_contacts(params) | ||
contacts = params.dig(:data, :contacts) || [] | ||
contacts.each do |contact| | ||
create_contact(contact) | ||
end | ||
end | ||
|
||
def process_messages(params) | ||
messages = params.dig(:data, :messages) || [] | ||
messages.each do |message| | ||
history_handle_message(message) | ||
end | ||
end | ||
|
||
def create_contact(contact) | ||
return unless contact[:id].present? && jid_user?(contact[:id]) | ||
|
||
phone_number = history_phone_number_from_jid(contact[:id]) | ||
name = contact[:verifiedName].presence || contact[:notify].presence || contact[:name].presence || phone_number | ||
::ContactInboxWithContactBuilder.new( | ||
# FIXME: update the source_id to complete jid in future | ||
source_id: phone_number, | ||
inbox: inbox, | ||
contact_attributes: { name: name, phone_number: "+#{phone_number}" } | ||
).perform | ||
end | ||
|
||
# TODO: Refactor jid_type method in helpers to receive the jid as an argument and use it here | ||
def jid_user?(jid) | ||
server = jid.split('@').last | ||
server == 's.whatsapp.net' || server == 'c.us' | ||
end | ||
|
||
# TODO: Refactor this method in helpers to receive the jid as an argument and remove it from here | ||
def history_phone_number_from_jid(jid) | ||
jid.split('@').first.split(':').first.split('_').first | ||
end | ||
|
||
def history_handle_message(raw_message) | ||
return unless history_message_valid?(raw_message) | ||
|
||
id = raw_message.dig(:key, :id) | ||
jid = raw_message.dig(:key, :remoteJid) | ||
|
||
history_cache_message_source_id_in_redis(id) | ||
begin | ||
contact_inbox = find_contact_inbox(jid) | ||
unless contact_inbox.contact | ||
Rails.logger.warn "Contact not found for message: #{id}" | ||
return | ||
end | ||
|
||
history_create_message(raw_message, contact_inbox) | ||
ensure | ||
history_clear_message_source_id_from_redis(id) | ||
end | ||
end | ||
|
||
def history_message_valid?(raw_message) # rubocop:disable Metrics/CyclomaticComplexity | ||
id = raw_message.dig(:key, :id) | ||
jid = raw_message.dig(:key, :remoteJid) | ||
|
||
id.present? && | ||
jid.present? && | ||
raw_message[:message].present? && | ||
raw_message[:messageTimestamp].present? && | ||
jid_user?(jid) && | ||
!history_message_type(raw_message[:message]).in?(%w[protocol context]) && | ||
!history_find_message_by_source_id(id) && | ||
!history_message_under_process?(id) | ||
end | ||
|
||
# TODO: Refactor this method in helpers to receive the raw message as an argument and remove it from here | ||
def history_message_type(message_content) # rubocop:disable Metrics/CyclomaticComplexity,Metrics/PerceivedComplexity,Metrics/MethodLength | ||
if message_content.key?(:conversation) || message_content.dig(:extendedTextMessage, :text).present? | ||
'text' | ||
elsif message_content.key?(:imageMessage) | ||
'image' | ||
elsif message_content.key?(:audioMessage) | ||
'audio' | ||
elsif message_content.key?(:videoMessage) | ||
'video' | ||
elsif message_content.key?(:documentMessage) || message_content.key?(:documentWithCaptionMessage) | ||
'file' | ||
elsif message_content.key?(:stickerMessage) | ||
'sticker' | ||
elsif message_content.key?(:reactionMessage) | ||
'reaction' | ||
elsif message_content.key?(:editedMessage) | ||
'edited' | ||
elsif message_content.key?(:protocolMessage) | ||
'protocol' | ||
elsif message_content.key?(:messageContextInfo) | ||
'context' | ||
else | ||
'unsupported' | ||
end | ||
end | ||
|
||
# TODO: Remove this method when include helpers in this module, after update the methods to receive arguments | ||
def history_find_message_by_source_id(source_id) | ||
return unless source_id | ||
|
||
Message.find_by(source_id: source_id).presence | ||
end | ||
|
||
def find_contact_inbox(jid) | ||
phone_number = history_phone_number_from_jid(jid) | ||
::ContactInboxWithContactBuilder.new( | ||
# FIXME: update the source_id to complete jid in future | ||
source_id: phone_number, | ||
inbox: inbox, | ||
contact_attributes: { name: phone_number, phone_number: "+#{phone_number}" } | ||
).perform | ||
end | ||
|
||
# TODO: Refactor this method in helpers to receive the source_id as an argument and remove it from here | ||
def history_message_under_process?(source_id) | ||
key = format(Redis::RedisKeys::MESSAGE_SOURCE_KEY, id: source_id) | ||
Redis::Alfred.get(key) | ||
end | ||
|
||
# TODO: Refactor this method in helpers to receive the source_id as an argument and deprecate setex, then remove it from here | ||
def history_cache_message_source_id_in_redis(source_id) | ||
key = format(Redis::RedisKeys::MESSAGE_SOURCE_KEY, id: source_id) | ||
::Redis::Alfred.set(key, true, nx: true, ex: 1.day) | ||
end | ||
|
||
# TODO: Refactor this method in helpers to receive the source_id as an argument and remove it from here | ||
def history_clear_message_source_id_from_redis(source_id) | ||
key = format(Redis::RedisKeys::MESSAGE_SOURCE_KEY, id: source_id) | ||
::Redis::Alfred.delete(key) | ||
end | ||
|
||
def history_create_message(raw_message, contact_inbox) | ||
conversation = get_conversation(contact_inbox) | ||
inbox = contact_inbox.inbox | ||
message = conversation.messages.build( | ||
skip_prevent_message_flooding: true, | ||
content: history_message_content(raw_message), | ||
account_id: inbox.account_id, | ||
inbox_id: inbox.id, | ||
source_id: raw_message[:key][:id], | ||
sender: history_incoming?(raw_message) ? contact_inbox.contact : inbox.account.account_users.first.user, | ||
sender_type: history_incoming?(raw_message) ? 'Contact' : 'User', | ||
message_type: history_incoming?(raw_message) ? :incoming : :outgoing, | ||
content_attributes: history_message_content_attributes(raw_message), | ||
status: 'read' | ||
) | ||
|
||
message.save! | ||
end | ||
|
||
# NOTE: See reference in app/services/whatsapp/incoming_message_base_service.rb:97 | ||
def get_conversation(contact_inbox) | ||
return contact_inbox.conversations.last if contact_inbox.inbox.lock_to_single_conversation | ||
|
||
# NOTE: if lock to single conversation is disabled, create a new conversation if previous conversation is resolved | ||
return contact_inbox.conversations.where.not(status: :resolved).last.presence || | ||
::Conversation.create!(history_conversation_params(contact_inbo 1C6A x)) | ||
end | ||
|
||
# TODO: Refactor this method in helpers to receive the contact_inbox as an argument and remove it from here | ||
def history_conversation_params(contact_inbox) | ||
{ | ||
account_id: contact_inbox.inbox.account_id, | ||
inbox_id: contact_inbox.inbox.id, | ||
contact_id: contact_inbox.contact.id, | ||
contact_inbox_id: contact_inbox.id | ||
} | ||
end | ||
|
||
# TODO: Refactor this method in helpers to receive the raw message as an argument and remove it from here | ||
def history_incoming?(raw_message) | ||
!raw_message[:key][:fromMe] | ||
end | ||
|
||
# TODO: Refactor this method in helpers to receive the raw message as an argument and remove it from here | ||
def history_message_content(raw_message) | ||
raw_message.dig(:message, :conversation) || | ||
raw_message.dig(:message, :extendedTextMessage, :text) || | ||
raw_message.dig(:message, :imageMessage, :caption) || | ||
raw_message.dig(:message, :videoMessage, :caption) || | ||
raw_message.dig(:message, :documentMessage, :caption).presence || | ||
raw_message.dig(:message, :documentWithCaptionMessage, :message, :documentMessage, :caption) || | ||
raw_message.dig(:message, :reactionMessage, :text) | ||
end | ||
|
||
def history_message_content_attributes(raw_message) | ||
{ | ||
external_created_at: baileys_extract_message_timestamp(raw_message[:messageTimestamp]), | ||
is_unsupported: history_message_type(raw_message[:message]).in?(%w[image file video audio sticker unsupported]) || nil | ||
}.compact | ||
end | ||
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🛠️ Refactor suggestion
Add safety check for account users to prevent nil reference error.
The code assumes
inbox.account.account_users.first.user
will always exist, but this could returnnil
if no account users are present.🤖 Prompt for AI Agents