ton/SQLLLM
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
tonaerospace
2025-03-18 08:37:35 +07:00
parent e391547991
commit 7fd0d6269a
2 changed files with 106 additions and 107 deletions
Download Patch File Download Diff File Expand all files Collapse all files

150
src/llmfunction.jl
View File

@@ -829,111 +829,115 @@ function compareState(query, highValueStateList)
- The user's attempted actions and their corresponding results - The user's attempted actions and their corresponding results
</At each round of conversation, you will be given the following> </At each round of conversation, you will be given the following>
<You should then respond to the user with the following> <You should then respond to the user with the following>
Selected_response: the number of the most accurate and relevant response Comparison: a comparison of the results from each attempt
Rationale: a brief explanation of why you selected this response Rationale: a brief explanation of why the selected response is the most accurate and relevant
Selected_response_number: the number the selected response in the list of results
</You should then respond to the user with the following> </You should then respond to the user with the following>
<You should only respond in format as described below> <You should only respond in format as described below>
- Selected_response: ... Comparison: ...
- Rationale: ... Rationale: ...
Selected_response_number: ...
</You should only respond in format as described below> </You should only respond in format as described below>
<Here are some examples> <Here are some examples>
User's question: "How many German wines do you have?" User's question: "How many German wines do you have?"
Attempt 1: Attempt 1:
Action: SELECT * FROM wines WHERE country = 'Germany' Action: SELECT COUNT(*) FROM wines WHERE country = 'Germany'
Result: 100 wines Result: 100 wines
Attempt 2: Attempt 2:
Action: SELECT * FROM wines WHERE country = 'Germany' AND type = 'Red' Action: SELECT COUNT(*) FROM wines WHERE country = 'Germany' AND type = 'Red'
Result: 50 red wines Result: 50 red wines
Selected_response: 1 Comparison: The second attempt counts only German red wines while the first attempt includes all German wines.
Rationale: The question is about German wines, so the most accurate response is the one that includes all German wines. Rationale: The user is asking for the number of German wines without specifying a type, so the most accurate response is the first attempt because it includes all German wines.
Selected_response_number: 1
</Here are some examples> </Here are some examples>
Let's begin! Let's begin!
""" """
thoughthistory = "" # thoughthistory = ""
for (k, v) in state[:thoughtHistory] # for (k, v) in state[:thoughtHistory]
thoughthistory *= "$k: $v\n" # thoughthistory *= "$k: $v\n"
end # end
errornote = "" # errornote = ""
for attempt in 1:10 # for attempt in 1:10
errorFlag = false # errorFlag = false
usermsg = # usermsg =
""" # """
Trajectory: $thoughthistory # Trajectory: $thoughthistory
Error_note: $errornote # Error_note: $errornote
""" # """
_prompt = # _prompt =
[ # [
Dict(:name=> "system", :text=> systemmsg), # Dict(:name=> "system", :text=> systemmsg),
Dict(:name=> "user", :text=> usermsg) # Dict(:name=> "user", :text=> usermsg)
] # ]
# put in model format # # put in model format
prompt = GeneralUtils.formatLLMtext(_prompt; formatname="qwen") # prompt = GeneralUtils.formatLLMtext(_prompt; formatname="qwen")
header = ["Trajectory_evaluation:", "Answer_evaluation:", "Accepted_as_answer:", "Score:", "Suggestion:"] # header = ["Trajectory_evaluation:", "Answer_evaluation:", "Accepted_as_answer:", "Score:", "Suggestion:"]
dictkey = ["trajectory_evaluation", "answer_evaluation", "accepted_as_answer", "score", "suggestion"] # dictkey = ["trajectory_evaluation", "answer_evaluation", "accepted_as_answer", "score", "suggestion"]
response = text2textInstructLLM(prompt) # response = text2textInstructLLM(prompt)
# sometime LLM output something like **Comprehension**: which is not expected # # sometime LLM output something like **Comprehension**: which is not expected
response = replace(response, "**"=>"") # response = replace(response, "**"=>"")
response = replace(response, "***"=>"") # response = replace(response, "***"=>"")
# make sure every header is in the response # # make sure every header is in the response
for i in header # for i in header
detected = GeneralUtils.detect_keyword(i, response) # detected = GeneralUtils.detect_keyword(i, response)
if detected === nothing # if detected === nothing
errornote = "Your previous response didn't provide $i" # errornote = "Your previous response didn't provide $i"
errorFlag = true # errorFlag = true
end # end
end # end
if errorFlag # if errorFlag
continue # skip to the next iteration # continue # skip to the next iteration
end # end
responsedict = GeneralUtils.textToDict(response, header; # responsedict = GeneralUtils.textToDict(response, header;
dictKey=dictkey, symbolkey=true) # dictKey=dictkey, symbolkey=true)
responsedict[:score] = responsedict[:score][1] # some time "6\nThe trajectories are incomplete" is generated but I only need the number. # responsedict[:score] = responsedict[:score][1] # some time "6\nThe trajectories are incomplete" is generated but I only need the number.
try # try
responsedict[:score] = parse(Int, responsedict[:score]) # convert string "5" into integer 5 # responsedict[:score] = parse(Int, responsedict[:score]) # convert string "5" into integer 5
catch # catch
continue # continue
end # end
accepted_as_answer::AbstractString = responsedict[:accepted_as_answer] # accepted_as_answer::AbstractString = responsedict[:accepted_as_answer]
if accepted_as_answer ∉ ["Yes", "No"] # [PENDING] add errornote into the prompt # if accepted_as_answer ∉ ["Yes", "No"] # [PENDING] add errornote into the prompt
error("generated accepted_as_answer has wrong format") # error("generated accepted_as_answer has wrong format")
end # end
# add to state here instead to in transition() because the latter causes julia extension crash (a bug in julia extension) # # add to state here instead to in transition() because the latter causes julia extension crash (a bug in julia extension)
state[:evaluation] = "$(responsedict[:trajectory_evaluation]) $(responsedict[:answer_evaluation])" # state[:evaluation] = "$(responsedict[:trajectory_evaluation]) $(responsedict[:answer_evaluation])"
state[:evaluationscore] = responsedict[:score] # state[:evaluationscore] = responsedict[:score]
state[:accepted_as_answer] = responsedict[:accepted_as_answer] # state[:accepted_as_answer] = responsedict[:accepted_as_answer]
state[:suggestion] = responsedict[:suggestion] # state[:suggestion] = responsedict[:suggestion]
# mark as terminal state when the answer is achieved # # mark as terminal state when the answer is achieved
if accepted_as_answer == "Yes" # if accepted_as_answer == "Yes"
# mark the state as terminal state because the evaluation say so. # # mark the state as terminal state because the evaluation say so.
state[:isterminal] = true # state[:isterminal] = true
# evaluation score as reward because different answers hold different value for the user. # # evaluation score as reward because different answers hold different value for the user.
state[:reward] = responsedict[:score] # state[:reward] = responsedict[:score]
end # end
println("\n~~~ Evaluator() ", @__FILE__, ":", @__LINE__, " $(Dates.now())") # println("\n~~~ Evaluator() ", @__FILE__, ":", @__LINE__, " $(Dates.now())")
pprintln(Dict(responsedict)) # pprintln(Dict(responsedict))
return responsedict[:score] # return responsedict[:score]
end # end
error("Evaluator failed to generate an evaluation, Response: \n$response\n<|End of error|>") # error("Evaluator failed to generate an evaluation, Response: \n$response\n<|End of error|>")
end end

59
system_prompt_template.jl
View File

@@ -85,12 +85,12 @@ Example:
- Manage stakeholder relationships - Manage stakeholder relationships
- Drive fundraising and business development - Drive fundraising and business development
</Your responsibilities include> </Your responsibilities include>
<At each round of conversation, you will be given> <At each round of conversation, you will be given the following>
- User's specific caregiving challenges Challenges: user's specific caregiving challenges
- Context and severity of the situation Context: context and severity of the situation
- Feedback from family caregivers Feedback: comments from family caregivers
- Potential solutions based on immediate and long-term impact Solutions: potential solution based on immediate and long-term impact
</At each round of conversation, you will be given> </At each round of conversation, you will be given the following>
<You must follow the following guidelines> <You must follow the following guidelines>
- Always prioritize patient safety and well-being - Always prioritize patient safety and well-being
- Maintain empathy and understanding in all interactions - Maintain empathy and understanding in all interactions
@@ -117,39 +117,34 @@ Example:
</You should only respond in format as described below> </You should only respond in format as described below>
<Here are some examples> <Here are some examples>
Example 1: Example 1:
User: "My mother has been bedridden for 3 months, and I'm struggling to monitor her condition while working from home. I'm worried I might miss important signs of deterioration." Challenges: "My mother needs constant monitoring at night, but I'm exhausted from lack of sleep."
Context: Elderly patient with dementia, requires 24/7 supervision
Comprehension: The user is facing challenges balancing work-from-home responsibilities with monitoring their bedridden mother's health condition. There's anxiety about potentially missing critical health changes. Feedback: "Need urgent solution for night monitoring"
Solutions: Smart monitoring system with motion sensors and alerts
Comprehension: The caregiver is experiencing severe sleep deprivation due to nighttime monitoring requirements
Plan: Plan:
1. Assess current monitoring needs and gaps 1. Assess current monitoring needs
2. Propose smart monitoring system setup 2. Propose smart monitoring system installation
3. Implement remote alerts and notifications 3. Set up emergency alert system
4. Provide training on using the system 4. Train family on system usage
5. Set up regular check-ins for system optimization
Action_name: CHATBOX Action_name: CHATBOX
Action_input: Gather specific information about mother's condition, current monitoring methods, and work schedule to tailor the smart monitoring solution effectively. Action_input: Discuss specific nighttime behaviors and incidents to determine optimal sensor placement and alert thresholds
Example 2: Example 2:
User: "I need to implement a fall detection system for my father who is partially mobile but at high risk of falls. What solutions do you recommend?" Challenges: "Managing medication schedules is becoming overwhelming"
Context: Patient on multiple medications with complex timing requirements
Comprehension: The user requires a reliable fall detection system for a partially mobile patient with high fall risk. Safety is the primary concern while maintaining the patient's limited mobility. Feedback: "Need help with medication management"
Solutions: Automated medication reminder and tracking system
Comprehension: Caregiver struggling with complex medication management tasks
Plan: Plan:
1. Review available fall detection technologies 1. Review current medication schedule
2. Check compatibility with existing infrastructure 2. Implement automated reminder system
3. Evaluate cost and implementation requirements 3. Set up medication tracking log
4. Propose specific solution options 4. Connect with pharmacy for refill automation
5. Create implementation timeline Action_name: IMPLEMENT
Action_input: Deploy medication management module with smart alerts and compliance tracking
Action_name: CHECKRESOURCES
Action_input: Query database for available fall detection systems, focusing on:
- Motion sensors
- Wearable devices
- Computer vision systems
- Integration capabilities
- Cost ranges
</Here are some examples> </Here are some examples>
Let's begin! Let's begin!