rich kopelow
V.I.P
Posts: 47
Joined: Thu Aug 25, 2016 7:45 am

exact expression match fails with score of -1

Mon Mar 05, 2018 1:02 am

I am finding that at times an exact match fails to be recognized (with a score of -1). Sometimes when I use an input string of "who" (in my app or in studio) the General.helpYes intent gets fired with a score of .333 while General.who gets a score of -1. Other times General.who fires off no problem.

Thanks,
Rich.

I have the following siml:
<Dialog Name="General">

<Intent Name="who">
<Expression Value="who" />
<Response>
...
</Response>
</Intent>

...

<Intent Name="helpYes">
<Context Name="Scene.Intro" />
<Context Name="Question.HelpMe" />
<Expression Value="@sys.positive" />
<Expression Value="how" />
<Expression Value="what do you" />
<Expression Value="with what" />
<Response>
...
</Response>
</Intent>

...

User avatar
Fantom
Help & Support
Posts: 304
Joined: Fri Oct 25, 2013 9:20 pm

Re: exact expression match fails with score of -1

Mon Mar 05, 2018 1:05 pm

can you tell me which version of syn bot are you using? I changed the siml code a little and tried to test in a console project and it is kinda working for me in version 2.9.4

SIML Code

<Siml>
  <Dialog Name="General">
    
    <Intent Name="who">
      <Expression Value="who" />
      <Response>You said WHO.<Context Add="scene.intro"/><Context Add="question.helpme"/></Response>
    </Intent>
    
    <Intent Name="helpYes">
      <Context Name="Scene.Intro" />
      <Context Name="Question.HelpMe" />
      <Expression Value="@sys.positive" />
      <Expression Value="how" />
      <Expression Value="what do you" />
      <Expression Value="with what" />
      <Response>User meant YES.</Response>
    </Intent>
    
  </Dialog>
</Siml>

CSharp Code

class Program
{
	static void Main(string[] args)
	{
		var bot = new OscovaBot();
		var document = XDocument.Load("siml-document.siml");
		bot.Import(document);

		bot.Trainer.StartTraining();

		bot.MainUser.ResponseReceived += (sender, eventArgs) =>
		{
			Console.WriteLine($"Response: {eventArgs.Response.Text}");
		};

		while (true)
		{
			Console.WriteLine("Type your message");
			var message = Console.ReadLine();
			var evaluationResult = bot.Evaluate(message);
			evaluationResult.Invoke();
		}
	}
}

rich kopelow
V.I.P
Posts: 47
Joined: Thu Aug 25, 2016 7:45 am

Re: exact expression match fails with score of -1

Thu Mar 08, 2018 3:30 am

My app (2.9.4) and studio (2.5.7.0) both exhibit the same behavior. I'm not simply trying to cascade A -> B my scenario is a little more involved, but I've added a "complete" siml here to illustrate the problem fully. The input stream is who, who, who, x, who and the result is good, good, good, good, fail. I've included the log as well. I realize that Contexts should be required to enable Intents that have Context elements, but I don't believe they should be turning off intents that don't have any Context elements. (Do I just need to add some "default" context?)

Thanks,
Rich.

<Siml>
<Dialog Name="General">

<Intent Name="who">
<Expression Value="who" />
<Response>You said WHO.</Response>
</Intent>

<Intent Name="x">
<Expression Value="x" />
<Response>You said X.<Context Add="scene.intro"/><Context Add="question.helpme"/></Response>
</Intent>

<Intent Name="helpYes">
<Context Name="Scene.Intro" />
<Context Name="Question.HelpMe" />
<Expression Value="@sys.positive" />
<Expression Value="how" />
<Expression Value="what do you" />
<Expression Value="with what" />
<Response>User meant YES.</Response>
</Intent>

</Dialog>
</Siml>

[18-03-07 04:49:54.209] [Info] [OscovaBot] Initializing Bot...
[18-03-07 04:49:54.212] [Info] [OscovaBot] Created new user with ID: '5418bef0-6205-48a4-bb35-f38a5579e081'
[18-03-07 04:49:54.215] [Info] [OscovaBot] Initializing Bot...
[18-03-07 04:49:54.215] [Info] [OscovaBot] Created new user with ID: '1d0939c5-ce2e-4d2a-9fc5-fc803020f57f'
[18-03-07 04:49:54.218] [Info] Training started.
[18-03-07 04:49:54.219] [Info] Training Dialogs...
[18-03-07 04:49:54.219] [Info] [Trainer] Training for Dialog: 'General'
[18-03-07 04:49:54.219] [Info] [EnglishTokenizer] Normalizing "who"
[18-03-07 04:49:54.219] [Info] [EnglishTokenizer] Normalized Value: "who"
[18-03-07 04:49:54.219] [Info] [Trainer] Successfully added intent: 'General.who'
[18-03-07 04:49:54.219] [Info] [EnglishTokenizer] Normalizing "x"
[18-03-07 04:49:54.222] [Info] [EnglishTokenizer] Normalized Value: "x"
[18-03-07 04:49:54.224] [Info] [Trainer] Successfully added intent: 'General.x'
[18-03-07 04:49:54.224] [Info] [EnglishTokenizer] Normalizing "Y8V0V8HKQZ2O8GGZ2BTDYZ5A8PZ3IKDEKIB2"
[18-03-07 04:49:54.225] [Info] [EnglishTokenizer] Normalized Value: "Y8V0V8HKQZ2O8GGZ2BTDYZ5A8PZ3IKDEKIB2"
[18-03-07 04:49:54.225] [Info] [EnglishTokenizer] Normalizing "how"
[18-03-07 04:49:54.226] [Info] [EnglishTokenizer] Normalized Value: "how"
[18-03-07 04:49:54.234] [Info] [EnglishTokenizer] Normalizing "what do you"
[18-03-07 04:49:54.234] [Info] [EnglishTokenizer] Normalized Value: "what do you"
[18-03-07 04:49:54.238] [Info] [EnglishTokenizer] Normalizing "with what"
[18-03-07 04:49:54.238] [Info] [EnglishTokenizer] Normalized Value: "with what"
[18-03-07 04:49:54.238] [Info] [Trainer] Successfully added intent: 'General.helpYes'
[18-03-07 04:49:54.238] [Info] Default expression not found adding 'DefaultDialog'
[18-03-07 04:49:54.238] [Info] [Trainer] Training for Dialog: 'DefaultDialog'
[18-03-07 04:49:54.239] [Info] [EnglishTokenizer] Normalizing ""
[18-03-07 04:49:54.239] [Info] [EnglishTokenizer] Normalized Value: ""
[18-03-07 04:49:54.239] [Info] [Trainer] Successfully added intent: 'DefaultDialog.Default'
[18-03-07 04:49:54.239] [Info] Loading all registered entity recognizers.
[18-03-07 04:49:54.240] [Warning] [EntryCollection] Skipped adding of Entry "Today" as it already exists in collection.
[18-03-07 04:49:54.240] [Warning] [EntryCollection] Skipped adding of Entry "Tomorrow" as it already exists in collection.
[18-03-07 04:49:54.240] [Warning] [EntryCollection] Skipped adding of Entry "Yesterday" as it already exists in collection.
[18-03-07 04:49:54.240] [Warning] [EntryCollection] Skipped adding of Entry "day after tomorrow" as it already exists in collection.
[18-03-07 04:49:54.240] [Warning] [EntryCollection] Skipped adding of Entry "day before yesterday" as it already exists in collection.
[18-03-07 04:49:54.255] [Info] Training finished.
[18-03-07 04:50:00.908] [Info] [EnglishTokenizer] Normalizing "who"
[18-03-07 04:50:00.908] [Info] [EnglishTokenizer] Normalized Value: "who"
[18-03-07 04:50:00.908] [Info] [OscovaBot] Evaluating a total of "4" intents.
[18-03-07 04:50:00.908] [Info] [OscovaBot] Evaluating request message: "who"
[18-03-07 04:50:00.908] [Info] [RecognizerCollection] Extracting entities for request: 'who'
[18-03-07 04:50:00.908] [Info] [RecognizerCollection] Extracted '0' entities.
[18-03-07 04:50:00.908] [Info] [OscovaBot] Evaluation completed. Total processing time: '0' ms
[18-03-07 04:50:00.920] [Info] [EvaluationResult] Invoking intent: 'General.who'
[18-03-07 04:50:02.697] [Info] [EnglishTokenizer] Normalizing "who"
[18-03-07 04:50:02.697] [Info] [EnglishTokenizer] Normalized Value: "who"
[18-03-07 04:50:02.697] [Info] [OscovaBot] Evaluating a total of "4" intents.
[18-03-07 04:50:02.697] [Info] [OscovaBot] Evaluating request message: "who"
[18-03-07 04:50:02.697] [Info] [RecognizerCollection] Extracting entities for request: 'who'
[18-03-07 04:50:02.697] [Info] [RecognizerCollection] Extracted '0' entities.
[18-03-07 04:50:02.697] [Info] [OscovaBot] Evaluation completed. Total processing time: '0' ms
[18-03-07 04:50:02.709] [Info] [EvaluationResult] Invoking intent: 'General.who'
[18-03-07 04:50:04.757] [Info] [EnglishTokenizer] Normalizing "who"
[18-03-07 04:50:04.757] [Info] [EnglishTokenizer] Normalized Value: "who"
[18-03-07 04:50:04.757] [Info] [OscovaBot] Evaluating a total of "4" intents.
[18-03-07 04:50:04.757] [Info] [OscovaBot] Evaluating request message: "who"
[18-03-07 04:50:04.757] [Info] [RecognizerCollection] Extracting entities for request: 'who'
[18-03-07 04:50:04.757] [Info] [RecognizerCollection] Extracted '0' entities.
[18-03-07 04:50:04.758] [Info] [OscovaBot] Evaluation completed. Total processing time: '0' ms
[18-03-07 04:50:04.769] [Info] [EvaluationResult] Invoking intent: 'General.who'
[18-03-07 04:50:06.048] [Info] [EnglishTokenizer] Normalizing "x"
[18-03-07 04:50:06.048] [Info] [EnglishTokenizer] Normalized Value: "x"
[18-03-07 04:50:06.048] [Info] [OscovaBot] Evaluating a total of "4" intents.
[18-03-07 04:50:06.048] [Info] [OscovaBot] Evaluating request message: "x"
[18-03-07 04:50:06.048] [Info] [RecognizerCollection] Extracting entities for request: 'x'
[18-03-07 04:50:06.048] [Info] [RecognizerCollection] Extracted '0' entities.
[18-03-07 04:50:06.049] [Info] [OscovaBot] Evaluation completed. Total processing time: '0' ms
[18-03-07 04:50:06.065] [Info] [EvaluationResult] Invoking intent: 'General.x'
[18-03-07 04:50:06.065] [Info] [Context] Context item "scene.intro" activated.
[18-03-07 04:50:06.065] [Info] [ContextItem] Context: "scene.intro" is now active.
[18-03-07 04:50:06.065] [Info] [Context] Context item "question.helpme" activated.
[18-03-07 04:50:06.065] [Info] [ContextItem] Context: "question.helpme" is now active.
[18-03-07 04:50:07.918] [Info] [EnglishTokenizer] Normalizing "who"
[18-03-07 04:50:07.918] [Info] [EnglishTokenizer] Normalized Value: "who"
[18-03-07 04:50:07.918] [Info] [OscovaBot] Evaluating a total of "4" intents.
[18-03-07 04:50:07.918] [Info] [OscovaBot] Evaluating request message: "who"
[18-03-07 04:50:07.918] [Info] [RecognizerCollection] Extracting entities for request: 'who'
[18-03-07 04:50:07.918] [Info] [RecognizerCollection] Extracted '0' entities.
[18-03-07 04:50:07.918] [Info] [OscovaBot] Evaluation completed. Total processing time: '0' ms
[18-03-07 04:50:07.933] [Info] [EvaluationResult] Invoking intent: 'General.helpYes'

User avatar
Leslie
Lead Software Architect
Posts: 353
Joined: Fri Sep 14, 2012 12:20 pm
Contact: Website

Re: exact expression match fails with score of -1

Thu Mar 08, 2018 2:11 pm

For the input sequence Who, Who, Who, X and Who the response is as expected. Activation of contexts when X is specified is not disabling any intent.


CSharp Code

private static void Main()
{
	OscovaBot.Logger.LogReceived += (sender, args) =>
	{
		File.AppendAllText("BOT.log", args.Log.ToString() + "\n");
	};

	var bot = new OscovaBot();
	var document = XDocument.Load("siml-document.siml");
	bot.Import(document);

	bot.Trainer.StartTraining();

	bot.MainUser.ResponseReceived += (sender, eventArgs) =>
	{
		Console.WriteLine($"Response: {eventArgs.Response.Text}");
	};

	while (true)
	{
		Console.WriteLine("Type your message");
		var message = Console.ReadLine();
		var evaluationResult = bot.Evaluate(message);
		Console.WriteLine($"Suggested Intent Score: {evaluationResult.SuggestedIntent.Score}");
		evaluationResult.Invoke();
	}
}

The generated Bot log.


[18-03-08 02:09:30.108] [Info] [OscovaBot] Initializing Bot...
[18-03-08 02:09:30.162] [Info] [OscovaBot] Created new user with ID: '8234214d-ce99-4d71-a52c-62f3e673af2e'
[18-03-08 02:09:30.270] [Info] Training started.
[18-03-08 02:09:30.276] [Info] Training Dialogs...
[18-03-08 02:09:30.299] [Info] [Trainer] Training for Dialog: 'General'
[18-03-08 02:09:30.319] [Info] [EnglishTokenizer] Normalizing "who"
[18-03-08 02:09:30.326] [Info] [EnglishTokenizer] Normalized Value: "who"
[18-03-08 02:09:30.334] [Info] [Trainer] Successfully added intent: 'General.who'
[18-03-08 02:09:30.370] [Info] [EnglishTokenizer] Normalizing "x"
[18-03-08 02:09:30.386] [Info] [EnglishTokenizer] Normalized Value: "x"
[18-03-08 02:09:30.391] [Info] [Trainer] Successfully added intent: 'General.x'
[18-03-08 02:09:30.460] [Info] [EnglishTokenizer] Normalizing "UAXKIJ1FUXLYP7O02E1GFE5ZVODIF375X477"
[18-03-08 02:09:30.471] [Info] [EnglishTokenizer] Normalized Value: "UAXKIJ1FUXLYP7O02E1GFE5ZVODIF375X477"
[18-03-08 02:09:30.477] [Info] [EnglishTokenizer] Normalizing "how"
[18-03-08 02:09:30.493] [Info] [EnglishTokenizer] Normalized Value: "how"
[18-03-08 02:09:30.507] [Info] [EnglishTokenizer] Normalizing "what do you"
[18-03-08 02:09:30.565] [Info] [EnglishTokenizer] Normalized Value: "what do you"
[18-03-08 02:09:30.574] [Info] [EnglishTokenizer] Normalizing "with what"
[18-03-08 02:09:30.583] [Info] [EnglishTokenizer] Normalized Value: "with what"
[18-03-08 02:09:30.592] [Info] [Trainer] Successfully added intent: 'General.helpYes'
[18-03-08 02:09:30.601] [Info] Default expression not found adding "DefaultDialog"
[18-03-08 02:09:30.633] [Info] [Trainer] Training for Dialog: 'DefaultDialog'
[18-03-08 02:09:30.639] [Info] [EnglishTokenizer] Normalizing ""
[18-03-08 02:09:30.646] [Info] [EnglishTokenizer] Normalized Value: ""
[18-03-08 02:09:30.654] [Info] [Trainer] Successfully added intent: 'DefaultDialog.Default'
[18-03-08 02:09:30.664] [Info] Loading all registered entity recognizers.
[18-03-08 02:09:32.174] [Info] Training finished.
[18-03-08 02:09:33.621] [Info] [EnglishTokenizer] Normalizing "who"
[18-03-08 02:09:33.626] [Info] [EnglishTokenizer] Normalized Value: "who"
[18-03-08 02:09:33.649] [Info] [OscovaBot] Evaluating a total of "4" intents.
[18-03-08 02:09:33.656] [Info] [OscovaBot] Evaluating request message: "who"
[18-03-08 02:09:33.666] [Info] [RecognizerCollection] Extracting entities for request: 'who'
[18-03-08 02:09:33.752] [Info] [RecognizerCollection] Extracted '0' entities.
[18-03-08 02:09:33.781] [Info] [OscovaBot] Evaluation completed. Total processing time: '120' ms
[18-03-08 02:09:33.790] [Info] [EvaluationResult] Invoking intent: 'General.who'
[18-03-08 02:09:34.858] [Info] [EnglishTokenizer] Normalizing "who"
[18-03-08 02:09:34.867] [Info] [EnglishTokenizer] Normalized Value: "who"
[18-03-08 02:09:34.875] [Info] [OscovaBot] Evaluating a total of "4" intents.
[18-03-08 02:09:34.889] [Info] [OscovaBot] Evaluating request message: "who"
[18-03-08 02:09:34.896] [Info] [RecognizerCollection] Extracting entities for request: 'who'
[18-03-08 02:09:34.902] [Info] [RecognizerCollection] Extracted '0' entities.
[18-03-08 02:09:34.907] [Info] [OscovaBot] Evaluation completed. Total processing time: '10' ms
[18-03-08 02:09:34.917] [Info] [EvaluationResult] Invoking intent: 'General.who'
[18-03-08 02:09:35.863] [Info] [EnglishTokenizer] Normalizing "who"
[18-03-08 02:09:35.868] [Info] [EnglishTokenizer] Normalized Value: "who"
[18-03-08 02:09:35.872] [Info] [OscovaBot] Evaluating a total of "4" intents.
[18-03-08 02:09:35.877] [Info] [OscovaBot] Evaluating request message: "who"
[18-03-08 02:09:35.890] [Info] [RecognizerCollection] Extracting entities for request: 'who'
[18-03-08 02:09:35.964] [Info] [RecognizerCollection] Extracted '0' entities.
[18-03-08 02:09:35.969] [Info] [OscovaBot] Evaluation completed. Total processing time: '79' ms
[18-03-08 02:09:36.072] [Info] [EvaluationResult] Invoking intent: 'General.who'
[18-03-08 02:09:36.995] [Info] [EnglishTokenizer] Normalizing "x"
[18-03-08 02:09:37.008] [Info] [EnglishTokenizer] Normalized Value: "x"
[18-03-08 02:09:37.018] [Info] [OscovaBot] Evaluating a total of "4" intents.
[18-03-08 02:09:37.025] [Info] [OscovaBot] Evaluating request message: "x"
[18-03-08 02:09:37.037] [Info] [RecognizerCollection] Extracting entities for request: 'x'
[18-03-08 02:09:37.051] [Info] [RecognizerCollection] Extracted '0' entities.
[18-03-08 02:09:37.057] [Info] [OscovaBot] Evaluation completed. Total processing time: '19' ms
[18-03-08 02:09:37.070] [Info] [EvaluationResult] Invoking intent: 'General.x'
[18-03-08 02:09:37.089] [Info] [Context] Context item "scene.intro" activated.
[18-03-08 02:09:37.098] [Info] [ContextItem] Context: "scene.intro" is now active.
[18-03-08 02:09:37.110] [Info] [Context] Context item "question.helpme" activated.
[18-03-08 02:09:37.122] [Info] [ContextItem] Context: "question.helpme" is now active.
[18-03-08 02:09:38.229] [Info] [EnglishTokenizer] Normalizing "who"
[18-03-08 02:09:38.237] [Info] [EnglishTokenizer] Normalized Value: "who"
[18-03-08 02:09:38.242] [Info] [OscovaBot] Evaluating a total of "4" intents.
[18-03-08 02:09:38.250] [Info] [OscovaBot] Evaluating request message: "who"
[18-03-08 02:09:38.255] [Info] [RecognizerCollection] Extracting entities for request: 'who'
[18-03-08 02:09:38.261] [Info] [RecognizerCollection] Extracted '0' entities.
[18-03-08 02:09:38.272] [Info] [OscovaBot] Evaluation completed. Total processing time: '17' ms
[18-03-08 02:09:38.279] [Info] [EvaluationResult] Invoking intent: 'General.who'



Is there a chance that some additional configuration is causing this behavior? Have you changed any default value in OscovaBot.Configuration?

rich kopelow
V.I.P
Posts: 47
Joined: Thu Aug 25, 2016 7:45 am

Re: exact expression match fails with score of -1

Wed Mar 14, 2018 8:43 am

No, I haven't changed the default settings. Does this work for you in studio? (the public version with framework 2.5.7.0)
What would generate a score of -1? This seems odd, as if it is an "absolute" rejection - like we're deliberately ignoring this option.

Thanks,
Rich.

User avatar
Leslie
Lead Software Architect
Posts: 353
Joined: Fri Sep 14, 2012 12:20 pm
Contact: Website

Re: exact expression match fails with score of -1

Wed Mar 14, 2018 9:22 pm

I was indeed able to reproduce this in Bot Studio, that runs 2.6.0 version of Syn.Bot framework, but it seems to be working just fine in a Console application that references Syn.Bot 2.9.4.

Although I understand that your actual SIML code would be slightly more complex, could you firstly, confirm that the same code that we've tested the Console application isn't working in a newly created Console application, that runs on Syn.Bot 2.9.4, on your side. Pardon me for requesting this but I really need to know why I am not able to reproduce the behavior.

Secondly, if the behavior is still the same in your newly created console app then could you try explicitly clearing the OscovaBot.Language.StopWords collection and driving the bot through the same sequence of inputs.

Lastly, if and only if you've got the time, you can also share a test SIML file and a simple Test Application as ZIP file to Support@Syn.co.in with the subject title Forum #F57T1896 so the team can look into the case and get back to you asap.

rich kopelow
V.I.P
Posts: 47
Joined: Thu Aug 25, 2016 7:45 am

Re: exact expression match fails with score of -1

Thu Mar 15, 2018 8:35 pm

The console app works as it should... so this is clearly a framework issue. I was seeing what appeared to be the same behavior in my app (running an up to date framework) but will need to research that more.

Is there a way to update the framework in studio (other than you releasing a new install)?

Thanks,
Rich.


Console Log:
Type your message
who
Suggested Intent Score: 1
Response: You said WHO.
Type your message
who
Suggested Intent Score: 1
Response: You said WHO.
Type your message
who
Suggested Intent Score: 1
Response: You said WHO.
Type your message
x
Suggested Intent Score: 1
Response: You said X.
Type your message
who
Suggested Intent Score: 1
Response: You said WHO.
Type your message

User avatar
Sreejith
Help & Support
Posts: 20
Joined: Sat Apr 06, 2013 6:16 pm

Re: exact expression match fails with score of -1

Mon Mar 19, 2018 2:46 am

Syn bot studio is fully dependent on the version of Syn Bot framework...so you cant change the framework/library...but we release a new version in a few weeks
If your experiment needs statistics, you ought to have done a better experiment - Ernest Rutherford

Return to “General Discussion”

Who is online

Users browsing this forum: No registered users and 1 guest