Correlation? What’s that?
If you think correlation has something to do with the fit of data points to a function curve on a graph, and the word has no meaning to you in the context of LoadRunner then this document is for you. It explains what correlation in LoadRunner is, why you have to do it, how to do it, and what to do when it goes wrong. If this is the first time that you have used LoadRunner, or if you have been using it a little but are not a guru, then read on.
Introduction
LoadRunner when recording a script simply listens to the client (browser) talking to the server (web server) and writes it all down. The complete transcript of everything that was said, the dates/time, content, requests and replies can be found in the Recording Log. (View-> Output Window-> Recording Log) The script is sort of an easier to read version of this. The main difference is that the script only contains the client’s communication.
If you imagine that LoadRunner is an impersonator pretending to be the client (browser) the script is LoadRunner’s note that tell it what to say to the server to successfully fool it. We want the server to believe that LoadRunner is a real client, and so send it the information requested.
This script has the hard coded information of the original conversation (Browser session) that occurred between the client and server. This hard coded information may not be enough to fool the server during replay however. It may have to be correlated.
What is correlation?
Correlation is where the script is modified so that some of the hard coded values in the script are no longer hard coded. Rather than have LoadRunner send the original value to the server, we may need to send different values.
For example, the original recorded script may have included the server sending the client a session identification number. Something to identify the client during that particular session. This session ID was hard coded into the script during recording.
During replay, the server will send LoadRunner a new session ID. We need to capture this value, and incorporate it into the script so we can send it back to the server to correctly identify ourselves for this new session. If we leave the script unmodified, we will send the old hard coded session ID to the server. The server will look at it and think it invalid, or unknown, and so will not send us the pages we have requested. LoadRunner will not have successfully fooled the server into believing it is a client.
Correlation is the capturing of dynamic values passed from the server to the client and back. We save this captured value into a LoadRunner parameter, and then use this parameter in the script in place of the original value. During replay, LoadRunner will now listen to what the server sends to it, and when it makes requests of the server, send this new, valid value back to the server. Thus fooling the server into believing it is talking to a real client.
Why do I have to correlate?
If you try to replay a script without correlating first, then most likely the script will fail. The requests it sends to the server will not be replied to. Either the session ID is invalid, so the server won’t allow you into the site, or it won’t allow you to create new records because they are the same as existing ones, or the server won’t understand your request because it isn’t what it is expecting.
Any value which changes every time you connect to the server is a candidate for correlation. A correlated script will send the server the information it is looking for, and so allow the script to replay. This will allow many Vusers to replay the script many times, and so place load on your server.
What errors mean I have to correlate?
There are no specific errors that are associated with correlation, but there are errors that could be caused because a value hasn’t been correlated. For example, a session ID. If an invalid session ID is sent to a web server, how that server responds depends on the implementation of that server. It might send a page specifically stating the Session ID is invalid and ask you to log in again. It might send an HTTP 404 Page not found error because the requesting user didn’t have permissions for the specified page, and so the server couldn’t find the page.
In general any error message returned from the server after LoadRunner makes a request that complains about permissions can point to a hard coded value that needs to be correlated.
The tools (functions) used to correlate.
In LoadRunner 7.X there four functions that you can use for correlation. A list of them, along with documentation and examples can be found in the on-line documentation. From VuGen, go to Help-> Function reference-> Contents-> Web and Wireless Vuser Functions-> Correlation Functions.
The first two functions are essentially the same, and I will talk about then together. The third function, the web_save_reg_param function differs in implementation, and the parameters it takes, but does the same job, and is used in much the same way. The last function is associated with the first two, and isn’t directly a correlation function, but rather a LoadRunner setting. It will be talked about later in a different section.
Web_create_html_param :-
This is the standard correlation function in LoadRunner 6.X and 7.X. This function takes three parameters.
web_create_html_param ( “Parameter Name”, “Left Boundary”, “Right Boundary” );
Each of these parameters is a pointer to a string. That means that if they are entered as literal text, they need to be enclosed in “quotes“. Each parameter is separated by a comma.
Parameter Name:- This is the name of the parameter, or place holder variable that LoadRunner will save the captured value into. After successfully capturing the value, the parameter name is used in the script in place of the original value. LoadRunner will identify the parameter / placeholder, and substitute the captured value for the placeholder during replay. This name should have no spaces, but apart from that limitation, it is entirely up to you what name you give.
Left Boundary:- This is where we tell LoadRunner how to find the dynamic value that we are looking for. In the Left Boundary we specify the text that will appear to the left of the changing value.
Right Boundary:- This is where we tell LoadRunner how to identify the end of the dynamic value we are looking for. Here we place the text that will appear after the value we are looking for.
web_create_html_param_ex
web_create_html_param_ex ( “Parameter Name”, “Left Boundary”, “Right Boundary”, “Instance”);
This function is the same as the web_create_html_param function, except it doesn’t look for the first instance of the boundaries, but rather the nth instance of those boundaries. The first three parameters are the same, name, left, right, the last parameter is a pointer to a string, so it must be enclosed in double quotes. It is the number of the occurrence. If you place the number one here (i.e. “1”) then the function behaves exactly as the web_create_html_param function. It looks for the first occurrence. If you put the number three here (i.e. “3”) it will look for the 3rd occurrence of the left and right boundaries, and place what appears in between into the parameter.
web_reg_save_param
web_reg_save_param ( “Parameter Name” , , LAST );
The first thing to note about this function as different from the web_create_html_param functions is that the number of parameters it takes can vary. The first one is still the name, but after that there are different attributes that can be used. These attributes can appear in any order because they contain within them what they are. For example, the attribute to identify the left boundary is “LB= followed by the text of the left boundary. I won’t be talking about all of the options for this function, they are listed in the documentation. Please have a look at it. (Help-> Function Reference)
The first parameter is the name, then the list of attributes or parameters, then the keyword LAST. This identifies the end of the function. The keyword is not enclosed in quotes, all parameters are. All parameters and keywords are separated by commas.
Identifying values to correlate.
So we have the tools, and we know why we need to use them, but how do we know what to use them on? What values in the script need to be correlated. The simplest answer is, “Any value that changes between sessions required for the script to replay.”
A hypothetical example. We are logging onto a web site. When we send the server our user name and password, it replies to us with a session ID that is good for that session. The session ID needs to be correlated for replay. We need to capture this value during replay to use in the script in place of the hard coded value.
To identify values to correlate, record the script, and save it. Open a new script, and record the same actions, and business process again. As much as possible, during recording, enter the same values in both scripts. For example, user ID, password, and fields and edit selections. Save the second script, and then run it with Extended log. (Vuser-> Run time settings-> Log-> Extended log. Check all three options)
Go to tools-> Compare with Vuser, and choose the first recorded script. WinDiff will open and display the two scripts side by side. Lines with differences in them will be highlighted in Yellow. Differences within the line, will be in red.
If WinDiff gives an error here, dismiss the error, WinDiff will be minimized in the task bar. Right click on it. Choose restore. Then go to File-> Select Files/Directories, and manually select the action sections for the two scripts.
Differences like "lr_think_time" can be ignored. They are load runner pacing functions, and don’t represent data sent to the server.
Locate the first difference and take note of it and search the script open in VuGen for that difference. That is the original value hard coded into the script that was different in the second script. Highlight it, and copy it.
Go to the Recording log, and place your cursor at the top. Hit Control F (Ctrl+F) to do a search and paste in the original value. We are looking for the first occurrence of this value in the recording log. If you don’t find the value in the recording log, check you are looking in the right scripts recording log. Remember you have two almost identical scripts here.
If you find the value, scroll up in the log, and make sure the value was sent as part of a response from the server. The first header you come across while looking up the script should be preference with a receiving response. This indicates that the value was sent by the server to the client. If the value first appears as part of a sending request, then the value originated on the client side, and doesn’t need to be correlated, but rather parameterized. That is a different topic all together. The response will have a comment before it that looks like this
*** [tid=640 Action1 2] Receiving response (10/8/200112:10:26)
So, we have a value that is different between subsequent recordings, it was sent from the server to the client. This value most likely needs to be correlated. If the value you were looking for doesn’t meet these criteria,
-
Different between recordings
-
Originated first on the server and sent to the client
It probably doesn’t need to be correlated.
Now that we know Why and What, How do we parameterize?
Step 1.
After confirming that the first occurrence was part of a received response from the server, we need to now figure out where to place the web_create_html_param( ) function. The web_create_html_param statement needs to go immediately before the request that fetched the dynamic value from the server. In order to find this request or URL in the script, we need to replay the script once with extended log and all the three options (In Vuser->Runtime Settings->Log) turned on.
In the recording log, pick up the text that is before the dynamic value. This text should remain constant no matter how many times you replay the script and highlight it and copy it. This is the text that will identify to LoadRunner where to find the start of the value we are capturing.
Now, go to the execution log and search for the text that you just copied from the recording log.
You should see a corresponding Action1.c() at the beginning of that line with a number in the brackets. That is the number of the line the script where you need to put the web_create_html_param( ) function. The function should go right above that line in the script.
So, add a couple of blank lines to your script before the function at that line, and then type in web_create_html_param(“UserSession” but give it a name that means more to you than UserSession.
Step 2.
Go back to the execution log and highlight the text to the left of the dynamic value and copy it. This should be some of the same text we searched for in the Execution log.
The amount of text you highlight should be sufficient so that it is unique in this reply from the server. I would suggest copying as much as possible without copying any special characters. These show in the execution log as black squares, and the actual character they represent is uncertain. After selecting a boundary, go to the top of the Servers reply, and hit Ctrl+F and do a search for that boundary. You want to make certain what you have selected is the first occurrence in the servers reply. If it isn’t select more text to make it unique, or consider using the web_create_html_param_ex function or the ORD parameter or the web_reg_save_param function.
Once you have finalized the static text that represents the left boundary, copy it into the web_create_html_param (or web_reg_save_param) statement. If it contains any carriage returns, place it all on one line. If there are any “ in the text, place the escape character before it so LoadRunner doesn’t incorrectly think it is the end of the parameter, but rather a character to search for. For example, if the Left boundary was 'input type=hidden name=userSession value=' without the single quotes and we are using the web_create_html_param statement, then the function we have so far would be
Web_create_html_param(“UserSession”, “input type=hidden name=userSession value=”,
Step 3.
We are now going to tell LoadRunner how to identify the end of the value we are trying to capture. That is the right boundary of what we are looking for. Again look in the execution log and copy the static text that appears to the right of the dynamic value we are looking for. For example, lets say the execution log contained the following
… userSession value=75893.0884568651DQADHfApHDHfcDtccpfAttcf>…
Then the example so far to save the number into the parameter UserSession would be
web_create_html_param(“UserSession”, “input type=hidden name=userSession value=”, “>”);
In choosing a right boundary, make sure you choose enough static text to specify the end of the value. If the boundary you specify appears in the value you are trying to capture, then you will not capture the whole value.
Recap:-
That was a lot of looking through the recording and execution logs and checking of values. Lets just recap what we have done. We have identified a value that we think needs to be correlated. We then identified in the script where to place the statement that would ultimately capture and save the value into a parameter. We then placed the statement, and gave LoadRunner the text strings that appear on either side of the value we are looking for so that it can find it.
The flow of logic for this is, the correlation functions tells LoadRunner what to look for in the next set of replies from the server. LoadRunner makes a request of the server. The server replies. LoadRunner looks thorough the replies for the left and right boundaries. If it finds then, what is in-between is then saved to a parameter of the name specified.
Remember, the parameter can’t have a value till AFTER the next statement is executed. The correlation statement only tells LoadRunner what to look for. It doesn’t assign a value to the parameter. Assignment of a value to the parameter doesn’t happen till after LoadRunner makes a request of the server and looks in the reply. If you have in your script a case where a correlation statement is followed by a function that attempts to use the parameter, the statement is in the wrong place, and the script will fail.
This is always incorrect:-
web_create_html_param(…);
Web_submit (… {Parameter}…);
There needs to be in-between the two, the request of the server that causes it to reply with the value we are trying to capture.
Replacing the hard coded value in the script with the parameter.
Once we have created the parameter, the next step is to replace the hard coded occurrences with the parameter. Look through the script for the original value. Where you find it, delete out the value and replace it with the parameter. Note, only the value we want replaces is deleted. The characters around it remain.
i.e.
Change :
..... .....userSession=75893.0884568651DQADHfApHDHfcDtccpfAttcf&username=test........
.....
To :
.....
.....userSession={UserSession}&username=test........
.....
At this point, you are ready to run the script to test if it works, or if it needs further correlation, or more work on this correlation.
Common errors when correlating.
When LoadRunner fails to find the boundaries for a web_create, it will print a warning message in the execution log like this:-
Warning: No match found for the requested parameter "Name". If the data you want to save exceeds 256 bytes, use web_set_max_html_param_len to increase the parameter size
Firstly, this is a warning not an error. There are times when you might want to use the web_create_html_param function for purposes other that correlation. These require the function to not cause an error, so this is a warning.
Secondly, the advice the warning message gives is good, but I recommend thinking about it first. Was the value you were trying to capture more than 256 characters long? In the above example it was only 20 characters long. Have a look at the recording log and see how long the original value is. Have a look at the second recording made earlier and see how long it was in that script. Turn on the extended log (Run-Time settings-> Log-> Extended log-> All data returned from server) and have a look at how long it is in the execution log. If at any time any of these values was close to being, say 200 characters, then yes, add a web_set_max_html_param_len statement to the start of the script to make the maximum longer than 256 characters. If all occurrences of the script were much shorter that the max parameter length then the problem is either the web_create_html_param is in the wrong place, or that the boundaries are incorrect. Go back and look at the boundaries that you have selected, look at the placement of the web_create_html_param function. Is it immediately before the statement that causes the server to reply with the data you are looking for?
The Parameter length is longer than the current maximum.
(The web_set_max_html_param_len function)
web_set_max_html_param_len ( “length” );
This statement tells LoadRunner to look for larger matches for the left and right parameter. When it finds the left boundary, it will look ahead the max parameter length for the right boundary. This setting is script wide, and takes effect from when it is executed. It only needs appear in the script once. Having LoadRunner look for longer matches uses up more memory, and CPU to search through the returned text from the server. For this reason, don’t set it too high, or you will be making your script less scalable. That is you will reduce the number of Vusers that can run it on a given machine. Try to have the maximum parameter length no more than 100 characters greater that what you are expecting.
Special cases for the boundaries:-
There are some special characters and cases when specifying the boundaries. Double quotes should be preceded by a so LoadRunner recognizes then as part of the string to look for. If your text includes any carriage returns, that are part of the http, and not just part of the wrap around in the recording log, these need to be specified as a character set. If the character is part of the text, it too needs to be preceded by a to indicate it is a literal.
Recording log boundary parameters (left, right)
Value=”57685” “Value=”” “””
Value_”item”value’7875’ “Value_”item”value’ “’”
Value= “Value= ”” “””
“7898756”
Debug help
Sometimes you want to print out the value that was assigned to a parameter. To do this, use the lr_eval_string function, and the lr_output_message function. For example, to print the value of the parameter to the execution log.
lr_output_message(“ Value Captured = %s", lr_eval_string("{Name}"));
If you find that the value being substituted is too long, too short, or completely wrong, printing out the value will help identify the changes you need to make to the correlation function. If you have extra characters at the start of the value, you need to add them to the end of the left boundary, if you have extra characters at the end of the parameter value, you need to add them to the start of the right boundary. If you are getting the wrong value all together, do searches in the recording log for the left boundary, and make sure that you have a unique boundary, and that LoadRunner isn’t picking up an earlier occurrence. You can then use the web_create_html_param_ex function, or add to the boundaries to make them unique.
