Sunday, March 11, 2012

Configure Extended Logging In HTTP Access Logs For WebLogic Application Server

Hello There,

Most of us are aware that Access Logs Prints Application Access logs Information as below.

192.168.238.1 - - [01/Mar/2012:15:52:45 +0530] "GET /HelloWorld/welcome.jsp HTTP/1.1" 200 895
192.168.238.1 - - [01/Mar/2012:15:52:46 +0530] "GET /HelloWorld/GoodMorning.jsp HTTP/1.1" 200 2963


Which Specifically uses common logging format, syntax/format of the common logging format is as below.

host RFC931 auth_user [day/month/year:hour:minute:second UTC_offset] "request" status bytes.

The Above information is very limited in case if you need to know lot from it, each parameter above are specific to below information.

host
Either the DNS name or the IP number of the remote client
RFC931
Any information returned by IDENTD for the remote client; WebLogic Server does not support user identification
auth_user
If the remote client user sent a userid for authentication, the user name; otherwise "-"
day/month/year:hour:minute:second UTC_offset
Day, calendar month, year and time of day (24-hour format) with the hours difference between local time and GMT, enclosed in square brackets
"request"
First line of the HTTP request submitted by the remote client enclosed in double quotes
status
HTTP status code returned by the server, if available; otherwise "-"
bytes
Number of bytes listed as the content-length in the HTTP header, not including the HTTP header, if known; otherwise "-"

with the above information the access log is only used for to check whether application is being accessed or not over the server and the time and date of access, finally it will provide the status of the access.

Whereas the Extended Logging Format in WebLogic Server uses the W3C Specifications for the log format, for more information, please refer to http://www.w3.org/TR/WD-logfile.html

According to this logging format we can specify the customized fields for the HTTP Log file as in below link :

http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/676400bc-8969-4aa7-851a-9319490a9bbb.mspx?mfr=true

And : http://docs.oracle.com/cd/E13222_01/wls/docs91/config_wls/web_server.html#elf

Follow the below steps to configure the Extended Logging format for HTTP Logs.

 Step 1 : Install the WebLogic Server Software, Create a Domain, Start the Admin Server and Deploy a Sample Application to keep everything ready to test the Extended Logging format.

Step 2 : Click on Admin Server and Go to Logging Tab in the console

Step 3 : Click on HTTP to modify the HTTP logging format for the Application Access.

Step 4 : Scroll down and click on Advanced

Step 5 : Change the Format to Extended

Step 6 : And Specify the fields for Extended Logging in "Extended Logging Format Fields" as you would like to see the information from : http://docs.oracle.com/cd/E13222_01/wls/docs91/config_wls/web_server.html#elf




In my example domain I have specified the following fields.


date time cs-method cs-uri sc-status cs-username cs(User-Agent)  s-sitename s-ip s-port time-taken cs(Cookie) sc-substatus cs-host cs-version


So as per the above link it will provide us the information for : 

date : Date at which transaction completed, field has type <date>, as defined in the W3C specification. 

time : Time at which transaction completed, field has type <time>, as defined in the W3C specification.

cs-method : The request method, for example GET or POST. This field has type <name>, as defined in the W3C specification. 

cs-uri: The full requested URI. This field has type <uri>, as defined in the W3C specification. 

sc-status: Status code of the response, for example (404) indicating a "File not found" status. This field has type <integer>, as defined in the W3C specification.

cs-username: The name of the authenticated user who accessed your server. Anonymous users are indicated by a hyphen.

cs(User-Agent) : The browser type that the client used.

s-sitename: The Internet service name and instance number that was running on the client.

s-ip: The IP address of the server. 

s-port: The Port number of the server.

time-taken: Time taken for transaction to complete in seconds, field has type <fixed>, as defined in the W3C specification. 


cs(Cookie)
The content of the cookie sent or received, if any.

sc-substatus: The substatus error code.

cs-host: The host header name, if any.

cs-version: The protocol version —HTTP or FTP —that the client used.

After this has been specified, save the same and activate the changes.

Step 7: Restart the server to activate the changes.

Step 8 : Access the Application deployed over the server.

Step 9: View the Access logs created 

You will find following information sample in your access logs : 

#Version:    1.0
#Fields:    date time cs-method cs-uri sc-status cs-username cs(User-Agent)  s-sitename s-ip s-port time-taken cs(Cookie) sc-substatus cs-host cs-version
#Software:    WebLogic
#Start-Date:    2012-03-01    16:02:41
2012-03-01    10:33:22    GET    /helloworld/welcome.jsp    200    -    "Mozilla/5.0 (Windows NT 5.1; rv:10.0.2) Gecko/20100101 Firefox/10.0.2"    -    192.168.238.1:7001    -    0.047    "JSESSIONID=QHclPPNTdXyGRJ5QRQFjC97t2PTnVKQTd2tB1hDVSFbf2pJNsw9t!-500938956"    -    -    -
2012-03-01    10:33:24    GET    /helloworld/hellothere    200    -    "Mozilla/5.0 (Windows NT 5.1; rv:10.0.2) Gecko/20100101 Firefox/10.0.2"    -    192.168.238.1:7001    -    0.015    "JSESSIONID=wZDZPPQSqFY6n25mgNcZGsF2MmVxT05XQZ0vLnxqCltrJcYjt7nw!-1723970954"    -    -    -
2012-03-01    10:33:25    GET    /helloworld/hello.jsp    200    -    "Mozilla/5.0 (Windows NT 5.1; rv:10.0.2) Gecko/20100101 Firefox/10.0.2"    -    192.168.238.1:7001    -    0.031    "JSESSIONID=wZDZPPQSqFY6n25mgNcZGsF2MmVxT05XQZ0vLnxqCltrJcYjt7nw!-1723970954"    -    -    -

You can check the each parameters and check for the log created, and cross check the information you have gathered.

Hope this posting was helpful for you.

Please share your feedback.

Regards
Shailesh Dyade

9 comments:

oligocart said...

It looks like the cs-username field always have an empty value "-" even when there is an authenticated user.

The auth_user is displayed correctly with the common log format however.

Anyone else have this issue?

using WebLogic 11g (10.3.5)

oligocart said...

Just found out this note "Missing Username In Extended Http Logs (Doc ID 1240135.1)" saying that cs-username field is unsupported.

Enhancement Request #10206721 has been filed on this issue.

Jason Hunt said...
This comment has been removed by the author.
q5p4k0 said...

oligocart: I can't not view that oracle document, but I am little perplexed.

When I visit http://www.w3.org/TR/WD-logfile.html I see no mention of "cs-username". Furthermore, http://docs.oracle.com/cd/E17904_01/web.1111/e13701/web_server.htm#CNFGD207 section " has no mention of cs-username as well.

So, I am not following. Was cs-username removed from w3c?

Mahmoud Lamriben said...

I also visited the link posted (http://www.w3.org/TR/WD-logfile-960221.html) but didn't find anything related to the use of the cs(xxx) format. Author, can you correct?
thanks.

ramana reddy said...

it's help ful to understand the access log.thanks you

Anonymous said...

Very useful thanks!

George Donaghy said...

Can the access type be logged. I mean I'd like to see if user was using http or https? We support both on our site and please don't ask why as it's a long story.

Thanks

Ankuj uniyal said...

Why there is tabes. Also what if i want to get date and time together like in common format.
Actually I just wan to add response time. for logging the records.