Shell Scripting

Monday, January 12, 2015

How to configure Weblogic with Iplanet

1. Login into iplanet console http://hostname:port/
2. Click on Configuration and Click on new button
3. In the new window, Provide details as mention below
4. Name: Name of the configuration, we can give our desired name ( ex: krishna-secure)
   Server Name: Name of the instance (Eg: https-krishna-secure)
   Document Root: Root folder for the instance (Eg: /apps/Docroot/krishna-secure)
   Server user : username of the server
5. Then click om Next button. Provide port number and IP address as * then click on next button
6. In next step, it will ask details for enabling java and regarding CGI and others. leave the default configuration as it is. No need to change anything.
7. In the Next screen, Select the available Node “krishna.mycomp.com”  and Click on Add then  Click on next
8. In the next screen we will see all the details we have entered previously for review and then click on finish button and then close button.
9. Now in the admin console, Click on Configurations tab. You can see the newly created instance.
10. Select the newly created instance and click on Deploy.
11. After clicking on deploy button you will see a popup window and click on deploy button in popup window.
12. After click on deploy button in popup window , In next step/window click on close button.
13. Now, select the newly created instance and click on Start Running.
14. To avoid the hostname conflict issues in instance configuration while starting instance, Update the hostname of server with existed entries in HTTP listener and Virtual Servers configuration.

Now we need to configure SSL and Weblogic proxy settings at Iplanet end.

1) Go to the newly created instance config directory
/apps/instances/https-krishna-secure/config
2) Open magnus.conf file add below lines and save the file
Init fn="load-modules" shlib="/apps/iplanet/lib/mod_wl.so"
Init fn="load-modules" shlib="libj2eeplugin.so" shlib_flags="(global|now)"
KernelThreads off
3) Open obj.conf, add the context path details of weblogic application like below.
<Object name="weblogic" ppath="*/krishnaapp/*">
Service fn="wl_proxy" WebLogicCluster="hostname1:port,hostname2:port,hostname3:port" PathTrim="/krishnaapp" PathPrepend="/Krishna_APPWL" WLProxySSL="ON" SecureProxy="ON" RequireSSLHostMatch="false" TrustedCAFile="/app/certs/hostnametrustedcas.pem" Debug="ERR" WLLogFile="/app/wlpluginlogs/wlproxy-krishna.log" WLTempDir="/app/wlpluginlogs" DebugConfigInfo="OFF"
</Object>

4) After changing the obj.conf and magnus.conf file you can see the in admin console as “Instance Configuration Modified”. To get reflect these change, deploy the changes on instance.
5) Generate the Iplanet certs with extension of .db, copy the same in config folder.
6)Go to admin console deploy the changes. Click on instance in admin console. Click on “HTTP Listeners”. Click on “http-listener-1”. Then click on SSL. Then click on “set passwords” button in next step/window. After clicking on “Set Passwords ” button, Please provide password and then click ok and close.
7) After clicking on Close button, Select SSL “Enabled” and “Server-cert” from dropdown. Click on Apply and close button.
8) Select “Deployment Pending” as top of screen .Select “Pull and deploy configuration from ” and click on ok.
9. Add the below parameters in server.xml to bypass starting of iplanet instance.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<pkcs11>
  <token>
    <name>internal</name>
    <pin>password</pin>
  </token>
</pkcs11>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
10. Now start the instance.

Sunday, January 11, 2015

Core dump is getting generated in weblogic managed servers machines of 10.3 domain

We have a weblogic domain in which we have 3 weblogic servers configured in horizontal clustering mode. From last few months Core dumps are getting generated(by throwing below error) @ different times on 3 machines where weblogic managed servers are resided. and the process getting killed after core dump getting generated and node manager is starting the killed process. All this process happening with in 5 minutes.
======================================
# An unexpected error has been detected by Java Runtime Environment:
#  SIGSEGV (0xb) at pc=0xfee15cf0, pid=*****, tid=40
# Java VM: Java HotSpot(TM) Server VM (10.0-b19 mixed mode solaris-sparc)
# Problematic frame:
# V  [libjvm.so+0x615cf0]
=====================================

By default Coredump files on Solaris are generated in the directory "/var/core/". But we have customized, so that it will get generated in domain home directory.

Then I have started to analyze the core dump using pstack and got the below output for a thread 40.

 ff29e8cc _lwp_kill (6, 0, ff317080, ff27de54, ffffffff, 6) + 8
 ff212950 abort    (2cf80, 1, fee76768, ffba0, ff315518, 0) + 110
 fee6d5c4 __1cCosFabort6Fb_v_ (1, ff1014f4, 1, ff0ea000, 174f4, 17400) + 5c
 fef66c18 __1cHVMErrorOreport_and_die6M_v_ (ff121ff0, 0, 1, ff0b261f, fef73bd2, 17400) + d1c
 fe9a5dc8 JVM_handle_solaris_signal (b, a8b3d8f0, a8b3d638, 8b000, 306400, fee15cf0) + a64
 fe9a7f50 __1cNCompileBrokerZinvoke_compiler_on_method6FpnLCompileTask__v_ (1b93ef0, e8400, 50, 0, fe9a7140, 306400
 fea24e94 __1cNCompileBrokerUcompiler_thread_loop6F_v_ (0, ff120fe0, 306400, 2ff210, 2c800, 2c800) + 65c
 fef16e74 __1cKJavaThreadRthread_main_inner6M_v_ (306400, 3068c0, 28, f, ff0ea000, 0) + 48
 --------------------------

Didn't understand exactly generated core using pstack command, So I have tried with "pflags" command, got the below output

 core 'core' of 29912:   /app/rsf/bea10/jdk160_05/bin/java -Dweblogic.Name=WLSVRname -Dbea.home data model = _ILP32  flags = MSACCT|MSFORK
 /40:   flags = DETACH
        sigmask = 0xfffffeff,0x0000ffff  cursig = SIGABRT
  /42:   flags = DETACH|STOPPED  pollsys(0x4,0x0,0xa8a1f948,0x0)
        why = PR_SUSPENDED  sigmask = 0x00000004,0x00000000

Now I got to know that thread 40 has received the SIGNAL "SIGABRT" (The SIGABRT signal is sent to a process to tell it to abort, i.e. to terminate. The signal is usually initiated by the process itself when it calls abort function, but it can be sent to the process from outside as well as any other signal.) I am not sure why SIGABRT was generated in Thread 40. Is it because of the code which I have mentioned above(thread 40) where analyzed pstack command, I am not sure.
----------------------------------------------------------------

As I didn't find the exact reason with pflags, I used another command "pmap". The "pmap" command display information about the address space of a process, in this case the program that causes the coredump.
Using pmap command, getting the below output, still I am not able to find the RCA.
A76C0000      16K r-x--  /app/rsf/bea10/wlserver_10.3/server/native/solaris/sparc/libnodemanager.so
A76D2000       8K rwx--  /app/rsf/bea10/wlserver_10.3/server/native/solaris/sparc/libnodemanager.so
A7780000       8K r-x--  /app/rsf/bea10/wlserver_10.3/server/native/solaris/sparc/libwlfileio2.so

Still working on the same, Will post soon the root cause of the issue.