December 2, 2011 ahydle

Solaris 11 Samba / ZFS Configuration

Sun x4500 in the boxThe following is a summary detailing step-by-step how to setup Solaris 11 as an active directory integrated file server using Samba and ZFS.
Most of this configuration comes from a tested configuration on Solaris 10 as well as Linux, but with the release of Solaris 11 and some changes in ZFS my previous instructions needed to be updated. The configuration has currently only been built and tested in a virtualized environment and though I do try and test as vigorously as possible please make sure to test before putting into production.

I make no promises/warranties regarding any of this information. The following information contains my opinions from my own experience with Solaris/Samba/ZFS. The following documentation is provided for educational purposes only and should be used at your own risk.

Install Samba

By default Solaris 11 does not come with all the installed packages of its predecessors and instead it uses the new Image Packaging System (IPS). You will need to install the packages from the Solaris catalog. To do this, you can use the pfexec command as a normal user (pretty much like sudo) or run the install as root.

As root:

pkg install samba

As a normal user, using pfexec:

pfexec pkg install samba

Set maximum groups to 1024

As of Solaris 10 08/11 and Solaris 11, the Solaris kernel allows a user to be a member of 1024 groups. If you do not set this the default maximum will be 16 and if a user is over this amount he or she will be denied access to all share points. Unfortunately nesting groups does not cut down on this number since the system will still see the user as a member of each nested group. CAUTION – This will create NFS server incompatibilities.

Edit /etc/system and make sure there are no spaces after the = sign and everything is spelled correctly.

set ngroups_max=1024

Note -If you make a mistake and your system fails to boot just boot interactively and point to /dev/null when it asks for the location of your /etc/system file.

Setup NTP to stay in sync with your domain controllers

Create the ntp file

cd /etc/inet

cp ntp.client ntp.conf

Edit the file taking out the multicast option and setting up your DC as the server to sync with.

# ident "@(#)ntp.client 1.3     00/07/17 SMI"

#
# /etc/inet/ntp.client
#
# An example file that could be copied over to /etc/inet/ntp.conf; it
# provides a configuration for a host that passively waits for a server
# to provide NTP packets on the ntp multicast net.
#

server <INSERT YOUR PDC EMULATOR AD DC IP ADDRESS>

#multicastclient 224.0.1.1

Start the xntpd service

svcadm enable network/ntp

Update your hosts file and enter mappings for all domain controllers as well as server

Edit the hosts file

vi /etc/inet/hosts

Setup host and alias entries for both of your DC’s and for your system. The host entries for your DC’s may not be completely necessary as long as you have DNS setup properly. I just always do this just in case.

#
# Internet host table
#
::1 fs0 localhost         
127.0.0.1 fs0 localhost loghost

192.168.5.50 fs0 fs0.domain.local

# Network Domain Controllers
192.168.5.10 dc0 dc0.domain.local
192.168.5.11 dc1 dc1.domain.local
192.168.6.10 child-dc0 child-dc0.child.domain.local

Make sure that /etc/resolv.conf contains nameserver entries for each of your Active Directory DNS servers as well as a domain and search field. This file is now auto generated from SMF, if you need to edit it use the appropriate svccfg -s dns/client commands to adjust your name server settings.

domain domain.local
search domain.local
nameserver 192.168.5.10
nameserver 192.168.5.11

Setup Kerberos

NEW WEIRDNESS – Not sure why but as of Solaris 11 I needed to manually create a key tab file from my domain controller otherwise I was getting kerberos errors. This is strange as this is normally all handled in Samba.

To begin, create an active directory user account that we will use to create kerberos tickets (It may also be possible to just use a normal user account but I haven’t tested what happens when the account is deleted). For this example I created a user account named “host-fs0” and set it so that the user cannot change the password and the password never expires. The account should need no special privileges and is just a normal user account.

Next, on the domain controller click Start → All Programs → Accessories and then right click on the command prompt and choose run as Administrator (specific to 2008 servers). Once the command prompt opens use the following command to generate a keytab file.

> cd 
> ktpass -princ host/fs0.domain.local@DOMAIN.LOCAL -mapuser DOMAINhost-fs0 -crypto All -pass <PASSWORD ENTERED HERE> -ptype KRB5_NT_PRINCIPAL -out fs0.keytab

Copy the fs0.keytab file to your Solaris server and place it in the /etc/krb5 directory as krb5.keytab

cp fs0.keytab /etc/krb5/krb5.keytab

Setup your /etc/krb5/krb5.conf with your domain controllers

[libdefaults]
default_realm = DOMAIN.LOCAL
dns_lookup_realm = false
dns_lookup_kdc = false

[realms]
DOMAIN.LOCAL = {
kdc = dc0.domain.local:88
kdc = dc1.domain.local:88
default_domain = domain.local
        }

CHILD.DOMAIN.LOCAL = {
kdc = child-dc0.child.domain.local:88
default_domain = child.domain.local
}

[domain_realm]
        .domain.local = DOMAIN.LOCAL
        domain.local = DOMAIN.LOCAL
        .child.domain.local = CHILD.DOMAIN.LOCAL
        child.domain.local = CHILD.DOMAIN.LOCAL

[logging]
default = FILE:/var/krb5/kdc.log
kdc = FILE:/var/krb5/kdc.log

kdc_rotate = {
# How often to rotate kdc.log. Logs will get rotated no more
# often than the period, and less often if the KDC is not used
# frequently.
period = 1d
# how many versions of kdc.log to keep around (kdc.log.0, kdc.log.1, ...)
versions = 10
}

[appdefaults]
kinit = {
renewable = true
forwardable= true
}

Test your file using kinit. No response is a good response, otherwise you will receive an error

kinit user@DOMAIN.LOCAL

Configure Samba

Create a new file named /etc/samba/smb.conf.master with the following information:

[global]

# Logging
log level = 1
syslog only = no
max log size = 50 
log file = /var/samba/log/%m.log 

realm = domain.local
workgroup = DOMAIN
security = ADS
encrypt passwords = true
unix extensions = no
password server = dc0.domain.local dc1.domain.local
server string = Corporate File Server
wins server = 192.168.5.10 192.168.5.11
domain master = no
map untrusted to domain = Yes

socket options = TCP_NODELAY SO_KEEPALIVE
client schannel = no 
client use spnego = yes
client NTLMv2 auth = yes
unix extensions = No
client signing = Yes
map acl inherit = Yes
interfaces = net*,lo
bind interfaces only = yes

#Currently testing cross-platform oplocks this may need be re-enabled as all my production servers have always had veto'ed files.
#kernel oplocks = yes
#oplocks = yes
#veto oplock files = /*.doc/*.DOC/*.xls/*.XLS/*.ppt/*.PPT/*.pst/*.PST/*.mdb/*.MDB/*.ldb/*.LDB/*.vsd/*.VSD/*.mpp/*.MPP/*.qbw/*.QBW/*.qbb/*.QBB/*.qbI/*.qbl/
*.dxf/*.DXF/*.dwg/*.DWG/*.cdr/*.CDR/*.bak/*.BAK/*.ord/*.xlo/*.igs/*.ipt/*.ipj/*.slp/*.stp/*.opt/*.xli/*.stl/*.cur/*.sjb/*.log/*.LOG/*.sbs/*.iam/*.idv/
*.pcbdoc/*.PcbDoc/*.PCBDOC/

# DFS
# host msdfs = yes
# strict locking
# strict sync

# winbind
winbind separator = +
idmap uid = 11000-19000
idmap gid = 11000-19000
winbind enum users = yes
winbind enum groups = yes
winbind nested groups = yes
allow trusted domains = yes
template homedir = /export/Corporate/Users/%U

printcap name = /dev/null
load printers = no

[Corporate]
   comment = Corporate Share
   path = /export/Corporate
   read only = No
   inherit permissions = Yes
   inherit acls = Yes
   wide links = Yes
   map archive = No
   map readonly = no
   vfs objects = zfsacl
   nfs4:mode = special
   nfs4:acedup = merge
   nfs4:chown = yes
   zfsacl: acesort = dontcare

#[Dfs]
#   path = /mnt/share/dfsroot
#   msdfs root = Yes

In order to optimize the config and take out any comments we will use the testparm command to generate a smb.conf. This also has the added benefit of checking your config for any errors and displaying the errors on the screen.

testparm -s /etc/samba/smb.conf.master > /etc/samba/smb.conf

We are now ready to join this box to the domain. If successful you should see “Joined ‘FS0’ to realm ‘DOMAIN.LOCAL”

net ads join -U Administrator@DOMAIN.LOCAL

Setup PAM and nsswitch

Here we are going to enable the use of winbind through PAM. This will give us the capability of using active directory users and groups when assigning permissions on files and directories. We could also use this for other authentication methods that use PAM such as apache web sites (but that is another discussion).

Enable winbind in pam.conf

cp /etc/pam.conf /etc/pam.conf-OLD
cp /etc/pam.conf-winbind /etc/pam.conf

Enable winbind in nsswitch.conf to set up the search order for all authentication. The system will now look at the local files followed by a check against winbind.

# svccfg -s name-service/switch 
> setprop config/password = "files winbind"
> setprop config/group = "files winbind"

> exit
# svcadm refresh name-service/switch

Make sure the entries were adjusted in nsswitch by:

cat /etc/nsswitch.conf

You should see the following entries in passwd and group:

passwd:     files winbind
group:      files winbind

Testing the configuration and starting services

Before starting anything we should do a reconfiguration reboot to ensure the kernel entries that were entered in the beginning are being seen.

touch /reconfigure
init 6

Before starting winbind we will perform a test so you can see what the nsswitch / pam changes did:

getent passwd
getent group

You should see a list of ONLY the users and groups in your Solaris /etc/passwd and /etc/groups files. So now lets enable services and run the commands again:

svcadm enable winbind
svcadm enable samba

getent passwd
getent group

If everything worked you should see all of your local users and groups plus all the users and groups from Active Directory. Winbind downloads this list from active directory maintaining a local cache in tdb files located /var/samba/locks on your system.

Setup ZFS Storage For Samba Shares

Since my first article much has changed with ZFS permissions and as of the latest releases it looks like I no longer need to specify explicit deny statements and so I have altered the config to reflect the new changes.

The following example illustrates how to setup root departmental shares along with user folders. The root departmental shares each have two groups associated with them:

  1. corporate-<DEPARTMENT NAME> – Departmental group. Members of the department should be put here.
  2. corporate-department-all – Access to ALL department folders (used for VP’s and privileged users).

I use the naming strategy shown above to help keep my groups organized. Notice that corporate is the share name and department is going to be the departmental folder name. This helps make it easy associating shares / folders / sub-folders to groups. You do not need to setup your groups this way but I provide this example to help the ACL‘s make sense.

Before we get started we need to create our ZFS file system. You may have already noticed that I am creating the file system in export. If you already have a file system loaded in export (Which looks like the default in Solaris 11) you will need to backup your existing files/directories, remove your vfstab entry for export and create your zpool with the -f option or you can create your file system elsewhere and redefine the paths in this example. The following zpool shows a smaller system with only six disks. Your configuration will probably be different.

zpool create export raidz2 c0t0d0 c1t0d0 c4t0d0 c6t0d0 c7t0d0 c0t1d0

Now that the pool is created I am going to create my top-level shared folders.

zfs create export/Corporate
zfs create export/Corporate/Users
zfs create export/Corporate/Users/jdoe
zfs create export/Corporate/Sales
zfs create export/Corporate/Accounting

Set ZFS ACL inheritance to control how ACL‘s are inherited between file systems.

zfs set aclmode=passthrough export/Corporate
zfs set aclinherit=passthrough export/Corporate

Ok now we are ready to lay down our ACL‘s on our Corporate folder. By looking at this ACL you should notice I am first deleting any existing ACL‘s (Important when testing ACL‘s), then I am setting trivial permissions for user and group as rwx and telling the user and group permissions to inherit. I am then adding Domain Users to the ACL and allowing them to read the directory and list its contents. I always setup the root directories so users cannot add or remove files. This way Administrators can maintain organization by creating new departmental shares.

chmod A- /export/Corporate
chmod 770 /export/Corporate
chmod A0=owner@:rwxpdDaARWc---:fd-n--:allow /export/Corporate
chmod A1=group@:rwxpdDaARWc---:fd-n--:allow /export/Corporate
chmod A2=everyone@::fdi---:allow /export/Corporate
chmod A+group:domain+domain users:rxcaR:fdn:allow /export/Corporate

Here we will define the permissions on the Sales and Accounting Folders. In this example I am defining the same permissions and inheritance for user and group, giving each rwx while disabling access for everyone. Then I am giving Domain Admins the ability to manage ACL‘s and the departmental groups rwx access to the directories and files created within it. As I am wiping my permissions, I am also resetting the inherited permissions for domain users on the folder.

chmod A- /export/Corporate/Accounting
chmod 770 /export/Corporate/Accounting
chmod A0=owner@:rwxpdDaARWc---:fd-n--:allow /export/Corporate/Accounting
chmod A1=group@:rwxpdDaARWc---:fd-n--:allow /export/Corporate/Accounting
chmod A2=everyone@::fdi---:allow /export/Corporate/Accounting
chmod A+group:domain+domain users:rcaR::allow /export/Corporate/Accounting
chmod A+group:domain+corporate-all:rwpdDxraRAwWc:fd:allow /export/Corporate/Accounting
chmod A+group:domain+corporate-accounting:rwpdDxraRAwWc:fd:allow /export/Corporate/Accounting
chmod A+group:domain+domain admins:rwxpdDaARWcCo:fd:allow /export/Corporate/Accounting 

chmod A- /export/Corporate/Sales
chmod 770 /export/Corporate/Sales
chmod A0=owner@:rwxpdDaARWc---:fd-n--:allow /export/Corporate/Sales
chmod A1=group@:rwxpdDaARWc---:fd-n--:allow /export/Corporate/Sales
chmod A2=everyone@::fdi---:allow /export/Corporate/Sales
chmod A+group:domain+domain users:rcaR::allow /export/Corporate/Sales
chmod A+group:domain+corporate-all:rwpdDxraRAwWc:fd:allow /export/Corporate/Sales
chmod A+group:domain+corporate-sales:rwpdDxraRAwWc:fd:allow /export/Corporate/Sales
chmod A+group:domain+domain admins:rwxpdDaARWcCo:fd:allow /export/Corporate/Sales

Here I create a folder to contain the user folders. The permissions are as such so that the users will be able to see the folders but will not be able to create files or folders in the Users directory.

chmod A- /export/Corporate/Users
chmod 770 /export/Corporate/Users
chmod A0=owner@:rwxpdDaARWc---:fd-n--:allow /export/Corporate/Users
chmod A1=group@:rwxpdDaARWc---:fd-n--:allow /export/Corporate/Users
chmod A2=everyone@::fdi---:allow /export/Corporate/Users
chmod A+group:domain+domain users:rxcaR:fdn:allow /export/Corporate/Users

Now that the root folder is created we can start crating user folders:

zfs create export/Corporate/Users/jdoe
chown corporate+jdoe /export/Corporate/Users/jdoe
chgrp corporate+domain users /export/Corporate/Users/jdoe
chmod A- /export/Corporate/Users/jdoe
chmod A0=owner@:rwxpAWCos:fd:allow /export/Corporate/Users/jdoe
chmod A1=group@::fd:allow /export/Corporate/Users/jdoe
chmod A2=everyone@:aRcs:fd:allow /export/Corporate/Users/jdoe
, , , , , , , ,

Comments (24)

  1. Peter

    Hi,

    With this setup, is it possible to access the samba share from any computer? Or should the computer that needs to access the share be part of the AD domain? So does authentication always happens via Kerberos?

    I want to build a setup that if a user browses to the samba share (e.g. via windows file explorer). That the user can enter in the authentication dialog:
    username: user1@domain.com
    password: userpass
    And that he can continue using the share with his groups assigned in the AD.

    Is this possible?

  2. dragos

    Guru!

    I have 2 questions:

    Is there any interface to manage the new ZFS ACL’s ?

    As I know the new ACL’s are Windows compatible: this means that
    Windows users will be able to set permissions from the Security tab ?

    Setting permissions frmo Windows with POSIX ACL and Samba can
    be hell sometimes ..

  3. Peter

    Hi,

    With this setup, is it possible to access the samba share from any computer? Or should the computer that needs to access the share be part of the AD domain? So does authentication always happens via Kerberos?

    I want to build a setup that if a user browses to the samba share (e.g. via windows file explorer). That the user can enter in the authentication dialog:
    username: user1@domain.com
    password: userpass
    And that he can continue using the share with his groups assigned in the AD.

    Is this possible?

  4. Andrew Hydle

    Hi Peter,

    I just updated the samba config here to include “map untrusted to domain = Yes” with this configuration all non-domain members will be mapped to the default domain without specifying the domain name. I haven’t put much testing into using username@domain.name but I do know that this config will work if you specify domainusername.

    I have also tested this config with Linux/Mac and Windows clients. Hope that answers your questions

    Thanks

    Andrew

  5. Andrew Hydle

    You should be able to manage the ACL’s through windows but I have never had much luck with it. From what I understand the permissions need to be set in a specific order or you will get all kinds of errors viewing it through the windows permissions dialog.

    One thing I can do for you if you like is send you some find/replace scripts that I use when I need to re-write permissions on a zfs file system? I will probably put them up here soon as they are pretty useful but if you want them sooner just send my your email through the site.

    Thanks!

    Andrew

    [quote]Guru!

    I have 2 questions:

    Is there any interface to manage the new ZFS ACL’s ?

    As I know the new ACL’s are Windows compatible: this means that
    Windows users will be able to set permissions from the Security tab ?

    Setting permissions frmo Windows with POSIX ACL and Samba can
    be hell sometimes ..[/quote]

  6. Peter

    Thanks for the reply

    It is stil not working for me :-). But I noticed I am using different packages than you are. I use smb service instead of samba service. I had this from Oracle docs http://docs.oracle.com/cd/E19963-01/html/821-1449/smboverview.html

    My config file is quite similar to yours:
    Is it normal that the realm of your smb.conf.master is in lower case?
    I am working with an aggregated interface so I had to add aggr* to interfaces (otherwise I got error that it couldn’t bind to interface).
    I also removed the line of winbind separator since I want to use the default (). Because we need group support.

    I will probably try with the smb package later on.

    The strange thing is that everything seems to work:
    smb -> with local accounts I can connect.
    winbind -> getent works as expected + I can logon using SSH and Domain account
    Kerberos -> I can get tickets for domainusers using kinit

    Only when I want to authenticate with a Domain user to the share it goes wrong. (Now it states: The specified network password is not correct but the same password works for SSH (using the same account ofcourse)

  7. Mieszko

    After I do all steps BEFORE chapter “Testing the configuration and starting services” I see users’ entires when using ‘getent passwd’, but they don’t persist after reboot. What’s wrong?

  8. Andrew

    Two possabilities that I can think of is either the winbind service is not started (svcs -a | grep win) or the winbind entries in nsswitch are not there (these need to be configured with the svcprop commands). Hope this helps.

    [quote]After I do all steps BEFORE chapter “Testing the configuration and starting services” I see users’ entires when using ‘getent passwd’, but they don’t persist after reboot. What’s wrong?[/quote]

  9. Andrew

    Unfortunately I haven’t played around with the Solaris samba services yet (although I have been meaning too).

    Those changes sound good. I have run it over an aggregated interface as well and the support can become important in multi-domain environments. I do it so I don’t need to always enclose everything in quotes in single domain environments but I probably should shift back at some point.

    I wish I could help you more with the samba services but I just haven’t had much time to play around with it lately.
    [quote]Thanks for the reply

    It is stil not working for me :-). But I noticed I am using different packages than you are. I use smb service instead of samba service. I had this from Oracle docs http://docs.oracle.com/cd/E19963-01/html/821-1449/smboverview.html

    My config file is quite similar to yours:
    Is it normal that the realm of your smb.conf.master is in lower case?
    I am working with an aggregated interface so I had to add aggr* to interfaces (otherwise I got error that it couldn’t bind to interface).
    I also removed the line of winbind separator since I want to use the default (). Because we need group support.

    I will probably try with the smb package later on.

    The strange thing is that everything seems to work:
    smb -> with local accounts I can connect.
    winbind -> getent works as expected + I can logon using SSH and Domain account
    Kerberos -> I can get tickets for domainusers using kinit

    Only when I want to authenticate with a Domain user to the share it goes wrong. (Now it states: The specified network password is not correct but the same password works for SSH (using the same account ofcourse)[/quote]

  10. Anbarasu

    HI ,

    I am using samba server with AD integration in that case how can i configure and set quota for AD Domain user which is used for Samba server authentication.

    Is there any graphical tool available to achieve this task.

    Thanks in advance,

    Anbarasu

  11. Anbarasu

    Hi ,

    Its really fantastic and most useful post that i was really looking for.

    Is there any way to implement quota for AD users on Solaris server and
    Solaris document suggests Quota implementation can be done on File system level either it is UFS or ZFS.Any workaround to implement Quota on Top folder levels for Samba.

    Please suggest,

    Thanks in advance,

    Anbarasu

  12. Andrew

    Hi Anbarasu,

    Sorry for the late reply. I havent tried using quotas yet but I would assume you would need to set them at the ZFS level. One problem to watch out for would be what sort of errors Samba reports once a quota is completely used.

    Sorry I cant be of more help.

    [quote]Hi ,

    Its really fantastic and most useful post that i was really looking for.

    Is there any way to implement quota for AD users on Solaris server and
    Solaris document suggests Quota implementation can be done on File system level either it is UFS or ZFS.Any workaround to implement Quota on Top folder levels for Samba.

    Please suggest,

    Thanks in advance,

    Anbarasu[/quote]

  13. Z3NF1N1TY

    Great instructions, I am having one small difficulty that I cannot get sorted out. When I try and map my share on a win7 machine I get “The specified network password is not correct”. All the services and features on the server appear to be working correctly. All you little test bits are working fine and showing all information.

    The only thing that could be wrong are the owners of the folders, they are set to root:root. Should they be changed to something else? That is how they were created by the zfs command.

  14. Andrew Hydle

    Hi Z3NF1N1TY,

    Thanks! I have root:root here on my shares and it is working ok. Specified network password is incorrect sounds like it could be an LM/NTLM compatibility issue. On older versions of samba you needed to set NTLM compatibility using local or domain security policy but I am pretty sure that this was fixed in newer versions. I would need to do some more checking though.

    [quote]Great instructions, I am having one small difficulty that I cannot get sorted out. When I try and map my share on a win7 machine I get “The specified network password is not correct”. All the services and features on the server appear to be working correctly. All you little test bits are working fine and showing all information.

    The only thing that could be wrong are the owners of the folders, they are set to root:root. Should they be changed to something else? That is how they were created by the zfs command.[/quote]

  15. Derrick

    Hi, thanks for the guide. Very useful. But I got confused after reading the Solaris 11 doc, I found that your method is quite different.

    For example, the doc uses:

    zfs create -o nbmand=on -o sharesmb=on -o casesensitivity=mixed ztank/misc
    zfs set share=name=misc,path=/ztank/misc,prot=smb ztank/misc

    to create a SMB share, which seems to be not needed in your guide. Later I finally realized you declared a smb share inside smb.conf under [Corporate] section.

    In fact to make above commands to work, install samba is not enough, also need service/file-system/smb.

    Any clarification about this?

    Cheers

  16. Derrick

    Okay, I finally figured out what is going on. Your guide is a modified version of your old one based on Solaris 10. That’s why you still used smb.conf.

    In Solaris 11, smb is in kernel driver (enabled by installing service/file-system/smb, not samba any more).

    So your method is Solaris 10’s implementation that works on Solaris 11.

  17. Andrew

    Hi Derrick,

    Yes, this is a straight samba implementation still. I have not had the chance to try Solaris’s implementation of Samba yet.

    [quote]Okay, I finally figured out what is going on. Your guide is a modified version of your old one based on Solaris 10. That’s why you still used smb.conf.

    In Solaris 11, smb is in kernel driver (enabled by installing service/file-system/smb, not samba any more).

    So your method is Solaris 10’s implementation that works on Solaris 11.[/quote]

  18. Pingback: My Homepage
  19. Pingback: click here
  20. Emil

    Hi Ahydle,

    I just stumbled to this blog and the attention to detail in your information is amazing.

    In this post you set the permissions for domain users. Do you have any acl examples when you have smb configured in “workgroup mode” and not when it is in domain mode?

    In solaris 11.2 you can add smb groups in workgroup mode but the information in what the acl entries should look like to give access to the group is none existent.

    Fantastic blog and keep up the good work.
    //Emil

    • ahydle

      Hi Emil,

      Sorry but I always tend to use samba in domain mode so I dont have any instructions to help you with that one. The one thing that I have had the most issues with though was the permissions. It seemed like Oracle would update the zfs module for samba periodically and it would completely reorder my permissions so I would need to refigure things out and sometimes rewrite permissions after updates.

Leave a Reply

Your email address will not be published. Required fields are marked *